Imaging in Mobile Mapping

Introduction

The Teledyne FLIR Integrated Imaging Solutions group has been operating for over 25 years, specializing in cameras for industrial machine vision and multi-sensor devices. This paper highlights the importance of high-resolution imaging and critical factors that make it imperative for mobile mapping.

Mobile Mapping Context

In the context of mobile mapping, the process begins with a Camera System Manufacturer who supplies the device to a System Seller. The System Seller then integrates the camera into a mobile mapping system for a Service Provider. The Service Provider plans the acquisition mission, captures, and processes the data to extract meaningful information.

The decision maker, who may never actually look at the raw data or other sensor data, is primarily interested in actionable insights. For instance, they want to know is which roads need to be repaired next or if utility poles nearing the end of their life can be identified for immediate replacement due to signs of corrosion. What they want is a recommendation and rationale for allocating next year’s limited budget while addressing a wide range of criteria.

This example illustrates how image data feeds into the entire value chain to support critical decision-making.

Factors to Consider Beyond Resolution

Based on the example we just showed, in addition to resolution, there are following other factors to consider.

Image Quality

The first factor when evaluating mobile mapping is to consider image quality. High-quality images from a camera system enable service providers to extract relevant information for the decision maker. Today, the process involves not only human operators but also computer vision software. The value of mobile mapping lies in the system’s ability to process large volumes of data and derive relevant information quickly and accurately. It is going to be computer vision software that performs the information extraction process and these algorithms are quantitative, objective, and precise. Aspects of image quality, such as wide dynamic range, allow details to be visible in both the darker and brighter portions of the scene, enabling algorithms to segment the scene into distinct parts with elevated detail. Accurate color information and texture, improves the confidence level during classification. Allowing to categorize each segment into an object category. Sharpness is also important because it enables optical character recognition algorithms to pick out alphanumeric characters on an asset detected in the scene, pull out the identifier, and tag the asset with the identifier. All these elements such as image quality, dynamic range, color accuracy, and image sharpness are incredibly important when it comes to mobile mapping.

Geometric Modeling and Spatial Accuracy

A second important consideration for the imaging system is its inherent geometric modeling and spatial accuracy. These factors are crucial to the mobile mapping system, which processes the data captured by the camera, to accurately determine the location of objects seen in an image and registering image data with other sensor data. For instance, when capturing laser scan data simultaneously with the camera images it may be desirable to apply the correct color value to each 3D point in post-processing. The geometric model, or the camera intrinsics, defines the relationship within a camera between the sensor and the lens. It is important to note that all lenses experience distortion, and it is imperative for spatial accuracy that the model accounts for this distortion.

Here is an example to illustrate the point. During an eye examination, there is an assessment of vision sharpness and the need for magnification. One aspect is astigmatism, where an object directly in front of the eye is in focus but surrounding objects may not be in focus. This occurs when the lens in the eye is not perfectly round, which causes light to bend unevenly when focusing on the retina. The prescription for lenses corrects astigmatism by ensuring all objects are in focus.

In geometric modeling, this is like calibrating a camera to consistently set its spatial accuracy. Spatial accuracy measures how closely an object's representation matches its actual location. For instance, if someone is standing ten meters away from another person and observing the button on their shirt, spatial accuracy determines the exactness of the button’s XY position within a few centimeters or millimeters. Laser scanned data is often required to be millimeter accuracy from meters away. Therefore, consistent spatial accuracy is essential for mobile mapping systems equipped with both a camera and a laser scanner.

Timing

Timing is another key factor in mobile mapping, referring to the timing of trigger decisions usually initiated by an external source. For example, operating the imaging system in free-running mode captures a fixed number of frames per second, suitable for applications like video that require a specific frame rate for smooth playback. However, mobile mapping requires data capture with optimized coverage along the transportation corridor, ensuring a specific distance between frames for overlapping images. This approach minimizes storage consumption and maximizes processing resources utility and, therefore, the time needed to complete the project. Using an external device, such as a Distance Measuring Instrument (DMI) that indicates the distance travelled via digital pulses, is useful for initiating triggers. This method helps optimize data capture for mission coverage while minimizing storage consumption and post-processing time.

Timing is also important for synchronizing because there are typically other key instruments in a mobile mapping system, such as GPS or positioning system, which provides both a timestamp and current location. Knowing the exact location ties in with the spatial accuracy of the color information captured at that location. It is not unusual to have additional sensors such as a thermal imaging camera or laser scanner that make up a mobile mapping system. Knowing the time and location when triggering takes place allows for geospatial data registration. For example, one can now use the 3D coordinates of a point from the laser scan point cloud and look up the corresponding color value captured by the camera at that same instance in time.

Evaluating Mobile Mapping Equipment

Purchasing mobile mapping equipment is a substantial investment, one that needs to operate reliably in the field consistently. Here are additional aspects to consider when evaluating a system for capturing high-resolution imagery:

How long has the manufacturer been in the industry? What reputation do they have with others in the ecosystem? If you buy multiple systems, are they consistent in both quality and performance?
How well designed is the product for the rigors of mobile mapping? Varying weather conditions in temperature, humidity, shock & vibration can affect operations. Are they field proven?
Does the manufacturer stand behind their product with both multi-year warranty and responsive, knowledgeable support?

These considerations are essential for maximizing one's investment in a mobile mapping system.

Advancements in Processing Power

Currently, we are experiencing exponential advancements in processing power, with the cost per unit of processing continually decreasing over time. This advancement has facilitated the growth in higher resolution senor data acquisition and expansion of mobile mapping applications. Almost two decades ago, Google introduced Street View. Today, mobile mapping includes scene segmentation for engineering surveys, asset management, pavement condition assessment, and urban forest management.

Also, advances in processing power have significantly enhanced the data analysis and information extraction portion of the value chain. Graphics Processing Units and Neural Processing Units are giving rise to algorithms that process large volumes of data with increased output and confidence levels in the classification of objects using AI and machine learning techniques. These advances are impressive, and we are only at the beginning.

As advanced as these downstream methodologies are in producing results, it is critical to recognize that acquiring high-resolution image data at the utmost quality, with a high degree of spatial accuracy and precise trigger timing, enables optimal results for critical business decision-making.

Conclusion

Advancements in imaging technology for mobile mapping have significantly improved data capture quality. High resolution, superior image quality, spatial accuracy, and precise timing are essential for mobile mapping and critical business decisions. These enhancements help extract meaningful information from collected data, leading to better decisions and optimized resource allocation. Leveraging advances in processing power and combining with sensor fusion provides a comprehensive understanding of the environment, driving further benefits from mobile mapping.