Driving Blind - with Assistance

July 1, 2008 By: Xavier Leroi, Francis Martinez, Aubert Carrel, Alain Lefaucheux GPS World

6D Positioning and Augmented Reality

The driver of an armored vehicle such as a tank often operates without direct view of the outside, and must incorporate external visual information through episcopes while also monitoring the vehicle's status on a dashboard. Military field vehicles of all kinds increasingly use a range of positioning sensors such as GPS and inertial measurement units (IMUs) for navigation. They also employ various perception sensors for observing the environment and driving. In some cases the direct optical channel (episcopes) are sufficient, but at night or under other visually obscured operating conditions, light-intensification devices and infrared cameras significantly increase the potential observable data. Technological advances will add further multisensors for an all-conditions vision capability. All these methods are characterized by the need to display the images to the operators through interfaces such as computer screens.

The French Defence Procurement Agency (DGA) has for several years explored advances in vetronics (onboard electronics, networks, and software) to improve the work environment for crews of such vehicles. Each crew member may have one or more computer-operated screens, connected to the vehicle's data networks — video (day and night), tactical information, geographic databases, vehicle status — affording each one access to all the information circulating on these networks.

The PISE mobile testbed

This multiple access can facilitate task redistribution for better efficiency and mobility in the field. For example, the driver may take over navigation preparation from the commander, based on orders received. The commander can then validate the itinerary before, while, or after it is done, by displaying the planned and achieved trajectories on a digital map. The driver can access the commander's observation inputs while the vehicle is stationary, if the latter is occupied with other tasks such as communication.

This article describes implementation of such principles in the DGA's Plateforme d'Intégration Système-Equipage (PISE, or System-Crew Integration Platform) demonstration testbed. It describes how the physical objects of the geographic and tactical databases of the operational site can be displayed in the driver's main screen, with an augmented-reality enhancement. We have developed advanced positioning and rendering techniques integrating GPS and inertial sensors for the 6D real-time vehicle positioning, and on 3D image synthesis.

This technique can also find application in non-military sectors, wherever the driver or pilot of a vehicle may be forced to operate under less-than-optimal visibility: in the dark when headlights are insufficient; in a snowstorm, fog, or driving rain; or on roads, in harbors, or other areas. Camera images mixed with 3D objects from a geographical information systems (GIS) database, projected either on a computer screen or directly onto the windshield of the vehicle, can provide assistance.