Testing autonomous vehicles inside and out

A test of Racelogic’s parking assistance system. (Photo: Racelogic)

Racelogic helps vehicle manufacturers develop autonomous vehicle technology and test them on indoor test tracks and the open road.

Racelogic helps vehicle manufacturers develop autonomous vehicle (AV) technology and testing houses test them. Over time, regulatory and consumer testing has evolved from indoor test tracks to outdoor open-road tests, and then to indoor controlled test environments.

“Due to their application, advanced driver-assistance systems (ADAS) originated and are still mainly developed and assessed on open-sky, controlled test tracks, tackling the most common killed and seriously injured (KSI) accident types,” said Wesley Hulshof, principal engineer – ADAS Testing at Racelogic. “These assessments usually make use of sophisticated driving robots for closed loop, centimeter-accurate path following and precise speed-controlled test-track assessments. The robots can only attain this accuracy by being fed the speed and positional data by GNSS sensors, such as the Racelogic VBOX.”

Racelogic’s VBOX GNSS receiver. (Photo: Racelogic)

Accuracy is key to conducting assessments for the European New Car Assessment Programme (Euro NCAP) and the U.S. National Highway Traffic Safety Administration (NHTSA). Using GNSS in conjunction with RTK base stations provides centimeter-level accuracy in position, said Hulshof, as well as accurate speed and heading information to measure ADAS data to both static and moving targets. Additionally, combining a GNSS receiver with an inertial measurement unit (IMU) allows for low-drift, high-accuracy speed and positioning information within areas of high GNSS multipath or temporary occlusions, such as gantries, bridges, forests or built-up areas.

However, “people do not just drive on closed test tracks with accurately positioned targets and infrastructure,” Hulshof said. “They do not drive at a constant throttle position and maintain an exact time-to-collision to the vehicle in front of them, like robots do. In fact, people often drive erratically.”

For these reasons, testing houses are conducting supplementary assessments on the open road, under real-world conditions. In these conditions it is still important to know vehicles’ positions and speeds to localize them and validate the system’s sensors, networks and algorithms.

Testing Stages

Stage I: Controlled
ADAS was developed for outdoor use because this is where car crashes occurred. For this, an open-sky GPS signal was essential for positioning. The types of tests and level of scientific rigor meant that the tests could be performed on closed test tracks.

Stage II: Randomized
Tests were brought to the open road to add elements not found within a closed environment such as traffic and higher speeds of the vehicle under test. For this, extra sensors were employed to add robustness in areas of obscured GNSS coverage.

Stage III: Controlled
Testing is brought back indoors for climate control and to assess L3/L4 AD functionalities such as valet parking.

Because open-road testing does not permit being constantly within range of a static base station, Racelogic developed a moving base solution for open-road testing that gives accurate relative positioning between two or more vehicles.

The increased demand for real-world testing of ADAS has generated demand for reliable ground truth data. “For example, if you consider a car driving on the winding roads of the Italian Alps and the position is out by 2m,” Hulshof said, “that is the difference between lovely scenery and falling off the side of a cliff. So, you need centimeter-level accuracy in the positional algorithms of the self-driving car, but also in the assessment tools, while we are testing it. For that reason, we still need GNSS and would ideally need RTK.”

To meet this demand, Hulshof said, Racelogic produced its own networked transport of RTCM via Internet protocol (NTRIP) solution, consisting of a modem and associated service provider. It allows for global coverage of high-accuracy, absolute positioning of a test vehicle in open-road conditions. Both the NTRIP and the moving base solutions allow ADAS testing to centimeter-level accuracy on the open road without the need to be in radio range of an RTK base station, thereby greatly expanding the testing possibilities.

“Whilst both the NTRIP and the moving base options allow for high-accuracy positioning,” Hulshof said, “they are still reliant on having an open sky for good GNSS coverage. IMU integration allows for improved accuracy over short periods of occlusion, but to truly give as accurate a signal as possible we need to be open to accept information from multiple satellite sources. That is why highest longevity accuracy is only achieved by using the GPS, GLONASS, Galileo and BeiDou constellations to provide the best RTK positioning performance in areas where that was not previously possible.”

To control the environment and allow for year-round testing, test laboratories such as the Insurance Institute for Highway Safety (IIHS) facility in Arizona and Asta Zero in Sweden have purpose-built covered test facilities, giving shelter from extreme heat or cold. Testing inside both set-ups, however, still relies greatly on the test vehicle positioning. Standard positioning techniques via GNSS in these situations is simply not possible. Therefore, Hulshof said, Racelogic designed the VBOX Indoor Positioning System (VIPS), which allows for seamless testing indoors or outdoors. “Because this system works as an alternative to satellites, with the in-vehicle VBOX allowing RTK-level performance without GNSS, the test vehicle can travel from open-sky outdoor testing to a closed environment seamlessly, with no drop in data during the transition or afterward.”

Finally, Hulshof said, ADAS and AD systems have moved on from straight-line highway scenarios to low speed turning scenarios often performed away from the open sky previously required for accurate GNSS coverage. Examples include multi-story parking garages and valet parking. “Scenarios such as self-parking and park-assist assessments, as well as indoor L1 ADAS, are becoming increasingly common requests by manufacturers on test facilities.”

These environmentally controlled facilities can simulate real-life conditions that affect specific sensors — such as sensor flare, fog, mist and water films. These types of facilities use VIPS to give outdoor GNSS accuracy in an indoor controlled environment. “There is a trend toward bringing the testing from closed test track to randomized real world back into a highly contained, climate-controlled area,” Hulshof said. “We then have an option for anything.”