The Kinematic GPS Challenge: First Gravity Comparison Results

The National Geodetic Survey (NGS) has issued a “Kinematic GPS Challenge” to the community in support of NGS’ airborne gravity data collection program, called Gravity for the Redefinition of the American Vertical Datum (GRAV-D). The “Challenge” is meant to provide a unique benchmarking opportunity for the kinematic GPS community by making available two flights of data from GRAV-D’s airborne program for their processing. By comparing the gravity products that are derived from a wide variety of kinematic GPS processing products, a unique quality assessment is possible.

GRAV-D has made available two flights over three data lines (one line was flown twice) from the Louisiana 2008 survey. For more information on the announcement of the Challenge and descriptions of the data provided, see Gerald Mader’s blog on November 29, 2011. The GRAV-D program routinely operates at long-baselines (up to 600 km), high altitudes (20,000 to 35,000 ft), and high speeds (up to 280 knots), a challenging data set from a GPS perspective. As of December 2011, ten groups of kinematic GPS processors have provided a total of sixteen position solutions for each flight. At two data lines per flight, this yielded 64 total position solutions. Only a portion of the December 2011 data is discussed here, but all test results will soon be available on when the Challenge website is completed.

Why use the application of airborne gravity to investigate the quality of kinematic GPS processing solutions? Because the gravity measurement itself is an acceleration, which is being recorded with a sensor on a moving platform, inside a moving aircraft, in a rotating reference frame (the Earth). The gravity results are completely reliant on our ability to calculate the motion of the aircraft— position, velocity, and acceleration. These values are used in several corrections that must be applied to the raw gravimeter measurement in order to recover a gravity value (Table 1). The corrections in Table 1 are simplified to assume that the GPS antenna and gravimeter sensor are co-located horizontally and offset vertically by a constant, known distance.

Table 1. GPS-Derived Values that are used in the Calculation of Free-Air Gravity Disturbances

All Challenge solutions are presented anonymously here, with f## designations. For each flight of data, the software that made the f01 solution is the same as for f16, f02 the same as f17, and so on.

Test #1: Are the solutions precise and accurate?

The first Challenge test compares each free-air gravity result versus the unweighted average of all the results, here called the ensemble average solution (Figure 1). This comparison highlights any GPS solutions whose gravity result is significantly different from the others, and will group together solutions that are similar to each other (precise). Precision is easy to test this way, but in order to tell which gravity results are accurate calculations of the gravity field, a “truth” solution is necessary. So, the Challenge data are also plotted alongside data from a global gravity model (EGM08) that is reliable, though not perfect, in this area.

Figure 1 shows two of the four data lines processed for the Challenge; these two data lines are actually the same planned data line, which was reflown (F15 L206, flight 15 Line 206) due to poor quality on the first pass (F06 L106, flight 6 Line 106). The 5-10 mGal amplitude spikes of medium frequency along L106 are due to turbulence experienced by the aircraft, turbulence that the GPS and gravity processing could not remove from the gravity signal.

Figure 1.

Figure 2.

Data from Flight 6, Line 106 (F06 L106, top) and Flight 15, Line 206 (F15, L206, bottom) for all Challenge solutions (anonymously labeled with f## designators). Figures 1 and 2. Comparison of Challenge free-air gravity disturbances (FAD) to the ensemble average gravity disturbance (dotted black line) and comparison to a reliable global gravity model, EGM08 (dotted red line).

Figure 3.

Figure 4.

Figures 3 and 4. Difference between the individual Challenge gravity disturbances and the ensemble average. The thin black lines mark the 2-standard deviation levels for the differences. For F15 L206, one solution (f23) was removed from the difference plot and statistics because it was an outlier. For both lines, the ensemble’s difference with EGM08 is not plotted because it is too large to fit easily on the plot.

The results of test #1 are surprising in several ways:

• The data using the PPP technique (precise point positioning, which uses no base station data) and the data using the differential technique (which uses base stations) produce equivalent gravity data results, where any differences between the methods are virtually indistinguishable.
• There was one outlier solution (f23) that was removed from the difference plots and is still under investigation. Also, on F15 L206, solution f28 had an unusually large difference from the average though it performed predictably on the other lines. Of the remaining solutions, four solutions stand out as the most different from all the others: f03/f18, f04/f19, f05/f20, and f07/f22.
• The solutions on the difference plots (right panels) cluster closely together, with 2-standard deviation values shown as thin horizontal lines on the plots. The Challenge solutions meet the precision requirements for the GRAV-D program: +/- 1 mGal for 2-standard deviations.
• However, the large differences between the Challenge gravity solutions and the EGM08 “truth” gravity (left panels) mean that none of the solutions come close to meeting the GRAV-D accuracy requirement, which is the more important criterion for this exercise.

Test #2: Does adding inertial measurements to the position solution improve results?

NGS operates an inertial measurement unit (IMU) on the aircraft for all survey flights. The IMU records the aircraft’s orientation (pitch, roll, yaw, and heading). Including the orientation information in the calculation of the position solution should yield a better position solution than GPS-only calculations, but it was not expected to be significantly better. Figure 2 shows the NGS best loosely-coupled GPS/IMU free-air gravity result versus the Challenge GPS-only results and Table 2 shows the related statistics.

Figure 5.

Figure 6.

Figures 5 and 6. F06 L105. (Figure 5) Comparison of Challenge FAD gravity solutions (ensemble=black dotted line) with EGM08 (red dotted line); (Figure 6) comparison of Challenge gravity solutions (all GPS-only; ensemble=black dotted line) with NGS’ coupled GPS/IMU gravity solution (red dotted line).

Table 2. Statistics for Comparison of GPS-only Challenge Ensemble Gravity and NGS GPS/IMU Gravity.

For all data lines, the GPS/IMU solution matches the EGM08 “truth” gravity solution more closely than any of the Challenge GPS-only solutions. In fact, the more motion that is experienced by the aircraft, the more that adding IMU information improves the solution. One conclusion from this test is that IMU data coupled with GPS data is a requirement, not optional, in order to obtain the best free-air gravity solutions.

Additional Testing and Future Research

Other testing has already been completed on the Challenge data and the results will be available on the Challenge website soon. Important results are:

• Two Challenge participants’ solutions perform better than the rest, two perform worse, and one is a low quality outlier. The reasons for these differences are still under investigation.
• A very small magnitude sawtooth pattern in the latitude-based gravity correction (normal gravity correction) is the result of a periodic clock reset for the Trimble GPS unit in the aircraft. This clock reset is uncorrected in the majority of Challenge solutions. The clock reset causes an instantaneous small change in apparent position, which results in a 1-2 mGal magnitude unreal spike in the gravity tilt correction at each epoch with a clock reset.
• There are significant differences, as noted by Gerry Mader, in the ellipsoidal heights of the Challenge solutions and the differences result in unusual patterns and magnitude differences in the free-air gravity correction.

In order to further explore these Challenge results, IMU data will be released to the GPS Challenge participants in the spring of 2012 and GPS/IMU coupled solutions solicited in return. Additionally, basic information about the Challenge participants’ software and calculation methodologies will be collected and will form the basis of the benchmarking study.

We will still accept new Challenge participants through the end of February, when we will close participation in order to complete final analyses. Please contact Theresa Diehl and visit the Challenge website for data if you’re interested in participating.

Brave New World of Data via the Cloud

The frightening thing about the Mobile World Congress in Barcelona, the bloody awful frightening thing is the sheer amount of data talked about, enthusiastically envisioned, planned for. Planned for in the sense of throwing up business cases and wheeling and dealing new products and services for millions and billions of users that will pump vast amounts of data, countless numbers of gigabytes, terabytes, petabytes, exabytes per second through the cloud.

Not planned for in the sense of actually making provision for.  Seeing if there's enough resource on hand. Calculating if the ecosystem will handle it.

No, wireless carriers and everyone else involved in this industry make money on data. So let's make, make, make, more, more, more.

Did anyone happen to estimate the amount of bandwidth needed to upload and download all this data? Has anyone thought about what pressure it might bring on other spectrum users such as, perhaps, GNSS?

My guess is no, and no, and we don't care. Because we are creating the future, don't you see?!!?

From this brave new world sprang LightSquared, born of the ravenous need for more wireless data. It doesn't take much time at the Mobile World Congress to see that venture as just the first very tentative probe. Armies are massed at our borders.

I didn't get to location as a blue-chip commodity, as promised yesterday. That will have to come tomorrow.

____________________________

Sleep was what I wanted, you know what I got. 
Wide awake, staying up late, wishing I was not.

Our Man in Barcelona

Smartphones are taking over the world, and not just modern industrialized societies. A Broadcom executive predicted today at the Mobile World Congress in Barcelona that, with costs going down for less expensive models, smartphones will not only be the first phone of any kind for many people in India and other developing nations, it will constitute their first Internet experience.

There's a whole lot of change coming for North America and European users, too, and much of that is being envisioned, enthusiastically promulgated, and occasionally even demonstrated at this global village of 60,000 modcom movers and shakers that congregate here every year.  Just a few examples:

• granting access to one's location data for only a set period, from 15 minutes to 4 hours, via Glympse.
• location-based display advertising, not just coupons, but glossy little ads on your screen, called up by proximity to the advertiser, via Sofialys.
• centimeter-accurate indoor navigation, to the product on the shelf and not to its competitor product next to it on the same shelf, via Wi-Fi and near-field communication (NFC), Broadcom again but others including LocAid are talking about it too.
• An alarm clock function on your phone that will wake you (or let you sleep) at exactly the right time for that morning, based on real-time traffic and weather conditions on your commute route, from Airbiquity.

All this with either a few deft touches of the smartphone screen, or automatically enabled.

And this is just the location aspect of smartphones, which represents maybe 5 percent of what's being talked about here.  Tons of other apps for health and entertainment and more.

Tomorrow: location as a blue-chip commodity.

Inside the Head of the Body Politic

In the exciting run-up to Election '12, we conducted a straw poll of selected voters, giving everyone a chance to see what the electorate thinks about the state of things, and its outlook on the future. This is y'all talking, now: a barely scientific subset of the GPS/GNSS community, the audience at last week’s webinar, “The Challenges of Global Navigation.” The poll results are hardly surprising, but illuminating nonetheless.

Question One. The greatest challenge to realizing new technical capabilities is:

A.   staying ahead of the competition.  4.3% voted for this one.
B.   funding.  34%
C.   meeting expectations of the consumer (user).  34%
D.   establishing standards.  8.5%
E.   overcoming opposition (policy, privacy, regulations, etc..).   19.1%

Few surprises here. The biggest problems are always getting hands on the money to make a product, and then getting someone to buy the product.  The latter, of course, by making the product enough of a value proposition for the discerning prospect to buy.

Question Two. The predominate source of technical vision/innovation is:
A.     Governments.   1.7%
B.     Industry on its own.   53.3%
C.     Industry responding to government requirements.   28.3%
D.     Academia.   16.7%

Most of you out there believe you know what you are doing and are best left to yourselves to do it. Good on ya.

By the way, all the questions here were devised by Doug Taggart, president of Overlook Systems Technologies, Inc., and moderator of the plenary session at the Institute of Navigation’s (ION’s) International Technical Meeting. The ION ITM plenary took place three hours before our webinar, and audience members voted on these same questions. We then adjourned to a hotel room at the conference site and essentially re-presented a portion of the webinar content, interspersed with the polling questions.

The full 60-minute webinar, with presentations by Jules McNeff, VP Strategy and Programs, Overlook Systems Technologies, Inc., and Chuck Schue, president and CEO of UrsaNav, is available for download and replay at www.gpsworld.com/webinar (scroll down).

Question Three. Successful innovation is most dependent on:
A.     technology revolution.   11.5%
B.     technology evolution.   39.3%
C.     market demand.   34.4%
D.      project management.   6.6%
E.      funding.   8.2%

The free-market Keynesians out there are exceeded (in numbers) only by the techno visionaries, who believe that technology itself is a live organism, evolving and developing under its own impetus (perhaps aided or driven in part by market demand). Unless I’m putting words into someone’s mouth.

Question Four. Should innovative military capabilities be made available for civil/commercial exploitation?
A.      Yes, always.  The commercial spin-off value is far greater.  31.3%
B.      Sometimes.  When military capability is not compromised.   68.7%
C.      No.  Military capabilities are for military use only.  Every advantage must be protected.   0%

“Sometimes” is always a safe answer. But a coalition of free-marketers and techno visionaries made a surprisingly strong showing, garnering nearly one-third of the votes on an unequivocal up-down issue. This pushback should not be ignored by those in power.

Question Five. GPS will continue to be the world’s space-based PNT “Gold Standard”:
A.    for the next 20 years.   50%
B.    until Europe’s Galileo system is declared operational.   20.8%
C.    until China’s Compass system is declared operational.   14.6%
D.    until Glonass incorporates L1C.   8.3%
E.    it is not the Gold Standard today.   6.2%

At first glance, one might find few worries here for those who design new products with GPS uppermost or even solely in mind. On the other hand, if you combine the four non-GPS gold standard answers, you get a separate but equal body politic. 

Mind you, the other 50% are not saying that any other system will surpass GPS and become a new gold standard. The question does not ask that. But it does leave the door open for anyone to conclude that there may not be a gold standard at all at some point in the future — that all or at least a plurality of systems will be equally capable, or that an interoperable, interchangeable GNSS will surpass any single system component.

Question Six. From a user perspective, what is the most concerning aspect of having access to PNT information derived from GNSS?
A.    It is susceptible to interference.   58%
B.    Without augmentation, it does not meet my needs.   26%
C.    It is overshadowing the need for complementary systems that do not have similar shortcomings.   8%
D.    No concerns.   8%

Interference is on nearly everyone’s mind. In fact, those who voted the B or C ticket can also be inferred to be driven by interference concerns, they are just taking their concern a step further by envisioning a solution. Chuck Shue’s webinar presentation (see above link) on e-Loran should be of interest to everyone here except the bottom 8.

Question Seven. Regarding GNSS systems, which is more important to design and field first?
A.      The Space segment (satellites).   21.4%
B.      The Ground Control Segment.   23.2%
C.      The User Equipment.   1.8%
D.      All are equally important, and should be fielded simultaneously.   53.6%

I feel this result is of little use to anyone except the U.S. Air Force, the European Space Agency, Roscosmos, and the China National Space Administration. And I’m pretty sure they all knew it already.

Question Eight. How does a country gain and maintain GNSS superiority?
A.      Create technological advantage (better mouse trap).   25%
B.      Create political/policy advantage (better playing field).   11.5%
C.      Create fiscal advantage (better funding).   36.5%
D.      Create public/private partnerships (better risk mitigation).   26.9%

A majority, but not a thumping one, opts for money.  Another safe vote in almost any circumstance.

David Last, another panel speaker at the morning’s plenary, made a cogent comment when this question was presented. He could understand, he said, how a country might want to gain and maintain military superiority. That’s a question of survival. But GNSS superiority? In this age of interoperability, surely that’s beside the point.

Well, we’ve tossed our chaff into the wind to see which way it blows. Now we must all put our heads down and our shoulders to the wheel, pushing on to Election ’12, coming up  November 4. 

But there’s an earlier Election ’12 that takes place September 20: the return showdown between the Satellite Party and the Signal Party. The Reds and the Blues. They last contested, you may or may not remember, in the previous election year, 2008; Put to a Vote, GPS World’s Leadership Dinner — held during ION-GNSS 2008 in Savannah, Georgia — convoked a lively debate: Would the community gain more from new signals, or from more satellites? A made-up scenario that elicited important insights.

The Satellite Party has been in power since its ’08 victory. Are you better off now than you were four years ago? We will return to the hustings in Nashville during ION-GNSS, as again GPS World hosts GNSS Election ’12.

Given the current tenor of debates around the country and around the world, I have a feeling we’ll be hearing from the Occupy GPS movement as well as the two frontrunners.

A Comparison of Lidar and Camera-Based Lane Detection Systems

Nearly half of all highway fatalities occur from unintended lane departures, which comprise approximately 20,000 deaths annually in the United States.  Studies have shown great promise in reducing unintended lane departures by alerting the driver when they are drifting out of the lane. At the core of these systems is a lane detection method typically based around the use of a vision sensor, such as a lidar (light detection and ranging) or a camera, which attempts to detect the lane markings and determine the position of the vehicle in the lane. Lidar-based lane detection attempts to detect the lane markings based on an increase in reflectivity of the lane markings when compared to the road surface reflectivity. Cameras, however, attempt to detect lane markings by detecting the edges of the lane markings in the image. This project seeks to compare two different lane detection techniques-one using a lidar and the other using a camera. Specifically, this project will analyze the two sensors’ ability to detect lane markings in varying weather scenarios, assess which sensor is best suited for lane detection, and determine scenarios where a camera or a lidar is better suited so that some optimal blending of the two sensors can improve the estimate of the position of the vehicle over a single sensor.

Lidar-based lane detection

The specific lidar-based lane detection algorithm for this project is based on fitting an ideal lane model to actual road data, where the ideal lane model is updated with each lidar scan to reflect the current road conditions. Ideally, a lane takes on a profile similar to the 100-averaged lidar reflectivity scans seen in Figure 1 with the corresponding segment.

Figure 1. Lidar reflectivity scan with corresponding lane markings.

Note that this profile has a relatively constant area bordered by peaks in the data, where the peaks represent the lane markings and the constant area represents the surface of the road.  An ideal lane model is generated with each lidar scan to mimic this averaged data, where averaging the reflectivity directly in front of the vehicle generates the constant portion and increasing the average road surface reflectivity by 75 percent mimics the lane markings.  This model is then stretched over a range of some minimum expected lane width to some maximum expected lane width, and the minimum RMSE between the ideal lane and the lidar data is assumed to be the area where the lane resides. For additional information on this method, see Britt, Rose & Levy, September 2011.

Camera-based lane detection

The camera-based method for this project was built in-house and uses line extraction techniques from the image to detect lane markings and calculate a lateral distance from a second-order polynomial model for the lane marking in image space. A threshold is chosen from the histogram of the image to compensate for differences in lighting, weather, or other non-ideal scenarios for extracting the lane markings. The thresholding operation converts the image into a binary image, which is followed by Canny edge detection. The Hough transform is then used to extract the lines from the image, fill in holes in the lane marking edges, and exclude erroneous edges. Using the slope of the lines, the lines are divided into left or right lane markings. Two criteria based on the assumption that the lane markings do not move significantly within the image from frame to frame are used to further exclude non-lane marking lines in the image. The first test checks that the slope of the line is within a threshold of the slope of the near region of the last frame’s second-order polynomial model. The second test uses boundary lines from the last frame’s second-order polynomial to exclude lines that are not near the current estimate of the polynomial. second-order polynomial interpolation is used on the selected lines’ midpoint and endpoints to determine the coefficients of the polynomial model, and a Kalman filter is used to filter the model to decrease the effect of erroneous polynomial coefficient estimates. Finally, the lateral distance is calculated using the polynomial model on the lowest measurable row of the image (for greater resolution) and a real-distance-to-pixel factor. For more information on this camera-based method, see Britt, et al.

Figure 2. Camera-based lane detection (green-detected lanes,blue-extracted lane lines, red-rejected lines).

Testing

Testing was performed at the NCAT (National Center for Asphalt Technology) in Opelika, Alabama, as seen in Figure 3.  This test track is very representative of highway driving and consists of two lanes bordered by solid lane markings and divided by dashed lane markings.  The 1.7-mile track is divided into 200-foot segments of differing types of asphalt with some areas of missing lane markings and other areas where the lanes are additionally divided by patches of different types and colors of asphalt.

Figure 3. NCAT Test Facility in Opelika, Alabama.

A precision survey of each lane marking of the test track as well as precise vehicle positions using RTK GPS were used in order to have a highly accurate measurement of the ability of the lidar and camera to determine the position of the vehicle in the lane. Testing occurred only on the straights, and the performance was analyzed on the ability of the lidar and camera to determine the position of the lane using metrics of mean absolute error (MAE), mean square error (MSE), standard deviation of error (σ­_error), and detection rate. The specific scenarios analyzed included varying speeds, varying lighting conditions (noon and dusk/ dawn), rain, and oncoming traffic. Table 1 summarizes the results for these scenarios. For additional results, please see [8].

Table 1. Lidar and camera results for various environments.

Additional testing on the effects of oncoming traffic at night was examined by parking a vehicle on the test track at a known location with the headlights on. Figure 4 shows the lateral error with respect to closing distance where a positive closing distance indicates driving at the parked vehicle, and a negative closing distance indicates driving away from the vehicle. Note that the camera does not report a solution at -200 m, which is due to track conditions and not the parked vehicle.

Figure 4. Error vs. Closing Distance.

Based on these findings it would appear that the camera provided slightly more accurate measurements than the lidar while having a decrease in detection rate. Additionally the camera performed well in the rain where the lidar experienced decreased detection rates.

References

Frank S. Barickman. Lane departure warning system research and test development. Transportation Research Center Inc., (07-0495), 2007.

J. Kibbel, W. Justus, and K. Furstenberg. using multilayer laserscanner. In Proc. Lane estimation and departure warning Proc. IEEE Intelligent Transportation Systems, pages 607 611, September 13 15, 2005.

P. Lindner, E. Richter, G. Wanielik, K. Takagi, and A. Isogai. Multi-channel lidar processing for lane detection and estimation. In Proc. 12th International IEEE Conference on Intelligent Transportation Systems ITSC '09, pages 1 6, October 4 7, 2009.

K. Dietmayer, N. Kämpchen, K. Fürstenberg, J. Kibbel, W. Justus, and R. Schulz. Advanced Microsystems for Automotive Applications 2005. Heidelberg, 2005.

C. R. Jung and C. R. Kelber, “A lane departure warning system based on a linear-parabolic lane model,” in Proc. IEEE Intelligent Vehicles Symp, 2004, pp. 891–895.

C. Jung and C. Kelber, “A lane departure warning system using lateral offset with uncalibrated camera,” in Intelligent Transportation Systems, 2005. Proceedings. 2005 IEEE, sept. 2005, pp. 102 – 107.

A. Takahashi and Y. Ninomiya, “Model-based lane recognition,” in Proc. IEEE Intelligent Vehicles Symp., 1996, pp. 201–206.

Jordan Britt, C. Rose, & D. Bevly, "A Comparative Study of Lidar and Camera-based Lane Departure Warning Systems," Proceedings of ION GNSS 2011, Portland, OR, September 2011.