Applanix: NOAA’s eye for hurricanes

High-resolution imagery geolocated by the sixth-generation Digital Sensor System (DSS) after Hurricane Ida. (Photo: NOAA)

Applanix, a division of Trimble, has been working with the National Oceanographic and Atmospheric Administration (NOAA) since the early 2000s to develop their response for emergency and coastal mapping activities. We discussed this collaboration with Joe Hutton, the company’s director of inertial technology, land and airborne products.

How has Applanix collaborated with NOAA regarding emergency response and coastal mapping?

Early on, we worked with them to develop a solution that allowed them to get out in the field and produce high accuracy map products with minimal touching of the data. In mid-2021, we delivered the next generation of this solution, or the DSS version six, which represents the culmination of everything learned over the years about how to produce imagery for emergency response, in terms of the types of collection, the types of imagery, and how to get it into first responders’ hands as quickly as possible.

At the heart of the system is our direct georeferencing technology. It’s a solution that allows us to assign the geographic location of every pixel of the digital imagery collected in the air. As soon as you land, you have the coordinates of every pixel, which means that you have a map that NOAA then pushes to the cloud for first responders to use in their emergency response efforts.

The collaboration consisted of Applanix working with Lead’Air to manufacture the next generation system that meets NOAA’s latest requirements. That’s what we delivered in 2021. Weeks after delivery, NOAA was called to respond to the hurricanes. They flew the new system with great success and were able to use it for their response.

What is your perspective on ground control points (GCPs) vs. direct georeferencing?

It is impossible to place GCPs in an emergency response when you cannot get on the ground. People who say they need GCPs do not really understand direct georeferencing. We’re having this debate even after 20 years of proving this technology. The NOAA system does not use GCPs and the map products are at centimeter level accuracy.

We use Trimble’s RTX technology, which enables centimeter-level GNSS positioning without base stations, which is important when the CORS or local RTN is unreliable due to a disaster. We have high accuracy inertial systems that get us the high accuracy orientation, so that we can go directly to ortho photos and ortho mosaics without running any triangulation or using GCPs in that process. That is a standard process these days. GCPs are only there for quality control if you want to deliver a final map product.

Did NOAA fly the mission with its own aircraft?

Yes, these are NOAA’s King Air or Twin Otter aircraft. The King Air aircraft is specifically outfitted for these types of emergency response and coastal mapping activities. The DSS system gets installed into the airplane and gets calibrated in terms of checking the system out for accuracy. Then it’s ready to fly the response. In the air, they collect the imagery over a flight path of interest to them. Then, it’s developed from raw imagery into JPEGs in the aircraft, and all the georeferencing data is logged with that imagery so that as soon as they land they can push a button and start to reference the JPEG imagery and push it to the cloud.

What are the components of your system?

What makes this system so unique is that it encompasses all the lessons learned over the years in terms of what NOAA needs to optimize for both their coastal mapping and their emergency response. It incorporates two pairs of color and near-infrared Phase One cameras that are configured in an oblique format with some overlap, forming a bowtie footprint on the ground.

You have 100% overlap of the color with the near-infrared and it’s on a high-performance stabilized mount that keeps everything perfectly level. The mount also has a special feature that enables the operators to rotate the cameras to go into nadir mode, mostly for traditional coastal mapping that requires stereo imagery. We were able to incorporate into a single system the requirements for both emergency response—where you want large coverage and obliqueness to look for damage—and nadir for coastal mapping.

Lead’Air built the sensor for you, on your specs, correct?

Yes, that’s correct. We’ve worked with Lead’Air for probably 20 years on flight management system (FMS) technology. They also have an amazing capability to build stabilized mounts and hardware systems. So, we decided to work together. We contracted them to implement some of their innovative hardware in this new design for us to deliver to NOAA. We contracted them to do all the manufacturing of the design and delivery to NOAA.

One of the quite innovative things that they did was to develop a new flight management capability that allows NOAA to fly ad hoc along highways or rivers, looking for damage. Traditionally, for aerial imagery you have to pre-flight plan trajectories. They designed an FMS that enables a pilot to fly a road or a river looking for damage without worrying about traditional block collections as with a more traditional FMS. So that feature further increases productivity. If you look at the most recent imagery at www.storms.ngs.noaa.gov you will see that it looks like spaghetti, not like blocks. That’s because they are following the roads and the rivers looking for specific damage.

Does the post-processing use your software?

Yes, it uses the POSPac MMS post processing software with POSPac Trimble Post-processed CenterPoint RTX correction service, allowing us to get that centimeter-level position accuracy, anywhere in the world with just an internet connection. You don’t have to worry about having a local base station—which, of course, if you’re in an emergency response situation, might not be there anyway. So, this is a very powerful way of getting global centimeter-level accuracy in real time, without having to worry about the ground-based GNSS infrastructure, that is, the local real-time network, that’s on the ground.

If you don’t have internet access, you can ship that data to the nearest place that does, right?

You could, however NOAA simply flies to wherever there is access. What takes the longest is to develop the imagery from the raw format to the JPEG format, because these are such large images. Doing that in the air saves an enormous amount of time. You have these JPEG-ready images that are compressed and can go right into the georeferencing process and make it really, really fast.

That’s a matter of computing power and smart software. What else did Lead’Air contribute?

This very efficient, fast image development process in the aircraft.

It sounds like it was a very integrated process between Applanix and Lead’Air. So, NOAA had the instrument mounted on their aircraft, their pilots did the flying, and then you processed the data?

No, NOAA’s team processes all the data. We just deliver the hardware and the software. They created the workflow software to push the data to their cloud environment.

NOAA uses this data to produce maps of the damage and highlight different situations and hazards?

Yeah. When these hurricanes go through, the first questions people have are “Where’s the damage? Are these roads passable? Did my house survive?” If you are doing response, you need to get teams in there. First, however, you need to know whether the roads are passable, so that you will not waste time going down a road that is not. So, the first thing they do is go up in the air and survey the main roads to push the imagery back, so that people can assess whether the roads are passable. Then they start to look for specific areas of damaged infrastructure, to triage where to put their resources. Then they ask “How do we manage disaster recovery?”

What lessons did you learn?

We are still learning about the power of the system, because these are Phase One 150 megapixel color cameras. It is such a powerful combination of sensors that they’re starting to look at different information they can get out of these things. They’re still learning new lessons in terms of what information can be useful for both the emergency response and the coastal mapping.

Ultimately, we’ll go to full ortho maps in the aircraft. That’s just going to be a matter of computational power. The holy grail would be to produce an orthophoto in the aircraft and radio it down to the ground in real time. Nothing prevents you from doing that now other than computational power and bandwidth. It’s not practical yet, but it will probably get there.

Do you have collaborations like the one with NOAA with any other major U.S. agencies?

We’ve worked extensively with NASA over the years. For example, we have worked with them on the ice bridge project. That is where they survey ice at both poles to measure its thickness and how global warming is affecting it. They use our system on that to do the georeferencing. We also work extensively with other branches of NOAA for their shoreline mapping from their ships. We have worked with them over the years to provide the georeferencing solution for the multibeam echo sounders to produce their nautical charts.