GIS Gives Simulators More Realism

IITSEC 2008.

What is a GIS guy doing at a training simulation conference? Having a lot of fun — and learning that GIS makes everything better.

By Art Kalinski, GISP

Last week I attended the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) in Orlando, Florida. It was an international gathering of more than 16,000 attendees, both military and civilian, that featured some 550 exhibits focusing primarily on military-related training and simulation. I know that Geospatial Solutions is a GIS-focused publication and that most of you are GIS professionals, but this conference clearly demonstrated the merging of many disciplines in formerly unrelated communities, including GIS.

When we hear the word “simulators,” we generally think of flight simulators — and there were many of those on display — but the exhibits also included ground combat, medical, logistic, and cultural immersion simulators. This conference demonstrated the most realistic and up-to-date training and simulation tools available in the world today, and GIS played a fundamental role as the underpinning of many applications. Autodesk, ESRI, and 3D model builders Precision Lightworks all had booths at the four-day event.

One interesting terminology quirk that I noticed is the difference in usage of the term “database.” We GIS folk generally use the word to describe our vector data and attributes, with imagery as separate layer. But in the training environment, the term is used to describe all the datasets that feed the simulators, including maps, imagery, GIS data, DEMs, CAD drawings, photos, objects, characters, TINS, wireframes, draping textures, skins, etc. The overlap with the video gaming community is significant, but GIS is growing in importance as many applications strive for real-world connectivity.

Experiencing the Exhibit Hall

Some of the exhibits showed a significant investment, not just in the technology but in the displays themselves. It was apparent that training and simulation is a big business, with lots of dollars at stake. One exhibitor summed it up this way: “With aircraft and ships costing tens and hundreds of millions each, simulators are a low-cost way of building competence without risking expensive equipment.” The same holds true for soldiers in the field. Actual combat is not the place to conduct training, and everyone is striving to create training that is so realistic that it is indiscernible from actual combat.

Many exhibitors showcased new display technology, such as very high-resolution projectors from Sony. Most simulators used large flat screens, tripled in some cases to provide panoramic views. One flight simulator used nine separate screens, but even more impressive were large concave screens similar to IMAX. that provided realistic and seamless images, even in 3D.

Big screen display technology.

Some trainers used head-mounted displays (HMDs) for combat action scenarios, but an unusual addition was a 10-foot spherical cage for humans that resembled a giant hamster ball. The subject enters the cage wearing a wireless HMD, and can move in any direction responding to the HMD display and program. The cage was so well designed that it moved with just light finger pressure. Users indicated that getting used to walking or even running in the cage became natural quite rapidly.

Users indicated that walking in the cage — or even running — became natural quite rapidly.

Some of the exhibits showed significant expense, not just the technology but the displays themselves.  The Augusta Westland helo trainer was one example.

The Augusta Westland helo trainer.

It was apparent that training and simulation is a big business with lots of dollars at stake. One exhibitor summed it up stating, “With aircraft and ships costing tens and hundreds of millions each, simulators are a low cost way of building competence without risking expensive equipment.” The same holds true for soldiers in the field.  Actual combat is not the place to conduct training and everyone is striving to create training that is so realistic that it is indiscernible from actual combat

I expected to see more simulators that imitated the movement of flight, but as one of the exhibitors told me, the motion and G forces of jet fighters are so great that simulators can’t even come close — so they replicate other aspects of the experience instead. Two motion simulators that were effective included a helicopter flight and gunner simulator, and a Hummer simulator that recreated the motion of a vehicle struck by an improvised explosive device (IED). The shock and rollover over of the vehicle trained soldiers how exit a turned over vehicle while under fire, the goal being to simulate and practice what could happen so there were no surprise situations.

Vehicle rollover trainer.

Lockheed Martin featured a shipboard engine-room simulator that enabled the training of engineering personnel by simulating various casualties and damage control actions, without risk to people or equipment. The operation was very similar to a video game, with realistic characters responding to the commands of the trainee. As a former Navy Surface Warfare Officer I had to try my hand at the Kongsberg ship bridge simulator. I can attest that not only did the simulator look good, the physics, behavior and reaction of the ship was dead on accurate.

As a former U.S. Navy surface warfare officer, I had to try my hand at the Kongsberg ship bridge simulator. I can attest that not only did the simulator look good, but the physics, behavior, and reaction of the ship were dead-on.

There were numerous small arms trainers and ranges on display that were capable of training individuals, multiple trainees, and even groups acting as teams within a combat environment. I also saw countless examples of wearable gear that permitted simulation of small arms combat using of laser weapons to simulate hits. Numerous urban building mock-up trainers incorporated video displays, realistic sounds, and even explosions and smoke.

Weapons team training.

Medical simulation included traumatic injuries and triage scenarios played via video screens. Sickeningly realistic moulage kits imitated all types of injuries, complete with exposed bones and spurting blood. There were many exhibits of peripheral equipment and supplies such a carrying cases, communication gear and even very high quality surface coated mirrors for projectors and displays. One little gem that I hands-on tested at the ESRI booth was a 3D mouse by 3D Connexion.  The “Space Navigator” was the smoothest, most intuitive, most precise, quality feeling mouse I’ve ever used.  I’m getting one.

3D mouse.

An area I found fascinating — because I didn’t expect to see it — featured exhibitors that focused on human simulation and cultural immersion. DI-Guy and MetaVR were displaying human simulations of crowds that looked realistic and could be programmed to behave in various ways, with each character acting as both an individual and part of the group. The photo shows a Mideast group walking toward an infantry squad while hiding two terrorists in their midst.  Some of the video characters are a little blocky, but new techniques derived from the video gaming industry include draping “skins” on mesh frames instead of boxes or tins. The end result is smoother-looking characters and objects, such as flowing robes.

Computer generated crowd.

One exhibitor that displayed human motion capture tools used by Hollywood to create animations. They had a young dancer wearing a capture suit whose motion was transferred to a character on a video display. She won the unofficial endurance record since she danced continuously during the four-day exhibit hall operation.

Human movement capture.

Another exhibitor, the Tatitlek Corporation, takes cultural immersion to the extreme with live, culturally correct role players that recreate Middle Eastern environments. The live players create the look, sounds, and feel of being in the region, so that it’s not a shock for our troops when they arrive in-theater. Tatitlek displayed one example of their cultural simulations: a sheik’s tent complete with rugs, seating, accessories, and even staged arguments and police intervention.

Live cultural immersion.

Enhancing Reality with GIS and Imagery

So how does GIS fit into this community? As I learned from people that have been in this business for years, training simulation first evolved from physical mock-ups to computer-based, CAD-like “cartoon” environments that were only loosely tied to reality. It was considered a challenge to get an image up on the screen that represented reality and make it move around in response to the trainee, but that’s no longer good enough. Now imagery and GIS-referenced data is enhancing the entire environment.

Lockheed Martin was a big presence at the conference, and created a buzz with flight simulation systems that incorporated photo-realistic and photo-accurate oblique imagery and 3D models into geo-referenced applications that simulate weather conditions and even provide night-time views. This kind of accurate simulation achieves realism not previously seen, and now has captured the attention of mission planners, who have significantly different needs from trainers. The mission planners, as their title implies, plan real missions just before actual execution. For their work they need up-to-date information that is accurate, geo-referenced, and measurable. They cannot tolerate made-up data or imagery — it may be perfectly fine for training, but could be disastrous if relied upon for real execution. The bar has been raised.

Attending I/ITSEC, I could see the growing convergence of GIS, CAD, BIM models (see my August column for the BIM – GIS convergence), computer display technology, and the training/simulation applications. It’s amazing to watch how everything is coming together. An especially interesting system displayed by the Navy was a flight simulator that collected brain wave data and recorded images of the pilot’s eyeball movements.

Human factors flight simulator.

As I viewed this rig I couldn’t help but recall a photo I saw at Moody Air Force Base of an early flight trainer crafted from plywood and broom handles. Imagine how alien this new Navy system would look to the designer of that plywood trainer. I wonder where we will be in 20 more years.