Expert Advice: What You Need to Know about Testing

May 1, 2012  - By

Five experts share what original equipment manufacturers need to know about testing their GNSS devices during product development.


System Health

Headshot: John Pottle

John Pottle, Spirent Positioning Technology

Most people are aware that simulation forms a key part of GPS receiver development and testing.

However, simulators are also used as critical tools in other areas, from the development of a new GNSS system to testing system problems and effects of interference.

From the beginning of the Galileo program, simulators have been used to enable development of the ground segment monitoring receivers. These Ground Sensor Stations continuously monitor the performance of the Galileo satellites and provide information to the Galileo Control Centre in Fucino, Italy, from where correction messages are generated.

Galileo RF Constellation Simulators were also used for research and development testing of the initial user segment receivers for the Galileo system. These included not only the Open Service receivers but also development of the initial Public Regulated Service receivers that include the encryption algorithms.

Similarly, simulators have been used for many years to test receivers that actually fly in space, including on the GPS satellites themselves as well as missions like the Space Shuttle.

When GPS has a problem, the industry oftentimes relies on simulators to recreate the problem in the laboratory to help understand the issues and find fixes.

For example, when SVN-49 satellite issues were first noticed in April 2009, simulator scenarios were generated and made available to the industry in co-operation between Spirent Federal and the GPS Directorate. These scenarios helped with the characterization of the problems on board the satellite and also with looking at possible fixes.

More recently, simulators and other receiver test approaches were widely used to help with the understanding and quantification of the impact of the proposed LightSquared broadband network on GPS systems.

A wide range of simulators was deployed in the testing that was led by the Technical Working Group set up under the auspices of the FCC. The sub-groups of the TWG used not only RF constellation simulators but also live sky sample and playback systems for testing. A wide range of test approaches was adopted, including conducted testing (from the RF simulator via co-axial cable into the receiver front-end, bypassing the antenna and with antenna effects being modeled as part of the simulation where required) and over-the-air test approaches in small and large chambers.

Following the LightSquared testing the current debate is whether it would be helpful to have certification or standardization of GPS and other GNSS receivers in some form. Standards for GPS systems already exist in the safety critical areas such as aviation and maritime as well as in areas such as emergency location (E-911). Discussion on extending current A-GNSS standards to include other positioning methods such as Wi-Fi positioning and MEMS sensor-based positioning are also underway in various standardization forums.

Whatever the problems the industry and systems face today and into the future, one thing seems assured — simulation will remain a key tool to help create a repeatable and controllable environment to enable understanding and continuous improvements in navigation and positioning technology.

John Pottle has more than 20 years of experience in technical, marketing, and business development positions in communications and navigation. He is responsible for marketing at Spirent Communications’ Positioning Technology division in Paignton, UK. He trained as a communications engineer and holds a master’s degree in business administration.

Debug, Verify

Headshot: Paul Myers, Spectracom

Paul Myers, Spectracom

The affordability, shrinking size, and power requirements of GPS and GNSS receivers are accelerating their integration into a multitude of products: personal navigation, safety devices such as alarms systems and cell phones, telematics devices, camera systems, and timing and control systems. But the additional capabilities of position, navigation, and time/frequency synchronization come with an increased cost of test and verification.

Traditional lab bench development and field testing demonstrate operational capabilities. However, these methods alone may not reveal the subtle issues and fatal flaws found in the real world. Lab testing often demonstrates only the best or worst your test cases can offer. Furthermore, field testing only checks conditions and GNSS constellation operation for your location at specific times. Most integrators do not have the luxury of testing their product in the multitude of places their customers might use their product. This is where the application of a GNSS simulator adds value. GNSS simulators allow repeatable testing of real-world situations under a variety of test conditions and in a diverse set of simulated places at different times.

The first step of GNSS integration is to define requirements based on the product use cases. The prudent test designer realizes lab tests, field tests, and simulation all have a place in the product development cycle, and later in maintenance.

Once the product requirements and use cases are known, the type of GNSS receiver can be selected, and supporting software and hardware design can begin. High-level test design is best performed as you design your product. This allows you to better schedule and estimate project time and costs.

Your type of product dictates your test plan design. You will need to allocate some testing to the lab, some to the field, and some to GNSS simulation. The right test case allocation depends on your product type.

Depending on your product requirements, you may have to define navigation test cases, positioning test case, or time and frequency synchronization performance test cases. Position and navigation products require a GNSS receiver with sufficient accuracy and update rates to provide accurate position and navigation data. Time and frequency products, whether mobile or stationary, require a 1PPS output with a serial time code output and sufficient stability and precision to discipline an oscillator to generate precise time and frequency.

Identify which test cases require execution in the lab setting, which require GNSS simulation, and which demand field testing, then allocate them in the test plan to project phase. The old adage that lab testing can’t catch everything that field testing finds can be cheated by the use of GNSS simulation. GNSS simulations reduce cost and schedule time by avoiding repetitive field testing and integration cycles. Plus, simulation testing allows iterative development and retesting by virtually testing in the field.

For example, positioning and time and frequency products can initially utilize lab testing to iteratively develop features and accurately measure system performance. The GNSS simulation can then be leveraged to model the field environment under many different conditions, locations, and times. Finally, field beta testing then can validate the lab and simulation results with real-world beta site experience.

Similarly, mobile navigation or time/frequency products benefit less from lab bench testing and require more in field testing to verify operation under real-world navigation scenarios. Solution accuracy can be baselined in the lab, but accuracy in the field is vital for product success. A GNSS simulator can be used to test conditions, remote locations, and time/dates impossible to achieve using the real GNSS signals. This reduces some of your testing to defining use cases and making simulator configuration files. Without simulation you can only develop, ship, and then fix bugs found by your customers — all the while sweating bullets waiting for users to report problems found from untested situations or when leap seconds occur.

Finally, don’t forget to create regression tests from the verification testing already performed; this enables you to continue to maintain and re-verify product performance. Again, leverage the lab environment, GNSS simulator test cases, and your shipping product to create a product maintenance process. Remember, a smart designer develops the requirements, use cases, and test cases before completing design and development. And a smart integrator uses a GNSS simulator to field test the product before it ever leaves the lab!

Paul Myers is a principal engineer at Orolia USA. He has more than 20 years of experience in embedded systems development in defense and commercial applications. He has a BSEE from Clarkson University, an MSEE from Syracuse University, and is a graduate of General Electric’s Advanced Course in Engineering.

Modern Requirements

Headshot: Markus Lörner,

Markus Lörner, Rohde & Schwarz

Receivers for satellite-based navigation systems such as GPS and GLONASS can be found nowadays in many electronic devices to support location-based services. The faster and more accurately the actual position can be determined, the better the user experience will be. The devices are typically used not only in open space, where the reception conditions would be ideal, but more often in densely populated cities, where harsh conditions such as urban canyons with obscuration and multipath propagation are prevalent. To ensure optimal performance, the receiver needs to be tested and verified with repeatable scenarios that can only be provided by a GNSS simulator.

Standard tests such as time-to-first-fix and location accuracy need to be conducted for all GNSS receivers and modules.

When using a GNSS simulator, this is a straightforward task. The definition of harsher scenarios with multiple obstacles that generate obscuration and reflections is already much more complex, as there are no common test procedures defined. As a result, vendors must specify and generate their own test plans. This requires very flexible GNSS simulation solutions that allow direct access to the satellite constellation configurations.

Many of today’s state-of-the-art receivers are multimode receivers, which means that they support, for example, both GPS and GLONASS. They can therefore use satellites from both systems and still provide a location fix, whereas a single system receiver does not see enough satellites to obtain a stable 3D fix. Each of the two systems must be verified on its own, of course, but additional tests with both systems active are also required to make sure the receiver works properly with these hybrid scenarios. One additional test is to verify receiver performance when the system time of the different GNSS systems is drifting, since these system clocks are controlled and monitored separately.

Increasingly more important is receiver performance in the simultaneous presence of many other signals, such as Bluetooth or WLAN, at a much higher signal level. Another aspect is that cross-correlation distortion from other GNSS systems degrades the desired GNSS signal. Again, no official test requirements are defined in general. The Federal Aviation Administration instructs aviation receiver manufacturers to perform tests with additive noise and CW interferer. Ideally, these tests can be done inside the GNSS simulator.

To summarize, GNSS systems are used more often in especially harsh reception conditions, but users expect perfect location information almost everywhere. To ensure optimal user experience, greater emphasis must be put on testing. Addressing these needs requires a full-featured GNSS simulator, which ideally can also be used as an interference generator for Bluetooth and other standards.

Markus Lörner is a product manager for RF signal generators and power meters at Rohde & Schwarz headquarters in Munich, Germany. He joined the company in 2000 after receiving his degree in electrical engineering from the University of Erlangen-Nürnberg.

Success Factors

Headshot: Mark Sampson, RaceLogic

Mark Sampson, RaceLogic

With more devices now using mobile location-based services and the completion of the GLONASS constellation, it has become more important than ever that companies who incorporate multi-GNSS engines into their products have a reliable, cost-effective way of accurately testing these devices and applications.

Developing GNSS-enabled products within budget and to timescale has, however, always been a challenge. The traditional methods of repetitive field testing and expensive signal laboratory simulation have proved ineffective at offering engineers the repeatability and realism required to test how their devices perform in everyday, real-world scenarios.

Introduction of multi-constellation GNSS simulators has enabled R&D departments to effectively record and replay real-world signals in testing facility conditions, all at a cost-effective price. Providing engineers with the repeatability, consistency, and reliability required to effectively test a range of GNSS-enabled devices, these compact and light-weight systems cut development times by reproducing genuine satellite signals, all from the comfort of your desk.

Before you begin to see how your device performs, there are a number of factors to consider to assure successful GNSS testing is carried out. One of these considerations is the need to clearly pre-define objectives depending on the device or application to be tested and the stage in the product’s development cycle. These can include specific tests for the development of the product chipset, its module, and verification testing to ensure the product meets targets before it is released.

The other consideration is having to test signal reception from multiple satellite constellations to a single GNSS receiver — a special challenge for R&D departments, with system-specific reference frames, system-specific propagation models, timing offsets, date rollover, and cross-system impacts all having to be taken into consideration before successful GNSS testing can be implemented.

After these points have been resolved, using a simulator to simulate scenarios via live-sky signals couldn’t be easier. Connecting directly to an RF antenna input of a GPS engine and simulating the signals associated with navigation using GPS/ GLONASS / Galileo and satellite-based augmentation systems (SBAS), you can carry out highly repeatable tests without leaving the office.

Working alongside a simulation software, engineers can generate a data file that can be replayed on a simulator based on a user-generated trajectory file. This allows you to simulate almost any kind of dynamic profile, at a set time and date, anywhere in the world.

Mark Sampson has more than 15 years of experience in GNSS technology. He works closely with businesses such as Bosch, Intel, Samsung, and Telefonica, providing expertise in testing GNSS devices, applications, and integrations.

Why Test?

Headshot: John F. Clark, CAST Navigation

John F. Clark, CAST Navigation

Testing the operation and performance of a GPS receiver can be a time-consuming and complicated process. To achieve this effort, some receiver manufacturers and system integrators use a combination of receiving live sky GPS signals with an outside antenna as well as receiving signals produced from a GPS simulator.

While you may think that it is easy enough to just go out and put up an antenna to receive the GPS signals from the live sky, you need to ask yourself what it is that you are actually evaluating. Are you evaluating a position solution that contains the effects of local variations such as antenna shading due to placement of the antenna in relation to an existing structure? Are you seeing some effects of multipath being induced to the receiver solution? Is the placement of the antenna causing a larger than expected error? Will you get different navigation results by testing at different times of day? How do you test your receiver under dynamic conditions that contain vehicle motion? Due to the volatility of the GPS constellation, a satellite simulator provides you with repeatable and customizable test conditions.

A GPS simulator must model all transmission paths, anomalies, satellite motion, and user motion to provide you with the ability to control all aspects of the GPS signal to accomplish repeatable testing under known environmental conditions. A GPS simulator should also be capable of allowing you to define a specific time, date, and almanac to be utilized during the simulation, thus enabling you to reproduce the same GPS constellation characteristics as seen from a live-sky antenna for a specific time and location.

You can also use a GPS simulator to assist with the evaluation of new software builds for receivers, characterize a receiver, or evaluate multiple GPS receivers under identical operating conditions. A few simulators also provide the ability to drive an inertial interface, to assist with aircraft avionics integration and testing in a dynamic environment without leaving the laboratory for expensive flight testing.

Using a GPS simulator provides you with the ability to evaluate some operational specifications like time-to-first-fix, time-to-subsequent-fix, low signal-to-noise ratios, receiver loss of RF, reacquisition after signal loss, tracking of rising and setting SVs, and more.

Some GPS simulators also allow you to define and simulate multipath signals. The ability to define the characteristics of multipath signals provides you with a very precise and repeatable signal source to accurately measure and quantify the effects of multipath signals on carrier-phase measurements and receiver performance. This allows you to accurately characterize multiple types of GPS receivers, enabling you to select the appropriate receiver for use in different types of applications and operating environments.

John F. Clark is vice president, engineering, for CAST Navigation, LLC. He has more than 25 years of experience in the GPS industry, and has worked at CAST since 1991.

This article is tagged with , , , , and posted in Opinions