Abstract
Keywords:GPS metrics, ground truth trajectory, vehicle localisation, quantitative indicators
The performance of car navigation systems depends on absolute location information provided by Global Navigation Satellite Systems like GPS. Whilst multiple techniques exist to determine a vehicle location, GPS estimations remain the sole source of absolute location information. This paper describes a metrics based method for the evaluation of GPS receivers in real traffic conditions. For this purpose, the trajectory of the test vehicle is measured in real-time using a high accuracy inertial based localisation system. The results are exploited as the ground truth with respect to which location estimations acquired concurrently from several GPS receivers are compared. Statistics on error spreads under test conditions are generated and quantitative results obtained. Information on the environment traversed by the vehicle, referred as 'context' in this paper, is included to facilitate interpretation. A series of tests runs were structured according to the availability of line of sight signals from the constellation of GPS satellites, on the vehicle onboard GPS receivers. These include rural, peri-urban and urban areas as well as special cases like urban canyons and tunnels. The results are included together with their statistical analysis and interpretation. The experiments have shown that estimations differ from manufacturer to manufacturer in most conditions. That is, location estimations are different following assumptions embedded in the receiver software, e.g. as the vehicle enters a tunnel, one GPS receiver will stop generating estimations, others will presume the vehicle moves at a constant speed and in a straight line, etc. At roundabouts, different levels of overshoot exist resulting from the use of different estimation filters and vehicle models. Urban Canyons are scenarios where most difficulties exist; the tested GPS struggled to attain good estimations, hence the need to use other observations to compensate for errors or missing GPS estimates. All GPS receivers respond differently, thus, if these are changed, extensive tuning and testing will be necessary on the car navigation systems and applications, to take into account the different responses.