On Performance Evaluation of Automotive Active Safety Systems

Licentiate thesis, 2010

Road traffic accidents are a major global problem, annually causing over 1.2 million fatalities. To improve road safety, active safety systems support the driver by monitoring the vehicle and its surroundings, identifying hazardous situations and actively intervening to prevent or mitigate consequences of accidents. A major challenge in active safety system development is to evaluate the system performance, e.g. that the system makes correct decisions, in a wide variety of traffic scenarios. In this thesis, novel methods for performance evaluation of active safety systems are presented. The system decision to intervene is commonly taken when a threat function reaches a specific threshold. The dimensionality of the input state space for the threat function is in general very large making exhaustive evaluation in real vehicles expensive and time consuming. An efficient theoretical method is proposed for estimating a bound on decision timing error, i.e. the worst case performance. Sensor errors are important to evaluate as they significantly affect system performance. Camera-based sensors use computer vision techniques to interpret the surrounding world, e.g. to detect, classify and track objects. In this thesis, a novel framework is proposed in which photo-realistic augmented imagery is generated, by adding virtual agents into real imagery, and use it for performance evaluation. Thus, performance of mobile computer vision systems in hazardous scenarios can be evaluated safely, with access to ground truth data, while still using a real image background from recorded data. Furthermore, a framework is proposed where experimental data is used for decision function analysis. Efficient methods for system performance evaluation are derived which can be used to analyze the decision function sensitivity to input perturbations, e.g. sensor errors, or for decision function tuning. The framework is specially efficient for specification of input perturbation restrictions, e.g. sensor requirements.