Outlier Detection as a Safety Measure for Safety Critical Deep Learning

Doctoral thesis, 2023

Context: Deep learning (DL) has proven to be a valuable component in object detection and semantic segmentation tasks, as the techniques have shown significant performance gains compared to hand-made image processing algorithms. DL refers to an optimization process where a model learns properties and parameters itself through in iterative process running on labeled data. The resulting model contains abstract features that are unintuitive to explain, thus challenging to ensure that the model will work as intended in safety critical applications (SCA).

Aim: The aim of this thesis has been to study how to connect parameters from DL with verification and testing for safety critical applications, and what extensions are necessary to verify deep neural networks. More specifically, this thesis has investigated the use of outlier detection as one testing method to detect when the model is operating on unfamiliar data.

Method: A comprehensive review of DL metrics and outlier detection metrics have been conducted. These metrics have been used to construct several new metrics to evaluate how the model behaves when encountering out-of-distribution (OOD) samples.
An evaluation framework has been constructed that performs objective evaluation of OOD detection methods. The framework has been applied on various ranges of image datasets, starting with small scale images and continuing with realistic camera based use-cases from the automotive domain.

Results: This thesis found that one major issue with deployment of DL in SCAs is quantifying and tracing performance measures. The issue exists due to the difficulty in defining requirements and test cases for DL models, and expressing the models performance in safety related metrics. While DL performance is commendable, if the performance cannot be ensured, the technique should not be deployed in SCA. Our experiments show that the effect of OOD samples can be mitigated by extending the model with safety measures, i.e., measures that reduce the impact of undesired behavior. This thesis show how to use a risk-coverage trade-off metric that connects DL performance with functional safety requirements, such that safety engineers may allocate safety requirements on DL components and evaluate their performance.

Future work: Future works recommend testing the outlier detectors on further real world scenarios and how the detector can be part of a safety argumentation.