Qually: A Quality Validation Toolbox for Automotive Perception Data Towards Trustworthy AI
Rapport, 2022
In this project, the objective is to develop an end-to-end quality control toolbox to detect errors and anomalies throughout the entire pipeline. To achieve this objective, we divide the project into three work packages, where the first step is to design a set of data properties and their corresponding requirements as quality specifications for data at each stage. Given these specifications, as a second step, we have developed a toolbox, Qually, to evaluate data quality metrics and detect errors and anomalies throughout the AI pipeline. In the last work package, as a demonstrator, Qually is applied to improve automated annotations. This is implemented in three steps: 1) errors are identified using the quality metrics evaluated by Qually; 2) Qually suggests an automatic correction using ensemble techniques; 3) the corrected annotations are evaluated by Qually to confirm the improvement in quality. The error detection and suggested corrections are manually inspected to statistically validate the outcome of Qually.
As the next step, besides further developing Qually as a software to improve its robustness, capacity, scalability and completeness, we plan to focus on enriching the set of data properties and quality specifications, especially by including technical and business requirements from various automotive stakeholders. We also plan to investigate the possibility and scalability of integrating formal verification techniques for quality control.
autonomous driving
data collection
ai reliability
deep learning
data quality
Författare
Yinan Yu
Chalmers, Data- och informationsteknik, Funktionell programmering
Samuel Scheidegger
Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik
Asymptotic AB
Jörg Bakker
Asymptotic AB
Kategorisering
Styrkeområden
Informations- och kommunikationsteknik
Transport
Ämneskategorier
Elektroteknik och elektronik
Övrigt
Utgivare
VINNOVA