Annotation Requirements to Annotation Errors: A Causal Modelling Analysis for AI-Enabled Perception Systems Development

Paper in proceeding, 2026

Context: Data annotation is a foundational software engineering activity in the development of AI-enabled perception systems (AIePS) for safety-critical automotive domains. Despite quality assurance, recurring data annotation errors (DAEs) still degrade robustness, safety, and trustworthiness. Prior work often treats DAEs as execution-level defects, leaving their upstream, requirements-level origins in data annotation requirements (DARs) underexplained. Objective: This study conducts a causal analysis of how DAR challenges propagate into DAEs across completeness, accuracy, and consistency. Method: We apply an integrative empirical synthesis grounded in a qualitative interview study (>50 hours of transcripts; 19 interviews, 20 participants) spanning the automotive AI supply chain. We evaluate five DAR challenges (edge-case coverage gaps, ambiguity, evolution, misalignment, and resource limitations) and an empirically grounded taxonomy of 18 recurring DAE types. We then construct theory-informed Directed Acyclic Graphs (DAGs) that trace propagation paths from DAR challenges through intermediate mechanisms to downstream DAEs. We release an interactive visualisation tool and an open dataset to facilitate the replication and practical use of DAGs.
Results: The DAGs reveal systematic multi-step propagation. Edge-case coverage gaps split into complexity-driven accuracy/consistency degradation (semantic-spatial confusion, disagreement) and definition-driven completeness degradation (missing triage, biased sampling). Ambiguous DARs distort labelling decisions, increasing wrong labels, bounding-box errors, granularity mismatch, bias-driven errors, and inter-annotator disagreement. Evolving DARs and misalignment sustain cross-batch drift via taxonomy/detail inconsistencies, hand-off rule drift, weak traceability, and cross-modality misalignment. Resource limitations amplify all pathways by weakening QA, feedback, and auditability.
Conclusion: DAEs are structural consequences of DAR weaknesses and governance constraints, motivating proactive DAR governance to stabilise specifications, improve hand-offs, and strengthen traceability and verification.