Measuring design compliance using neural language models: An automotive case study
Paper in proceeding, 2022

As the modern vehicle becomes more software-defined, it is beginning to take significant effort to avoid serious regression in software design. This is because automotive software architects rely largely upon manual review of code to spot deviations from specified design principles. Such an approach is both inefficient and prone to error. In recent days, neural language models pre-trained on source code are beginning to be used for automating a variety of programming tasks. In this work, we extend the application of such a Programming Language Model (PLM) to automate the assessment of design compliance. Using a PLM, we construct a system that assesses whether a set of query programs comply with Controller-Handler, a design pattern specified to ensure hardware abstraction in automotive control software. The assessment is based upon measuring whether the geometrical arrangement of query program embeddings, extracted from the PLM, aligns with that of a set of known implementations of the pattern. The level of alignment is then transformed into an interpretable measure of compliance. Using a controlled experiment, we demonstrate that our technique determines compliance with a precision of 92%. Also, using expert review to calibrate the automated assessment, we introduce a protocol to determine the nature of the violation, helping eventual refactoring. Results from this work indicate that neural language models can provide valuable assistance to human architects in assessing and fixing violations in automotive software design.

software design patterns

language model evaluation

neural programming language models


Dhasarathy Parthasarathy

Volvo Group

Cecilia Ekelin

Volvo Group

Anjali Karri

Student at Chalmers

Jiapeng Sun

Student at Chalmers

Panagiotis Moraitis

Student at Chalmers

PROMISE 2022 - Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering, co-located with ESEC/FSE 2022

9781450398602 (ISBN)

18th ACM International Conference on Predictive Models and Data Analytics in Software Engineering, PROMISE 2022, co-located with the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022
Singapore, Singapore,

Subject Categories

Software Engineering

Embedded Systems

Computer Systems



More information

Latest update

1/3/2024 9