Journeys in vector space: Using deep neural network representations to aid automotive software engineering
Doctoral thesis, 2023

Context - The automotive industry is in the midst of a transformation where software is becoming the primary tool for delivering value to customers. While this has vastly improved their product offerings, vehicle manufacturers are facing an urgent need to continuously develop, test, and deliver functionality, while maintaining high levels of quality. Increasing digitalization in the past decade allows us to turn to an interesting avenue for addressing this need, which is data. With activities in engineering and operating vehicles being increasingly recorded as data, and with rapid advances in machine learning, this work takes a data-driven, deep learning approach to solve tasks in automotive software engineering.

Scope - This work focuses upon two automotive software engineering tasks, (1) assessing whether embedded software complies with specified design guidelines, and (2) generating realistic stimuli to test embedded software in virtual rigs.

Contributions - First, as the main tool for solving the design compliance task, we train tasnet, a language model of automotive software. Then, we introduce DECO, a rule-based algorithm which assesses the compliance of query programs with the Controller-Handler automotive software design pattern. Utilizing the property of semantic regularity in language models, DECO conducts this assessment by comparing the geometric alignment between query and benchmark programs in tasnet's representation space. Second, focusing upon stimulus generation, we train logan, a deep generative model of in-vehicle behavior. We then introduce MLERP, a rule-based algorithm which takes user-specified test conditions and samples logan to generate realistic test stimuli which adhere to the conditions. Using the property of interpolation in representation space for semantic combination, MLERP generates novel stimuli within the boundaries of specification. Third, staying with the testing use case, we improve logan to train silgan, which simplifies the specification of test conditions. Then, noting that sampling a generative model is less efficient, we introduce GRADES, a rule-based algorithm that uses a specially constructed objective to search for stimuli. GRADES is built upon the fact that neural networks in silgan are differentiable, and, given an appropriate objective, a gradient descent-based search in model representation space efficiently yields suitable stimuli. Fourth, we note that our recipe for solving automotive software engineering tasks consistently pairs a self-supervised foundation model with a rule-based algorithm operating in the model's representation space. This paradigm for building predictive models, which we refer to as 'pre-train and calculate', not only extracts nuanced predictions without any supervision, but is also relatively transparent. Fifth, with our predictive approach relying heavily upon properties in abstract representation space, we develop techniques that explain and characterize selected high-dimensional vector spaces. Overall, by taking a data-driven deep learning approach, techniques we introduce reduce manual effort in undertaking two crucial engineering tasks. This has a direct effect on improving the cadence of automotive software engineering without compromising the quality of delivery.

automotive software design and testing

generative adversarial networks

latent space arithmetic

generative AI

explainable AI

large language models

Analysen, EDIT, Rännvägen 6B, Gothenburg, Sweden. For Zoom, use password 003863
Opponent: Professor Earl T. Barr, University College London, London, United Kingdom


Dhasarathy Parthasarathy

Chalmers, Computer Science and Engineering (Chalmers), Functional Programming

Measuring design compliance using neural language models: An automotive case study

PROMISE 2022 - Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering, co-located with ESEC/FSE 2022,; (2022)p. 12-21

Paper in proceeding

SilGAN: Generating driving maneuvers for scenario-based software-in-The-loop testing

Proceedings - 3rd IEEE International Conference on Artificial Intelligence Testing, AITest 2021,; (2021)p. 65-72

Paper in proceeding

Controlled time series generation for automotive software-in-the-loop testing using GANs

Proceedings - 2020 IEEE International Conference on Artificial Intelligence Testing, AITest 2020,; (2020)p. 39-46

Paper in proceeding

Does the dataset meet your expectations? Explaining sample representation in image data

Belgian/Netherlands Artificial Intelligence Conference,; (2020)p. 194-208

Paper in proceeding

Modern vehicles are largely controlled by software. Software has become the primary mechanism for delivering new vehicle functionality, and automotive companies are beginning to resemble software companies. However, vehicle software displays several characteristics that sets it apart from many other kinds of software. Embedded vehicle software needs to simultaneously consider aspects like safety, security, time criticality, privacy, etc., which makes its development and testing quite challenging. This work applies tools from deep learning and generative AI to account for these special characteristics and automate selected software engineering tasks. Using large language models and generative adversarial networks, this work introduces techniques that help speed up automotive software design and testing. Put together, this helps automakers introduce new functionality at a faster cadence, speeding up the development of safe, productive, and sustainable transport solutions.

Subject Categories

Other Mechanical Engineering

Software Engineering

Embedded Systems

Computer Systems



Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 5411



Analysen, EDIT, Rännvägen 6B, Gothenburg, Sweden. For Zoom, use password 003863


Opponent: Professor Earl T. Barr, University College London, London, United Kingdom

More information

Latest update