The integration of machine learning into automated test generation: A systematic mapping study
Journal article, 2023

Machine learning (ML) may enable effective automated test generation. We characterize emerging research, examining testing practices, researcher goals, ML techniques applied, evaluation, and challenges in this intersection by performing. We perform a systematic mapping study on a sample of 124 publications. ML generates input for system, GUI, unit, performance, and combinatorial testing or improves the performance of existing generation methods. ML is also used to generate test verdicts, property-based, and expected output oracles. Supervised learning—often based on neural networks—and reinforcement learning—often based on Q-learning—are common, and some publications also employ unsupervised or semi-supervised learning. (Semi-/Un-)Supervised approaches are evaluated using both traditional testing metrics and ML-related metrics (e.g., accuracy), while reinforcement learning is often evaluated using testing metrics tied to the reward function. The work-to-date shows great promise, but there are open challenges regarding training data, retraining, scalability, evaluation complexity, ML algorithms employed—and how they are applied—benchmarks, and replicability. Our findings can serve as a roadmap and inspiration for researchers in this field.

automated test generation

test case generation

machine learning

test oraclegeneration

test input generation


Afonso Fontes

University of Gothenburg

Gregory Gay

University of Gothenburg

Software Testing Verification and Reliability

0960-0833 (ISSN) 1099-1689 (eISSN)

Vol. 33 4 e1845

Context-Infused Automated Software Test Generation

Swedish Research Council (VR) (2019-05275), 2020-01-01 -- 2023-12-31.

Subject Categories

Software Engineering

Computer Science



More information

Latest update