Using Machine Learning to Generate Test Oracles: A Systematic Literature Review
Paper in proceeding, 2021
Based on a sample of 22 relevant studies, we observed that ML algorithms generated test verdict, metamorphic relation, and - most commonly - expected output oracles. Almost all studies employ a supervised or semi-supervised approach, trained on labeled system executions or code metadata - including neural networks, support vector machines, adaptive boosting, and decision trees. Oracles are evaluated using the mutation score, correct classifications, accuracy, and ROC. Work-to-date show great promise, but there are significant open challenges regarding the requirements imposed on training data, the complexity of modeled functions, the ML algorithms employed - and how they are applied - the benchmarks used by researchers, and replicability of the studies. We hope that our findings will serve as a roadmap and inspiration for researchers in this field.
Automated Test Oracle Generation
Automated Test Generation
Machine Learning
Test Oracle
Author
Afonso Fontes
University of Gothenburg
Gregory Gay
University of Gothenburg
TORACLE 2021 - Proceedings of the 1st International Workshop on Test Oracles, co-located with ESEC/FSE 2021
1-10
978-145038626-5 (ISBN)
Athens, Greece,
Context-Infused Automated Software Test Generation
Swedish Research Council (VR) (2019-05275), 2020-01-01 -- 2023-12-31.
Subject Categories
Software Engineering
Information Science
Computer Science
DOI
10.1145/3472675.3473974
ISBN
9781450386265