Understanding The Impact of Solver Choice in Model-Based Test Generation
Paper i proceeding, 2020

Background: In model-based test generation, SMT solvers explore the state-space of the model in search of violations of specified properties. If the solver finds that a predicate can be violated, it produces a partial test specification demonstrating the violation.

Aims: The choice of solvers is important, as each may produce differing counterexamples. We aim to understand how solver choice impacts the effectiveness of generated test suites at finding faults.

Method: We have performed experiments examining the impact of solver choice across multiple dimensions, examining the ability to attain goal satisfaction and fault detection when satisfaction is achieved---varying the source of test goals, data types of model input, and test oracle.

Results: The results of our experiment show that solvers vary in their ability to produce counterexamples, and---for models where all solvers achieve goal satisfaction---in the resulting fault detection of the generated test suites. The choice of solver has an impact on the resulting test suite, regardless of the oracle, model structure, or source of testing goals.

Conclusions: The results of this study identify factors that impact fault-detection effectiveness, and advice that could improve future approaches to model-based test generation.

Satisfiability Modulo Theories

Model-based Test Generation

Model-Driven Development

Författare

Ying Meng

University of South Carolina

Gregory Gay

Göteborgs universitet

Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

19493770 (ISSN)

Vol. ESEM '20

Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)
Bari, Italy,

Ämneskategorier

Programvaruteknik

DOI

10.1145/3382494.3410674

ISBN

9781450375801

Mer information

Senast uppdaterat

2021-01-08