Random Structured Test Data Generation for Black-Box Testing
Doktorsavhandling, 2014

We show how automated random testing can be used to effectively find bugs in complex software, such as an optimising compiler. To test the GHC Haskell compiler we created a generator of simple random programs, used GHC to compile them with different optimisation levels, and then compared the results of running them. Using this simple approach we found a number of optimisation bugs in GHC. This approach for finding bugs proved to be very effective, but we found that implementing a generator of random programs by hand required a large amount of effort. Therefore, we developed an automatic method for deriving random generators of complex test data based on computable boolean predicates that specify the well-formed values of the data type. Defining such a predicate is usually much quicker than implementing a dedicated generator, even if its performance might be comparably lower. In addition, we discovered that the pseudorandom number generator used by us for random testing is unreliable, and that no reliable contruction exists that supports our particular requirements. Consequently, we designed and implemented a high-quality pseudorandom number generator, which is based on a known and reliable cryptographic construction, and whose correctness is supported by a formal argument. Finally, we present how random testing can be used to rank a group of programs according to their relative correctness with respect to their observed behaviour. The ranking method removes the influence of the distribution of the random data generator used for testing, which results in a reliable ranking.

Lecture room EA, ED&IT building, Rännvägen 6B, Chalmers University of Technology
Opponent: Prof. John Regehr, School of Computing, University of Utah

Författare

Michal Palka

Chalmers, Data- och informationsteknik, Programvaruteknik

Splittable Pseudorandom Number Generators using Cryptographic Hashing

Proceedings of the Haskell Symposium 2013,; (2013)p. 47-58

Paper i proceeding

Generating Constrained Random Data with Uniform Distribution

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics),; Vol. 8475(2014)p. 18-34

Paper i proceeding

Ranking programs using black box testing

Proceedings - International Conference on Software Engineering,; (2010)p. 103-110

Paper i proceeding

Ämneskategorier

Datavetenskap (datalogi)

ISBN

978-91-7385-996-7

Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie

Lecture room EA, ED&IT building, Rännvägen 6B, Chalmers University of Technology

Opponent: Prof. John Regehr, School of Computing, University of Utah

Mer information

Skapat

2017-10-07