Testing an Optimising Compiler by Generating Random Lambda Terms

Licentiate thesis, 2012

This thesis tries to improve on the relatively uncommon practice of random testing of compilers. Random testing of compilers is difficult and not widespread for two reasons. First, it is hard to come up with a generator of valid test data for compilers, that is a generator of programs. And secondly, it is difficult to provide a specification, or test oracle, that decides what should be the correct behaviour of a compiler. This work addresses both of these problems. Existing random compiler test tools do not use a structured way of generating well-typed programs, which is a often a requirement to perform comprehensive testing of a compiler. This thesis proposes such a method based on a formal calculus. To address the second problem, this thesis proposes using two variants of differential testing, which allows for detecting bugs even when a very limited partial specification of the tested compiler is available. This setup is evaluated practically by performing effective testing of a real compiler.