Enumerative Testing and Embedded Languages
Licentiate thesis, 2012
This thesis explores rapid experimental development of programming
languages, with particular emphasis on effective semi-automatic testing. Our
results are actualised in two Haskell libraries: BNFC-meta and Feat.
BNFC-meta is an extension of the BNF Converter (BNFC) tool. As such it is
capable of building a complete compiler front end from a single high level
We merge this with the practice of embedding languages in Haskell, both by
embedding BNFC itself and embedding all languages defined using BNFC-meta.
Embedding is carried out by means of quasi-quotation enabling use of
the languages concrete syntax inside Haskell code.
A simple extension to the grammar formalism adds anti-quoting, in turn allowing
Haskell code embedded in the concrete
syntax of the embedded languages. The end user can thus seamlessly mix concrete
and abstract syntax. Our automatic approach improve on existing manually defined
Haskell anti-quoters by not polluting the AST datatypes.
Our second major contribution,
Feat (Functional Enumeration of Algebraic Types) automatically enables
property based testing on the large AST types generated by
BNFC-meta and such tools, but it is useful more generally for algebraic types.
Feat is based on the mathematical notion of an enumeration as a bijective
function from natural numbers to an enumerated set. This means that unlike
previous list-based enumeration methods it is not intrinsically serial and can be used for
both random and exhaustive testing.
We describe a theory of functional enumeration as a simple
algebra closed under sums, products, guarded recursion
and bijections. We implement these ideas in a library and show that it compares
favourably to existing tools when testing AST types.