Verification of Erlang Programs using Testing and Tracing

Licentiate thesis, 2005

Producing reliable computer programs is a difficult and expensive task, and the constant demand for more and more complex systems does not make the task easier. In this thesis we present some program verification tools and techniques which can improve the situation. We introduce a testing technique, where the idea is to first run the system with some test data, thereby creating traces of events and states. Then, using an abstraction function specified by the user, an abstract state transition diagram of the system is automatically generated from the traces. We have created a special language for expressing the abstraction functions. The abstract state transition diagram can be nicely visualized and thus greatly helps in debugging the system. Lastly, formal requirements of the system specified in temporal logic can be checked automatically for the abstract state transition diagram. We present a new implementation of a leader election algorithm used in the generic leader behavior known as gen_leader. The new implementation is based on a formally verified algorithm, which has been adopted to fulfill the existing requirements. The testing technique with traces and abstractions has been used to check the implementation we propose here. We also used the new testing tool QuickCheck to further increase our confidence in the implementation of this very tricky algorithm. The new implementation passed all tests successfully. We propose an extension to an existing formal single-node semantics for Erlang. The extension models the concept of nodes. The motivation is that there are sequences of events that can occur in practice, but are impossible to describe using a single-node semantics. The consequence is that some errors might not be detected by model checkers based on the existing semantics, or by other single-node verification techniques such as testing. Our extension is modest; it re-uses most of the existing semantics, but adds a new top-layer. We present a case study in which we test the implementation and adaptation of a formally verified algorithm. Algorithms described in the literature can often be used to solve practical, industrial problems. Nevertheless, we observe little transfer of algorithms from research papers into products. Most of the time, algorithm modifications are necessary to meet all requirements. We propose adaptation and testing of formal properties as the cheapest way to check the correctness of the modifications, since performing a formal proof seems unrealistic for industrial systems. We show how the properties stated in the articles guide our tests.