Information Flow in Software Testing - An Interview Study with Embedded Software Engineering Practitioners

Journal article, 2019

Background: In order to make informed decisions, software engineering practitioners need information from testing. However, with the trend of increased automation, there is exponential growth and increased distribution of this information. This paper aims at exploring the information flow in software testing in the domain of embedded systems. Method: Data was collected through semi-structured interviews of twelve experienced practitioners with an average work experience of more than fourteen years working at five organizations in the embedded software industry in Sweden. Seventeen hours of audio recordings were transcribed and anonymized into 130 pages of text that was analyzed by means of thematic analysis. Results: The flow of information in software testing can be represented as feedback loops involving stakeholders, software artifacts, test equipment, and test results. The six themes that affect the information flow are how organizations conduct testing and trouble shooting, communication, processes, technology, artifacts, and the organization of the company. Seven main challenges for the flow of information in software testing are comprehending the objectives and details of testing; root cause identification; poor feedback; postponed testing; poor artifacts and traceability; poor tools and test infrastructure; and distances. Finally, five proposed approaches for enhancing the flow are: close collaboration between roles; fast feedback; custom test report automation; test results visualization; and the use of suitable tools and frameworks. Conclusions: The results indicate that there are many opportunities to improve the flow of information in software testing: a first mitigation step is to better understand the challenges and approaches. Future work is needed to realize this in practice, for example, on how to shorten feedback cycles between roles, as well as how to enhance exploration and visualization of test results.