Outliers and Replication in Software Engineering
Paper in proceedings, 2014

Empirical software engineering is a research field of growing interest. Studies within this field handles an increasing amount of data. In order to replicate a study the data needs to be accessible and all processing of this data needs to be reproducible. Specifically, the handling of deviating data points, also known as outliers, needs to be documented in order for a study to be replicated. This study investigated the data availability for recently published studies within empirical software engineering. Furthermore, it also investigated if outliers are documented in the same research field. Papers were reviewed using a literature review and the presence of outliers was investigated using an unsupervised outlier detection method. Only 37% of the papers reviewed had their data accessible. Furthermore, in many cases outliers were present in the reviewed studies but 63% of the papers studies did not mention how outliers were handled. The data availability within empirical software engineering research is low and is hindering replication of studies. Additionally, the lack of documentation regarding how outliers are handled is hindering replication.

Empirical Software Engineering

Data handling

Outlier Detection

Replication

Data availability

Statistics

Software engineering

Outliers

Reproducibility.

Literature reviews

Author

Erik Lindqvist

Student at Chalmers

Henrik Larsson

Student at Chalmers

Richard Torkar

University of Gothenburg

Proceedings The 21st Asia-Pacific Software Engineering Conference

1530-1362 (ISSN)

207-214

Subject Categories

Software Engineering

DOI

10.1109/APSEC.2014.40

More information

Latest update

11/15/2019