Outliers and Replication in Software Engineering
Paper i proceeding, 2014

Empirical software engineering is a research field of growing interest. Studies within this field handles an increasing amount of data. In order to replicate a study the data needs to be accessible and all processing of this data needs to be reproducible. Specifically, the handling of deviating data points, also known as outliers, needs to be documented in order for a study to be replicated. This study investigated the data availability for recently published studies within empirical software engineering. Furthermore, it also investigated if outliers are documented in the same research field. Papers were reviewed using a literature review and the presence of outliers was investigated using an unsupervised outlier detection method. Only 37% of the papers reviewed had their data accessible. Furthermore, in many cases outliers were present in the reviewed studies but 63% of the papers studies did not mention how outliers were handled. The data availability within empirical software engineering research is low and is hindering replication of studies. Additionally, the lack of documentation regarding how outliers are handled is hindering replication.

Reproducibility.

Empirical Software Engineering

Outlier Detection

Software engineering

Literature reviews

Outliers

Data handling

Statistics

Replication

Data availability

Författare

Erik Lindqvist

Chalmers, Data- och informationsteknik

Henrik Larsson

Chalmers, Data- och informationsteknik

Richard Torkar

Göteborgs universitet

Proceedings The 21st Asia-Pacific Software Engineering Conference

1530-1362 (ISSN)

207-214

Ämneskategorier

Programvaruteknik

DOI

10.1109/APSEC.2014.40

ISBN

978-1-4799-7425-2