The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-Driven Organization at Scale
Paper i proceeding, 2017

© 2017 IEEE. Software development companies are increasingly aiming to become data-driven by trying to continuously experiment with the products used by their customers. Although familiar with the competitive edge that the A/B testing technology delivers, they seldom succeed in evolving and adopting the methodology. In this paper, and based on an exhaustive and collaborative case study research in a large software-intense company with highly developed experimentation culture, we present the evolution process of moving from ad-hoc customer data analysis towards continuous controlled experimentation at scale. Our main contribution is the 'Experimentation Evolution Model' in which we detail three phases of evolution: Technical, organizational and business evolution. With our contribution, we aim to provide guidance to practitioners on how to develop and scale continuous experimentation in software organizations with the purpose of becoming data-driven at scale.

customer feedback

product value

data science

A/B testing

Experiment Owner

continuous product innovation

Experimentation Evolution Model

continuous experimentation


A. Fabijan

Malmö universitet

P. Dmitriev


Helena Holmström Olsson

Malmö universitet

Jan Bosch

Chalmers, Data- och informationsteknik, Software Engineering

Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering, ICSE 2017

978-153863868-2 (ISBN)


Data- och informationsvetenskap





Mer information

Senast uppdaterat