Continuous Experimentation for Software Organizations with Low Control of Roadmap and a Large Distance to Users: An Exploratory Case Study
Paper in proceeding, 2019

With the increasing popularity of A/B testing and other experimentation practices in web systems, companies from a range of different domains are starting to look at continuous experimentation as a way to guide product development and feature prioritization. Research in continuous experimentation traditionally focused on companies that have easy access to user data and that have a high degree of control of the product roadmap. However, little research has been conducted to understand how companies that have a low control of roadmap and have a large distance to the users, such as consultancy companies, can benefit from continuous experimentation practices. To address this problem, we performed an exploratory case study with a software consultancy company combined with a validation procedure with four additional companies. The contribution of this work is three-fold. First, we devised a model to classify a company in the terms of the distance to users and the control of roadmap. Second, we show how control of roadmap and distance to user impacts continuous experimentation. Finally, we present several perceived challenges and benefits of continuous experimentation for companies and directions for future work.


Control of roadmap

A/B testing

Distance to users


Continuous experimentation


Robin Sveningson

Student at Chalmers

David Issa Mattos

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers)

Jan Bosch

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers)

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

03029743 (ISSN) 16113349 (eISSN)

Vol. 11915 528-544
978-303035332-2 (ISBN)

20th International Conference on Product-Focused Software Process Improvement, PROFES 2019
Barcelona, Spain,

Subject Categories

Human Aspects of ICT

Software Engineering

Human Computer Interaction



More information

Latest update