Your System Gets Better Every Day You Use It: Towards Automated Continuous Experimentation
Paper in proceedings, 2017

Innovation and optimization in software systems can occur from pre-development to post-deployment stages. Companies are increasingly reporting the use of experiments with customers in their systems in the post-deployment stage. Experiments with customers and users are can lead to a significant learning and return-on-investment. Experiments are used for both validation of manual hypothesis testing and feature optimization, linked to business goals. Automated experimentation refers to having the system controlling and running the experiments, opposed to having the R&D organization in control. Currently, there are no systematic approaches that combine manual hypothesis validation and optimization in automated experiments. This paper presents concepts related to automated experimentation, as controlled experiments, machine learning and software architectures for adaptation. However, this paper focuses on how architectural aspects that can contribute to support automated experimentation. A case study using an autonomous system is used to demonstrate the developed initial architecture framework. The contributions of this paper are threefold. First, it identifies software architecture qualities to support automated experimentation. Second, it develops an initial architecture framework that supports automated experiments and validates the framework with an autonomous mobile robot. Third, it identifies key research challenges that need to be addressed to support further development of automated experimentation.

Controlled Experiments

Software architecture

Autonomous systems

Automated experimentation

Author

David Issa Mattos

Software Engineering for People, Architecture, Requirements and Traceability

Jan Bosch

Software Engineering for Testing, Requirements, Innovation and Psychology

Helena Holmström Olsson

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers)

Euromicro Conference on Software Engineering and Advanced Applications (SEAA)

978-1-5386-2140-0 (ISSN) 978-1-5386-2141-7 (eISSN)

2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA)
Vienna, Austria,

Subject Categories

Software Engineering

Information Science

Computer Systems

DOI

10.1109/SEAA.2017.15

More information

Created

4/2/2019 9