ACE: Easy deployment of field optimization experiments

Paper in proceeding, 2019

Optimization of software parameters is a recurring activity in the life-cycle of many software products, from prototypes and simulations, test beds and hardware-in-the-loop scenarios, field calibrations to the evolution of continuous deployment cycles. To perform this activity, software companies require a combination of software developers and optimization experts with domain specific knowledge. Moreover, in each of life-cycle steps, companies utilize a plethora of different tools, tailored for specific domains or development stages. To most companies, this scenario leads to an excessive cost in the optimization of smaller features or in cases where it is not clear what the returned value will be. In this work we present a new optimization system based on field experiments, that is aimed to facilitate the adoption of optimization in all stages of development. We provide two main contributions. First, we present the architecture of a new optimization system that allows existing software systems to perform optimization procedures in different domains and in different development stages. This optimization system utilizes domain-agnostic interfaces to allow existing systems to perform optimization procedures with minimal invasiveness and optimization expertise. Second, we provide an overview of the deployments, discuss the advantages and limitations and evaluate the optimization system in three empirical scenarios: (1) offline optimization with simulations; (2) optimization of a communication system in a test bed in collaboration with Ericsson; (3) live optimization of a mobile application in collaboration with Sony Mobile. We aim to provide practitioners with a single optimization tool that can leverage their optimization activities from offline to live systems, with minimal invasiveness and optimization expertise.