Business as Unusual: A Model for Continuous Real-Time Business Insights Based on Low Level Metrics

Paper i proceeding, 2019

© 2019 IEEE. A wide variety of tools to monitor and track software systems, such as websites or smartphone applications, during runtime already exists. However, their aggregated results are often not sufficient to answer questions on a product management level since these questions address several levels of complexity and abstractions, and tend to be formulated on a rather high level, for instance concerning the efficiency of their website structure for their users. A straightforward mapping between low level metrics and high level insights is typically not possible. This causes a gap that makes it challenging to continuously provide quantitative high-level insights in real-time. In order to address this challenge, we conducted a study within three distinct platforms and products, and propose a model based on our results. After defining a case for each of the independent platforms and products, we implemented a process to measure high level insights using low level metrics for each of these cases. Next, we compared the procedures and steps that were taken in each of the cases and derived a model that describes a generic approach how to utilize and process data in order to gain higher level insights. Our model structures the steps from data to knowledge over different levels of complexity and abstraction, namely operational, tactical, and strategic. Thereby, the knowledge acquired in each phase serves as input in the next phase which increases the measurable level of complexity with each iteration. Since the steps in our model are specifically arranged as a pipeline, it enables practitioners to automate a continuous and quantitative measurement of high level insights in real-time.