Combining Monitoring and Autonomous Feedback Requests to Elicit Actionable Knowledge of System Use

Paper in proceeding, 2019

[Context and motivation] To validate developers’ ideas of what users might want and to understand user needs, it has been proposed to collect and combine system monitoring with user feedback. [Question/problem] So far, the monitoring data and feedback have been collected passively, hoping for the users to get active when problems emerge. This approach leaves unexplored opportunities for system improvement when users are also passive or do not know that they are invited to offer feedback. [Principal ideas/results] In this paper, we show how we have used goal monitors to identify interesting situations of system use and let a system autonomously elicit user feedback in these situations. We have used a monitor to detect interesting situations in the use of a system and issued automated requests for user feedback to interpret the monitoring observations from the users’ perspectives. [Contribution] The paper describes the implementation of our approach in a Smart City system and reports our results and experiences. It shows that combining system monitoring with proactive, autonomous feedback collection was useful and surfaced knowledge of system use that was relevant for system maintenance and evolution. The results were helpful for the city to adapt and improve the Smart City application and to maintain their internet-of-things deployment of sensors.