Interactive Production Performance Feedback in the IDE

Paper in proceeding, 2019

Performance problems are hard to track and debug, especially when detected in production and originating from development. Software developers try to reproduce the perfor- mance problem locally and debug it in the source code. However, production environments are too different to what profiling and testing can simulate locally in development environments. Software developers need to consult production monitoring tools to reason about and debug the issue. We propose an integrated approach that constructs an In-IDE performance model from monitoring data from production environments. When developers change source code, we perform incremental analysis to update our performance model to reflect the impact of these changes. This allows us to provide performance feedback to developers in near real time to enable them to prevent performance problems from reaching production. We present a tool, PerformanceHat, an Eclipse plugin that we evaluated in a controlled experiment with 20 professional software developers, in which they work on soft- ware maintenance tasks using our approach and a representative baseline (Kibana). We found that developers were significantly faster in (1) detecting the performance problem, and (2) finding the root-cause of the problem. We conclude that our approach helps detect, prevent and debug performance problems faster.