On tackling quality threats for the assessment of measurement programs: A case study on the distribution of metric usage and knowledge
Journal article, 2017

Context Measurement programs are subject to changing requirements, causing the need for a robustness to compensate these changes. Existing methods enable the assessment of measurement programs and their robustness with the help of company representatives. Goal: However, it is not clear how much the spread of knowledge among roles and differences between company internal organizations influence the result and quality of the assessment. Thus, it is not known today who and how many representatives should be interviewed to get reliable results. Method: To address these points we performed a case study on the assessment method MeSRAM, spanning 18 interviews with 14 representatives of 4 different organizations within a large international communication company. Results: Managers are able to answer more questions and their answers are more often correct than those of engineers. Furthermore, most errors occur for questions about the use of concrete metrics, while for aspects about the metrics' infrastructure no conflicting answers occurred. We also identified 6 reasons for gaining incorrect answers. Conclusion: The assessment of measurement programs requires additional care to ensure the quality of the results. (C) 2016 Elsevier B.V. All rights reserved.

OCEEDINGS25th International Conference on Software Engineering (ICSE 2003)

OCESS AND PRODUCT MEASUREMENT (IWSM-MENSURA)Joint Conference of the 23rd International


Robust assessment



software process improvement






Empirical study


Computer Science

MAY 03-10


XP 201314th International Conference on Agile Processes in Software Engineering and

Measurement programs



Regina Hebig

University of Gothenburg

H. Y. Wang

Ericsson (China) Communications Co. Ltd

Science of Computer Programming

0167-6423 (ISSN)

Vol. 135 45-74

Subject Categories

Software Engineering