Exploring Software Cost Estimation Inaccuracy

Doctoral thesis, 2012

The inaccuracy of software cost estimates has for long been a source of frustration to software practitioners and cost estimation researchers. Despite huge efforts to improve this important practice, estimation accuracy is still low. This thesis uses empirical and theoretical evidence to investigate why this problem persists. Empirical evidence was collected from software practitioners in large, traditional, Swedish companies in order to examine their perspectives on why estimates are inaccurate, and whether (and if so why) estimates are used for purposes other than those formally specified by their companies. Argyris's learning theory is used to explain why, despite the efforts of practitioners and researchers, estimate accuracy has not improved over time. Theory from the field of organizational politics is used to increase understanding of the empirical findings related to the drivers, manifestations and consequences of informal use of estimates. The results of this research point to six main reasons for persistent estimate inaccuracy in the software industry. First, expectations about the specificity of project managers' estimates are unrealistic: it is not possible to produce point estimates based on uncertain requirements. Second, negotiation among stakeholders with conflicting interests, affects the accuracy of estimates. Third, resources frequently are shared among projects and distortions in one project affect the outcomes of others. Estimates are seldom updated after the initial project phases. Project assessments are based on initial estimates and resulting mismatches between estimates and actuals are often blamed on estimate inaccuracy. Fourth, although feedback is produced when projects are closed it is neither correct nor useful if the recorded estimates and actuals do not correspond to the project carried out. Using this feedback to inform future projects means that inaccuracy is carried forward to those estimates. If the goal is to learn from past mistakes the learning efforts will be based on inaccurate information. Fifth, in order to improve the estimation of software projects, it is necessary to gather information on the current state of practice. This information is collected from practitioners, however, the information about situations that are by practitioners perceived as threatening or embarrassing is often incorrect, with the result that its use as the basis for improvement efforts will have a reduced likelihood of success. Lastly, in the software industry, where product specification uncertainty and development complexity are high, the likelihood of political behaviours, such as informal use of estimates, increases. Project estimates play an important role in decisions about project selection, termination and assessment. As long as they continue to do so, it is likely they will be used to secure project initiation and survival, and to increase the career opportunities of those who produce them. This source of inaccuracy affects software estimates regardless of the estimation method used and it is important to consider it in efforts to improve software practice.