Quantified Security is a Weak Hypothesis: A critical survey of results and assumptions

Paper in proceeding, 2009

This paper critically surveys previous work on quantitative representation and analysis of security. Such quantified security has been presented as a general approach to precisely assess and control security. We classify a significant part of the work between 1981 and 2008 with respect to security perspective, target of quantification, underlying assumptions and type of validation. The result shows how the validity of most methods is still strikingly unclear. Despite applying a number of techniques from fields such as computer science, economics and reliability theory to the problem it is unclear what valid results exist with respect to operational security. Quantified security is thus a weak hypothesis because a lack of validation and comparison between such methods against empirical data. Furthermore, many assumptions in formal treatments are not empirically well-supported in operational security and have been adopted from other fields. A number of risks are present with depending on quantitative methods with limited or no validation.