Some Problems in Quantified Security

Licentiate thesis, 2010

This thesis contains work related to quantitative representation and analysis of computer and information security. The ability to accurately describe security using quantitative methods could offer better control and evaluation of security in operational settings. However, a number of challenges remain, generally in modeling but also in validation and usability. In this work, we improve knowledge about two identified challenges: (i) validation of methods and (ii) decision-making using quantified risk. The first part of the thesis critically surveys many of the proposed methods to quantitatively describe security, by focusing on their validity. After defining a taxonomy, we survey assumptions and methods for validation that have been used in a large fraction of previous work on the subject. We find that many methods lack clear validation with respect to operational environments, and how some model assumptions are not empirically well-supported. We also discuss the characteristics of operational security that make modeling and quantification a remaining challenge. Furthermore, we discuss what future efforts could target in validating quantitative methods for operational security. In the second part we consider a specific type of quantified security: quantified risk, an existing proposal to analyze security quantitatively in terms of probabilities and losses of events. We relate this to the usability of quantified information when people make risky decisions, drawing on previous experimental work in behavioral economics. A common assumption in economic and quantitative analysis of security is that correct knowledge about quantified risk leads to rational decision-making. However, previous experimental results show that people are not always handling quantitative information rationally. We hypothesize that this may impact security decision-making using quantified risk, and study this for two security decision-making problems by a combined theoretical and numerical study. This thesis has two main conclusions. First, validity of many current methods in quantified security is unknown, but there is room for improvement. Second, there are potential decision-making problems in using quantified risk for control of operational security.