Advancing formative control room system evaluation – Decision support for human factors evaluation planning and method development

Doctoral thesis, 2018

The design of the nuclear power plant (NPP) control room system affects the operation of the plant it controls, as well as the well-being of its operators. One important activity in control room system development is Human Factors (HF) evaluation. Previous research indicates that HF evaluation practice for NPP control room systems can be improved. For example, there is a need for methods that are flexible and simple to use. In order to advance evaluation practices as part of the development process the purpose of this thesis was to increase understanding of HF evaluation of NPP control room systems. The main object of study was the evaluation activity. The first research question concerned the aspects that need to be assessed so as to be able to evaluate the control room system’s ability to fulfil its intended purpose. The second research question asked if, and how, HF evaluation can better support control room system development. The methods used were two literature studies and empirical studies in the form of an interview study, three case studies, and three focus groups.

In response to the first research question, the interview study investigated those aspects of the NPP control room system that contribute to safe operation. In the first literature study these aspects were used together with aspects found in other studies to identify categories of measures relevant for assessing NPP control room systems. The identified categories – system performance, task performance, use of resources, user experience, and identification of design discrepancies – complement each other and should all be included in control room system evaluation during the course of the development process. In response to the second research question, the second literature study identified a gap in today’s evaluation practice and the research efforts focused on formative evaluation of more general (higher-level) design decisions, preferably undertaken early in the development process. A combination of two methods, heuristic evaluation and scenario-based talkthrough, was used in the case studies and focus groups to explore the evaluation activity in practice. This method combination was found to be useful for formative assessment of higher-level design decisions in NPP control room systems. In addition, HF specialists from other domains who participated in the focus groups believed that the method combination would be useful outside the nuclear power domain too. A description of the method combination is included in the thesis to provide concrete guidance for HF practitioners. The experiences from the case studies were also used to identify guidelines for developing HF evaluation methods that are useful in practice.

From the knowledge gained through exploration of the research questions five perspectives that provide decision support in HF evaluation planning and method development emerged: 1) the purpose of the evaluation activity, 2) the object to be evaluated, 3) the tactic used in the evaluation activity, 4) the evaluation procedure, and 5) the use of the evaluation method. Individual results from the studies, such as the categories of measures and guidelines for developing methods that are useful in practice, can be used as more detailed support within these perspectives.