Improving quality of summative eHealth evaluations

Doktorsavhandling, 2021

Summative evaluation, which is conducted at the end of an eHealth trial or implementation, assesses outcomes, produces evidence, and advances knowledge of eHealth implementations in healthcare provisions. Therefore, its high quality is essential in order to reap the benefits of the results generated by evaluation studies. Quality is achieved in eHealth evaluation through contextual sensitivity, adequate research design, adherence to standards, a mixed-method research approach, and ethical handling of data. However, insufficient quality in eHealth evaluation studies leads to decision makers and other potential users disregarding their results, which leads to the resources and effort involved in conducting an evaluation being wasted.
The purpose of this thesis is to study how summative eHealth evaluations can be improved to support the determination of eHealth value in a specific context as well as the use of evidence produced during eHealth interventions. This thesis is built on a single case study of a summative eHealth evaluation of an eHealth implementation project within different healthcare contexts.
The thesis focuses on the different phases of evaluation process, assesses adequacy of standards, explores value that the eHealth intervention delivered in different contexts, and studies how evidence from evaluation is further used.
The thesis extends knowledge on eHealth evaluation quality by providing deeper insights into the problems in the existing quality criteria and by introducing two new criteria for quality in eHealth evaluations: capturing value of an eHealth solution and involving healthcare professionals in the intervention and its evaluation. The thesis reveals that meeting some of the criteria is not always practical, and that evaluators might make trade-offs among the criteria.
The findings point to a need to improve methodologies for eHealth evaluations by providing better guidance to evaluators and validating evaluation standards in different locations. The thesis also suggests viewing value of an eHealth solution as a holistic view of the created monetary and nonmonetary benefits of eHealth that require monetary and nonmonetary sacrifices in a particular context. In addition, the thesis proposes a model for assessing value of an eHealth solution.