Improving quality of summative eHealth evaluations

Doktorsavhandling, 2021

Summative evaluation, which is conducted at the end of an eHealth trial or implementation,
assesses outcomes, produces evidence, and advances knowledge of eHealth implementations
in healthcare provisions. Therefore, its high quality is essential in order to reap the benefits of
the results generated by evaluation studies. Quality is achieved in eHealth evaluation through
contextual sensitivity, adequate research design, adherence to standards, a mixed-method
research approach, and ethical handling of data. However, insufficient quality in eHealth
evaluation studies leads to decision makers and other potential users disregarding their results,
which leads to the resources and effort involved in conducting an evaluation being wasted.
The purpose of this thesis is to study how summative eHealth evaluations can be improved to
support the determination of eHealth value in a specific context as well as the use of evidence
produced during eHealth interventions. This thesis is built on a single case study of a summative
eHealth evaluation of an eHealth implementation project within different healthcare contexts.
The thesis focuses on the different phases of evaluation process, assesses adequacy of standards,
explores value that the eHealth intervention delivered in different contexts, and studies how
evidence from evaluation is further used.
The thesis extends knowledge on eHealth evaluation quality by providing deeper insights into
the problems in the existing quality criteria and by introducing two new criteria for quality in
eHealth evaluations: capturing value of an eHealth solution and involving healthcare
professionals in the intervention and its evaluation. The thesis reveals that meeting some of the
criteria is not always practical, and that evaluators might make trade-offs among the criteria.
The findings point to a need to improve methodologies for eHealth evaluations by providing
better guidance to evaluators and validating evaluation standards in different locations. The
thesis also suggests viewing value of an eHealth solution as a holistic view of the created
monetary and nonmonetary benefits of eHealth that require monetary and nonmonetary
sacrifices in a particular context. In addition, the thesis proposes a model for assessing value of
an eHealth solution.