Transformative innovation policy evaluation: characteristics, challenges, and lessons from practice
Doctoral thesis, 2023

The emergence of the transformative innovation policy (TIP) framing, which builds upon previous science, technology, and innovation policy (STI) approaches to address transformative change or “grand challenges,” implies new challenges for policy evaluation. While most frameworks for TIP evaluation are quite similar to those of previous STI policy evaluation approaches, it is here argued that issues such as capturing the effect of such policies, such as related to the direction of transformative change and its additionality, remains unaddressed in the literature. This PhD thesis focuses on three research questions: (RQ1) From a theoretical perspective, what could be the distinguishing characteristics of an “ideal type” TIP evaluation compared with previous innovation policy evaluation approaches? (RQ2) How can additionality and directionality be accounted for in TIP program evaluation? (RQ3) What are the challenges related to translating an “ideal type” TIP evaluation approach into practice?

RQ1 is mainly addressed by insights provided by Papers I and II. Specifically, Paper I develops a systematic literature review of TIP and identifies challenges for TIP evaluation, while Paper II develops an analytical framework specifying how key evaluation categories differ for an “ideal type” TIP evaluation compared with previous STI approaches. RQ2 is covered mostly by Papers III and IV. Paper III illustrates the development of the framework, which combines insights from theory-based evaluation and TIP to address directionality and additionality. Paper IV complements Paper III by specifying key transformative processes evaluators should look for when evaluating the effects of TIP interventions. RQ3 in turn is covered mainly by Papers II and V. Paper II uses the aforementioned analytical framework to make a cross-case analysis of three Swedish innovation programs funded by Vinnova, the Swedish Innovation Agency: Vinnväxt, Challenge-Driven Innovation (CDI), and the Strategic Innovation Programs (SIPs). Based on this analysis, it identifies three challenges for making evaluations more aligned with an “ideal type” TIP evaluation. Paper V critically analyzes the feasibility of the integrated framework from Paper III by performing an evaluation of the BioInnovation SIP to understand the challenges that emerge for evaluation practice. Finally, the thesis discusses the key characteristics of an “ideal type” TIP evaluation, reflects on the feasibility of the integrated framework for TIP program evaluation, highlights remaining issues for TIP evaluation, and concludes by presenting the key contributions of the thesis.

Additionality

Transformative innovation policy

Directionality

Policy evaluation

Vasa C, Vera Sandbergs Allé 8, 411 33, Gothenburg
Opponent: Johan Schot, Utrecht University, The Netherlands

Author

Carolina Resende Haddad

Chalmers, Technology Management and Economics, Environmental Systems Analysis

Transformative innovation policy: A systematic review

Environmental Innovation and Societal Transitions,;Vol. 43(2022)p. 14-40

Journal article

Haddad, C.R., Wise, E., Arnold, E., 2023. When theory meets practice in transformative innovation policy evaluation: Experiences from Sweden

Evaluating Transformative Innovation Policy Outcomes as Unfolding Processes Of Change In Socio-Technical Configurations

Transformative Metrics: Contributions to the Studies for Monitoring and Evaluating How Science, Technology, and Innovation Can Address Social and Environmental Challenges,;(2022)p. 63-105

Book chapter

Haddad, C.R., 2023. Identifying challenges for the evaluation of transformative innovation policy programmes: insights from the BioInnovation SIP in Sweden.

Previous research has shown that the way we assess policies related to science, technology, and innovation (STI) has not kept pace with the latest ideas about innovation. A new approach called Transformative Innovation Policy (TIP) focuses on addressing societal challenges, such as resource scarcity and climate change, and raises challenges for evaluating these policies. While the methods used to evaluate TIP are similar to the ones used before, issues remain that need to be addressed. For example, there is a need to assess the effects of such policies regarding the additional benefits they bring about and how they facilitate change in society and technologies.

To tackle new evaluation challenges, this PhD thesis first investigates what makes TIP different from previous approaches to evaluating STI policies. Second, the thesis studies how to assess the effect of TIP programs by proposing a framework and identifying key processes evaluators should look at during the evaluation. Lastly, the thesis identifies challenges for translating an “ideal type” TIP evaluation into practice, by looking at some innovation programs funded by the Swedish Innovation Agency, Vinnova. In conclusion, the thesis discusses its contributions to improving the evaluation of TIP, reflects on the practicality of the aforementioned TIP evaluation framework, points out the remaining issues in TIP evaluation, and summarizes its main findings and contributions.

Driving Forces

Sustainable development

Innovation and entrepreneurship

Subject Categories

Other Engineering and Technologies

Economics and Business

Other Social Sciences

ISBN

978-91-7905-936-1

Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 5402

Publisher

Chalmers

Vasa C, Vera Sandbergs Allé 8, 411 33, Gothenburg

Online

Opponent: Johan Schot, Utrecht University, The Netherlands

More information

Latest update

10/24/2023