D6.2: Evaluation framework, plans and material - an update. Deliverable of the Horizon-2020 PANACEA project, Grant Agreement No. 953426

Report, 2023

This deliverable presents the evaluation framework, plans and material for all data collections connected to work package 6 (WP6) of the PANACEA project. It describes the objectives of the studies and how they will be realised. The purpose of the PANACEA evaluation framework is to create a common framework to be used in all studies to make sure the data are collected in a way that makes it possible to consolidate the results at the end and to provide what is needed for impact analysis (WP7). The first version of the deliverable (D6.1: ‘Evaluation framework, plans and material´) had its focus on setting the framework and the work process. In this updated version, the focus is on the evaluation protocols for all studies, including templates for the pilot sites, questionnaires to use, key performance indicators (KPI), log files to use, crucial timelines, etc. The experimental plans are described per pilot site and type of evaluation activity. The key content of D6.2 is structured as follows:

Chapter 1 is the introduction to the deliverable, specifying its purpose, the intended audience, and interrelations with other project activities. Chapter 2 introduces the project objectives related to the WP6 data collections. Chapter 3 provides a brief overview of each Use Case and Chapter 4 presents the various studies within the project including descriptions of the main actors, environment, vehicles, PANACEA sensors/technologies, and countermeasures. Chapter 5 describes the PANACEA evaluation framework. Chapters 6-15 then describe the steps defined in the evaluation framework. Chapters 6-11 include the planning phase and present the Use Case Scenarios, Research Questions, Key Performance Indicators, study designs, data gathering tools, and data analysis plan. Chapters 12-13 describe the implementation phase, including pilot site preparations, and data collection. Chapters 13-15 describe the data analysis phase and includes chapters about data delivery, data analysis, results reporting, results consolidation, and impact assessment. Lastly, Chapter 16 provides the conclusions of the deliverable.

The deliverable presents both a horizontal perspective of the pilot sites as well as more detailed descriptions of what will be included in the different studies. The main text of the deliverable provides an overview of all studies and evaluations within PANACEA. Research questions and KPIs are defined for each study (Appendix III). The general data gathering tools (objective and subjective) are identified. The questionnaires used for the evaluations are included in Appendix IV. A set of guidelines on practicalities and ethical aspects to take into consideration before and during data collection are presented. Experimental plans for all WP6 data collections are included as appendices to the deliverable (Appendix II).