Why Measure the Impact of Horizon 2020 Projects?
As was introduced in a previous blog post, measuring the impact of projects is more important in Horizon 2020 than it was for its predecessors. Horizon 2020 has new performance indicators to measure the impact of the funded actions. However, these indicators are not considered sufficient to fully capture the outcomes and impacts of projects. Reasons for this are for example that the time between the funding of activities and achieving impact can be several years and the difficulty of attributing impact to a single funding source. It is often difficult to attribute outcomes to funding sources, as most research teams receive funding from multiple and different sources. It is therefore critical to develop the right indicators and ensure the quality and quantity of underlying data is done such that the impact of a project can be measured and attributed to its investments.
The Role of Impact-SC5
IMPACT‐SC5 aims to assess the impacts of the 87 funded RIAs and IAs under the Horizon 2020 Societal Challenge 5 Work Programme 2014-2015, through a multilevel analysis, starting at project level, through portfolio, network and context analyses. It applies a rigorous methodology that enables to capture the various impacts of projects and groups of projects.
Horizon 2020 contributes, via its focus on ‘Societal Challenges’ (SCs), to the ongoing paradigm change in the European Research and Innovation Policy. The pursued objectives are not only to generate economic growth and jobs, but to help resolve pressing socio‐economic challenges, including those related to climate, environment, resource efficiency, and raw materials (SC5). The challenge orientation of Horizon 2020 implies monitoring the progress of the projects towards the achievement of the targeted objectives and impacts.
The scientific, economic and societal/environmental performance of the 87 projects will be measured by means of an extensive set of indicators. A portfolio analysis will complement the project‐level analysis in order to determine how projects perform together in relation to different objectives.
IMPACT‐SC5 will explore impact pathways to identify good practices and draw recommendations for policymakers and potential EU research funding beneficiaries to increase the impacts of future SC5 projects. Particular attention will be paid to the broad and changing policy contexts to ensure a holistic approach for a better understanding of the performance (and the underlying factors) of the 87 projects.
Throughout the project, stakeholder engagement will be strongly fostered. The methodology and findings will be validated with high‐level experts and practitioners. The methodology will be made available to the European Commission for its use in the ex‐post evaluation of Horizon 2020 or future evaluation exercises.
The novelty of the work is that the indicators present the Research & Innovation pathways from short-term project outputs (deliverables) to longer-term expected scientific, societal and economic impact.
Indicators
The project impact indicators that Impact-SC5 is developing will:
- Assess the impacts of all the RIAs and IAs of SC 5 under the 2014-2015 WP through in-depth quantitative and qualitative analysis-87 projects in total.
- Measure and better understand the progress made by and achievements of these projects.
- Compare the projects’ impacts with the expected impact statements for each of the relevant topics.