OH4Surveillance Deliverable 3.2 Interim Evaluation Report

Deliverable 3.2 Interim Evaluation Report Dissemination Level: Public Work Package: 3 Evaluation Task: 3.2 Continuous monitoring and evaluation Lead Beneficiary: Statens Serum Institut EU4Heatlh Programme OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or Health and Digital Executive Agency (HaDEA). Neither the European Union nor the granting authority can be held responsible for them. OH4Surveillance I Deliverable 3.2 Page 2 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. Grant Agreement: 101132473 Project Acronym: OH4Surveillance Project Name: Setting up a coordinated surveillance under the One Health approach Title of Deliverable: D3.2 Interim Evaluation Report Dissemination Level: Public Due Date: M18 (June 2025) Work Package: Work Package 3 Task: 3.2 Continuous monitoring and evaluation Lead Beneficiary: Statens Serum Institut (SSI) Author(s): Stine Kjær Lefèvre, Pikka Jokelainen Contributor(s): Consortium members Project Website: https://en.ssi.dk/surveillance-and-preparedness/internationalcoorporation/oh4surveillance Document history Version Author(s) Date Changes 1.0 Stine Kjær Lefèvre, Pikka Jokelainen 17.06.2025 Draft for consortium comments 2.0 Stine Kjær Lefèvre, Pikka Jokelainen 27.06.2025 Final version ready for submission OH4Surveillance I Deliverable 3.2 Page 3 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. Table of contents Document history .................................................................................................................. 2 Executive Summary .............................................................................................................. 4 1. Introduction ....................................................................................................................... 5 2. Methods ............................................................................................................................ 5 3. Evaluation Overview ......................................................................................................... 7 4. Interim Evaluation Highlights ............................................................................................. 8 Action level indicators ........................................................................................................ 8 Project level indicators ....................................................................................................... 8 Need 1: Setting up and scaling up One Health surveillance ........................................... 8 Need 2: Collaboration and sharing to avoid duplication of efforts and to ensure synergies ....................................................................................................................... 9 Surveillance activity level indicators ................................................................................. 10 5. Towards One Health Surveillance: Facilitators and Barriers ............................................ 11 5.1. Facilitators identified ............................................................................................... 11 5.2. Barriers and Challenges Identified .......................................................................... 12 6. Recommendations .......................................................................................................... 14 7. Conclusions .................................................................................................................... 15 OH4Surveillance I Deliverable 3.2 Page 4 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. Executive Summary The OH4Surveillance project adopts a continuous, multi-level evaluation strategy designed to support iterative learning, responsiveness, and long-term sustainability. As detailed in Deliverable D3.1 (Evaluation Plan), the evaluation framework encompasses three interconnected levels. The evaluation activities of OH4Surveillance focus on action level, project level and surveillance activity level indicators. The methods of verification are adapted to each indicator. This interim report outlines the progress of the evaluation activities conducted under Work Package 3 (WP3) of OH4Surveillance. WP3 is dedicated to continuous, formative evaluation throughout the project lifecycle, allowing timely reflection, feedback, and course correction across all work streams. Its aim is to ensure that the project remains aligned with its objectives and maximizes its potential impact. Key evaluation activities to date have focused on a set of objective evaluation criteria, discussed with project partners during the Kick-off Meeting. These criteria, described in the Evaluation Plan (D3.1), serve as the backbone for the continuous monitoring across work packages. WP3 maintains close collaboration with other WPs to embed evaluation practices into routine project activities and foster an integrated learning approach. Preliminary observations and findings suggest that the project is progressing steadily, with several early signs of success and impact. In particular, the usefulness of evaluation is clear. Key identified facilitators include strong engagement from consortium partners, openness to reflective evaluation practices, and the early alignment of evaluation criteria with project deliverables. The planned use of the OH-EpiCap benchmarking tool1 offers a structured avenue for both interim assessment and end-of-project re-evaluation. In addition, several barriers have been identified. These include the varying familiarity with evaluation methods among participants, occasional delays in data flow between WPs, and the complexity of harmonizing evaluation inputs across diverse work streams and country contexts. These challenges are being addressed through iterative coordination and capacitybuilding efforts, and WP3 will continue to refine the evaluation framework as the project evolves. 1 OH-EpiCap Tool https://freddietafreeth.shinyapps.io/OH-EpiCap/ OH4Surveillance I Deliverable 3.2 Page 5 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. 1. Introduction The majority of new emerging infectious diseases that affect humans are zoonoses, infections that can be transmitted between animals and humans. There are many factors that drive the emergence and spread of zoonotic diseases, and it has become evident that animal health, human health and the environment are interconnected. Therefore, there is a pressing need for more rapid and effective responses to zoonotic diseases, which can be achieved with a conceptual shift from siloed approaches towards a One Health approach. The surveillance on the animal health side and in the environment need to be scaled up to set up a One Health surveillance for emerging and re-emerging pathogens. Ideally, this is done in collaboration across countries. The general objective of OH4Surveillance (‘Setting up a coordinated surveillance under the One Health approach’) is to support the participating countries to set up and scale up One Health surveillance to priority pathogens in an efficient, coordinated and collaborative manner. The project includes capacity building and surveillance activities. OH4Surveillance comprises of 11 beneficiaries and 11 affiliated entities and is coordinated by Statens Serum Institut (SSI). The OH4Surveillance project aims to support participating countries in setting up and scaling up coordinated surveillance under the One Health approach, with a specific focus on prioritized emerging and re-emerging zoonotic pathogens. In this context, evaluation plays a central role, not only in assessing progress but also in fostering learning, responsiveness, and alignment across countries, sectors, and work streams. This interim evaluation report covers the first 18 months of the 3-year project and is produced under Work Package 3 (WP3), which leads the evaluation activities of the project. The aim of this report is to summarize the evaluation activities conducted during the first half of the project, highlight key findings and observations, and support strategic reflection across the consortium. The evaluation takes place continuously throughout the project, ensuring that relevant insights are captured and acted upon in real time. 2. Methods Work package 3 of OH4Surveillance is responsible for the evaluation of the project and works closely with other work packages. The key strategy applied is that evaluation takes place throughout the project, thus allowing addressing relevant observations during the project lifetime. In line with this, evaluation was among the key topics discussed at Kick-off meeting of OH4Surveillance, held in Copenhagen, Denmark, on February 27-28, 2024 and the project meeting in Uppsala Sweden 25-26 February 2025. The evaluation activities of OH4Surveillance will focus on action level, project level and surveillance activity level indicators. The methods of verification are adapted to each indicator. The Key Performance Indicators reflect the action level indicators and have numerical target values. The results of the evaluation activities are reported in the Interim Evaluation Report (Deliverable D3.2) and in the Final Evaluation Report (Deliverable D3.3). OH4Surveillance I Deliverable 3.2 Page 6 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. The methodology used to gather evaluation insights reflects the multi-level evaluation framework established in Deliverable D3.1. Insights have been gathered through internal consultations, monitoring of WP activities and consortium discussions, and a questionnaire that was shared during the project meeting in Uppsala in February 2025. The questionnaire was targeting project participants. Questions were open-ended and responses were anonymous. The survey gathered responses from 35 participants representing all partner countries, a wide geographical range and a diversity of institutional perspectives. The questionnaire also supported monitoring of WP4 Sustainability activities and contributed towards Deliverable 4.1 ‘Interim Sustainability report’. WP3 works closely with other work packages to integrate evaluation into routine project work and to support feedback loops for ongoing adjustments. This report lays the foundation for continued learning and performance tracking in preparation for the final evaluation report (D3.3). OH4Surveillance I Deliverable 3.2 Page 7 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. 3. Evaluation Overview The Key Performance Indicators (KPIs) reflect the action level indicators and have numerical target values. Project level indicators are the Specific Objectives that OH4Surveillance aims to reach to address the key needs identified, and the related Key Performance Indicators. The third evaluation level is the surveillance activity level, focusing on specific surveillance activities targeting a specific pathogen or disease. For the surveillance activity level, an established tool for evaluating One Healthness of surveillance systems is used. The tool, OH-EpiCap tool (Tegegne et al., 2023), was developed during the One Health European Joint Programme (European Union's Horizon 2020 Research and Innovation programme, Grant Agreement No. 773830) Joint Integrative Project MATRIX. The tool can be applied to any system where some multi-sectoral collaboration exists at any step of the surveillance, regardless of the level of formalization of the surveillance activity. The evaluation consists of three dimensions, each targeting four indicators. This structured multi-level approach ensures that evaluation is not a one-time activity but an ongoing process that informs project implementation and fosters sustainability beyond the project's lifespan. OH4Surveillance I Deliverable 3.2 Page 8 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. 4. Interim Evaluation Highlights The project is progressing according to the planned timeline and remains on track. All key activities are being implemented as scheduled, and the project continues to meet established quantitative KPIs and formalized performance criteria. At this stage, there are no foreseen deviations to report. Action level indicators Several of the key quantitative aspects such as the number of zoonotic pathogens covered, countries involved, capacity-building initiatives conducted, and samples tested have been already reached. Project level indicators Need 1: Setting up and scaling up One Health surveillance Specific objective 1: “Obtain an overview of current One Health surveillance and surveillance needs for selected priority pathogens in the participating countries” Several respondents affirmed increased understanding, particularly through exposure to challenges in other countries and cross-country comparisons, related to the working group activities. Given the cross-border nature of the targeted zoonotic health threats, addressing Need 1 is effective through international collaboration. Examples of responses: "Yes. Useful to get overview between countries plus what areas need improvement." "Yes, seeing the activities of other countries was helpful." Specific objective 2: “Set up and scale up One Health surveillance activities addressing regional and country-specific needs” Several responses noted existing or improved structures (e.g., "Zoonosis expert network," "More building on existing collaborations"). Others indicated partial establishment or ongoing efforts, reflecting different country-specific starting points. Concerns about funding and critical mass were noted ("No funding for sustained activities," "Need focus on critical surveillance issues"), and challenges to sustainability were mentioned already at this point. These insights reflect real-world barriers to scaling up. Specific Objective 3: “Contribute to harmonising and collaborative approaches across countries to prepare for cross-border health threats” Recurring themes mentioned in the responses included joint policy development, regular meetings, and increased communication. Examples: "Joint policy strategies," "More exchanges with public health authorities," "Funding for these activities." OH4Surveillance I Deliverable 3.2 Page 9 of 15 OH4Surveillance is co-funded by the European Union (Grant Agreement No 101132473). Disclaimer: Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them. Responses further included considerations on funding, enhanced communication, and stronger collaboration between human health and animal health sectors. These responses reflect the underlying systems and needs. The responses offered grounded suggestions for improving interoperability and collaboration, demonstrating that the questionnaire captured input relevant to harmonisation goals. Need 2: Collaboration and sharing to avoid duplication of efforts and to ensure synergies Specific objective 4: “Establish structures and processes for collaboration and sharing of experiences among countries participating in OH4Surveillance” Multiple respondents acknowledged the value of exposure to other countries’ experiences: "Yes, seeing the activities of other countries was helpful." "Useful to get overview between countries plus what areas need improvement." This indicates learning and information flow across the participating countries. The reported frequent involvement in working group meetings and capacity building initiatives suggested structured engagement. These interactions are channels through which shared learning occurs. Both formal and informal processes for sharing experiences across countries are in place under this project and are perceived as valuable, addressing this objective. Specific Objective 5: “Establish structures and processes for collaboration with stakeholders at national, EU and international levels” Responses to questions related to which extent structures and processes for collaboration with stakeholders are in place at national level included: "Yes, Zoonosis expert network," "More building up on existing formal collaborations," "Partly, we have some structures already in place." The comments indicated both existing and evolving collaborations, in particular with nationallevel stakeholders. There was clear evidence of both existing structures and aspirations to deepen collaboration with external stakeholders, supporting this objective. Specific Objective 6: “Establish structures and processes for disseminating experiences to support countries that are not part of OH4Surveillance” This objective is supported by activities coordinated by European Food Safety Authority (EFSA), where the countries involved in OH4Surveillance actively attend. Moreover, the coordinator of OH4Surveillance has been invited to attend consortium meetings of sister consortia.