D4.9 - First annual report on the progress and success of ecosystems

A new, interdisciplinary, multistakeholder European network to improve healthcare services, research and outcomes for Adolescents and Young Adults with cancer. Deliverable Report WP4 – Operation of STRONG-AYA ecosystem, stakeholder and patient involvement, dissemination, exploitation, communication Deliverable D4.9 First annual report on the progress and success of ecosystems. Due date of deliverable: 30/09/2024 Actual submission date: 20/01/2025 Project: STRONG-AYA Lead Contributor Dan Stark Email [email protected] Other Contributors Dan Stark, Oana Lindner, Emily Connearn, Richard Feltbower, Nicola Hughes (UOL); Leonard Wee, Simone Hanebaum (NKI) Emails [email protected].uk; [email protected].uk; [email protected].uk; [email protected]; [email protected]l; [email protected]; Due date 30/09/2024 Delivery date 20/01/2025 Deliverable type R Dissemination level PU Description of Work Version Date Final draft for V1.0 20/01/2025 First draft for review V1.0 31/10/2024 Description: Annual report of the first year of operation of the Strong AYA local, national and international ecosystems. Including a report on the methodology to assess the reliability of Strong-AYA outcomes by assessing data quality and data processing by Strong-AYA systems. 1. Publishable summary (max ½ page) Summary: This document details progress over the past 12 months towards developing the appropriate infrastructure for the functioning of local, national, and a pan-European ecosystem for the generation, analysis and interpretation of data related to AYA living with or after cancer. Deliverable D4.9 aims to describe what actions from each individual roadmap have been successfully defined and achieved to deliver our ecosystems. It also lists which actions are still to be achieved, with risks, mitigations, facilitating processes and metrics to overcome them. We have revised some of our terminology, to refer to a clearer meaning of ‘data ecosystems’, and we have set ourselves a new internal metric for what we consider partly and fully national reach for our ecosystems. Table 2, summarises the detailed findings of this report in relation to our initial objectives and details our next steps for the next 12 months, based on the status of ecosystem development at each of the three levels i.e. locally, nationally and European-wide. Contents 1. PUBLISHABLE SUMMARY (MAX ½ PAGE) ................................................................................................. 3 2. OUR STRONG-AYA ECOSYSTEMS – DEFINITIONS AND INTERNAL METRICS .................................................. 5 2.1 WHAT WE HAVE ACHIEVED ON OUR ECOSYSTEMS, IN THE LAST 12 MONTHS ......................................................... 10 3 METHODOLOGY TO ASSESS THE RELIABILITY OF STRONG-AYA OUTCOMES BY ASSESSING DATA QUALITY AND DATA PROCESSING BY STRONG-AYA SYSTEMS ...................................................................................... 14 3.1 TECHNICAL STANDARDS ............................................................................................................................. 14 3.2 ETHICAL STANDARDS ................................................................................................................................. 14 3.3 STANDARDS AND RULES AROUND DAY-TO-DAY OPERATIONS WITHIN THE ECOSYSTEM ............................................. 15 3.4 RULES ON USING THE ECOSYSTEM FOR FEDERATED LEARNING AND ANALYTICS ....................................................... 15 3.5 KPIS DEFINED .......................................................................................................................................... 15 3.6 MANAGEMENT OF NATIONAL AND INSTITUTIONAL DIFFERENCES ................................................................................ 15 3.7 METHODOLOGY: ...................................................................................................................................... 16 3.7.1 Stakeholder consultations: .................................................................................................................. 16 3.7.2 Stakeholder interviews and surveys: ................................................................................................... 16 4. THE RESEARCH PRIORITIES FOR OUR ECOSYSTEMS ............................................................................... 25 4. PLANS TO EXPAND OUR ECOSYSTEMS FROM 2025 ONWARDS .............................................................. 25 2. Our STRONG-AYA ecosystems – definitions and internal metrics STRONG-AYA brings together an international multi-disciplinary consortium across seven European countries, led by the Netherlands Cancer Institute (NKI) and composed of academic research organisations (European Organisation for Research and Treatment of Cancer (EORTC), University of Southampton, University of Leeds, University of Manchester, Maastricht University, Netherlands Comprehensive Cancer Organisation (IKNL)), clinical partners (Italian National Tumour Institute, Léon Bérard Centre, Gustave Roussy Institute, Maria Sklodowska-Curie National Research Institute of Oncology, the Leeds Teaching Hospitals National Health Service Trust), stakeholder and patient organisations (Youth Cancer Europe, European Cancer Organisation) and a consulting company (FFUND). The integral components of our Strong-AYA ecosystems are: 1) An inter-professional, interdisciplinary and service user community, working together and sharing knowledge, ideas, processes and information that meet the overall purposes of STRONG-AYA. 2) A set of local databases a) To store pseudonymised individual-level patient data in a local private repository, or ‘safe space’ (often termed a Secure Research Environment) within the data governance structure of each partner. Our primary purpose is not to grant external access to these data held locally. Specific pieces of statistical algorithms, generated by partners UNIMAAS and IKNL, will summarise each set of local data, and bring only the aggregated or summary-level analyses from each partner centre together forming an overall analysis. 3) Develop applications through which local data managers will all be able to: a) Prepare, and make data FAIR inside their local ecosystem (so that they remain as data owners ethically and legally). b) Examine their own data to monitor completeness and evaluate data quality (at a level of detail that fits with their permissions). 4) Tools (such as ‘Apps’) that can optimise the knowledge gained by us and others from our work to: a) Run predefined analyses automatically (‘in real time’, ‘on demand’, ‘on the fly’) upon these data. b) Build AYA-specific aspects, into (for example): i) Existing research questions (EuroCare) ii) Existing patient-driven questions (their websites) iii) Existing policy questions (ECO) 5) A sophisticated web-based software application, which a) Serves multiple stakeholders (e.g. clinicians, patients, policy makers) or multiple communities or related stakeholders. b) Provides specific access to an appropriate layer or level of the web space, for each specific individual logging in, according to their own requirements and characteristics. 6) A design and culture that promotes co-operation and engagement, to maximise shared learning. As the technical infrastructure and personnel taking part are only of value alongside the data they hold, we are considering amending our terminology from ‘ecosystems’ to ‘data ecosystems’. Data ecosystems around their data are currently being set up within five participating European nations. Each partner has put in place its own local ecosystem including people, skill-mix, regulatory and approval processes, local retrospective data (often with regional reach), and a plan to create its own local prospective data. The five nations are: • The UK – with centres in Leeds and Manchester • France – with centres in Paris and Lyon • Italy – a centre in Milan • The Netherlands – a centre in Amsterdam • Poland – a centre in Warsaw These are each aspiring to true national reach. However ‘national’ lacks precise definition in this context; much longer-standing data systems often do not have population-based national reach, for many parameters (such as cancer stage, treatment, or relapse status) and even less so for parameters we will collect (such as care received, quality of life and symptoms). Our current initiative is unlikely to rapidly achieve population coverage. Therefore, we have defined what we consider national reach for our national ecosystems, to enable us to map our progress towards that ultimate goal: In very simple operational terms we consider - More than five clinical or population-based data sources within a nation is ‘national’. - Two to five is ‘partly national’. - One data source only is ‘local’. We accept this is an interim and internal metric for ourselves, but we consider it helpful to work against, to ensure appropriate overall progress. Along with this structure to define what we feel a ‘national data ecosystem’ is, we have also worked to define a structure that characterizes local, national and international data ecosystems, to assist us in working, step by step, towards national and international reach: Table 1 Scope City/communityspecific Country-wide Pan-European Stakeholders Local agencies, residents National Government, NGOs Global institutions, multinational bodies Data sources Local data (eg. Municipal) National registries, surveys Crossborder research, global databases Challenges addressed Context-specific issues (neighbourhood level health trends) National priorities (eg disease prevalence) Global challenges (eg pandemic, climate change) Collaboration Regional partnerships Federal-state coordination International coalitions Well-developed systems and discussions to create national-reach existing AYA data access are underway in the Netherlands, France and the UK. Existing national AYA-specific PRO data is limited to the Netherlands (with multicentre national coverage) and partly national in England (with 2 centres contributing existing PRO data and the national register providing a third source of clinical data from its AYA-onset cancer outcomes dataset, but not including PRO data). Prospective data will also be partly national in France, with PRO data from two centres. Achieving national reach more widely is limited by: 1. Delayed engagement of national policy-makers for population-based national AYA cancer data, meaning that data availability is based upon data altruism, which is not as fast or effective as if supported by policy incentives. Were policy-makers to issue guidance that all centres with AYA practices should collect outcomes data based upon the STRONG_AYA COS, then national reach would be facilitated for prospective data. Work to achieve this policy engagement is ongoing in Netherlands, France and England. 2. Cultural and practical differences in the approach to data of various types. For example different parts of our consortium have different medical cultures in relation to the importance of AYA as a specific oncology patient group, and the importance of patient reported outcomes in clinical practice. 3. Delayed confirmation and refinement of the Core Outcome Set for Strong-AYA, resulting in a. Limitations in the testing of ecosystems upon real AYA data. However, tests on dummy data have been entirely secure and effective. b. Limitations in the quality assurance of the CoS data held in each location and nation. The key cross-cutting actions for the next year of our national ecosystems are: (i) to obtain retrospective data with national reach, and (ii) to create and test a roadmap and mechanism for prospective data with national reach. Our pan-European umbrella ecosystem will include the data from each of our 7 local ecosystems, and operate from the Netherlands. The progress of the national ecosystems is also set out in the revised deliverable 4.4 submitted September 2024. Our actions to demonstrate our ambition of achieving truly national reach for our ecosystems is summarised below: Table 2. Milestones towards national ecosystems Country Progress Barriers Action Timeline Milestone UK Agreement with NCRAS for data to be shared and application started Time taken for NCRAS to process requests and provide data. No agreement for NCRAS to have their own Vantage 6 hub Obtain retrospective epidemiology data focusing on age tumour type and survival Winter 2024/2025 Dataset exported from NCRAS to Manchester for FL analysis France HORUS and UniAJA research grant processes ongoing, to collect AJAonset cancer outcomes in multiple French AJA units. The COS in terms of technical feasibility and data federation and evaluation of the acceptability and impact will be assessed within the timescales and context of StrongAYA ahead of discussions with other French AYA units. Test the feasibility and acceptability of the COS collection. Further grant application (IGR-led) Spring 2025 Grant submitted by IGR Italy JANE2 launched in Autumn 2024 No national AYA or PRO collection infrastructure, lack of resource Continue discussions within JANE2 about national federated data Ongoing throughout programme duration Netherlands Existing infrastructure is in place, with existing national data Bureaucratic delays due to filing an amendment to existing study to collect additional STRONG AYA core outcome set not already collected as part of Dutch COMPRAYA study (anticipated January 2025) Ethics amendments Winter 2024/5 Data sharing confirmed in writing for secondary use in STRONGAYA of existing COMPRAYA and SURVAYA dataset Poland Discussion with team at Maria Sklodowska-Curie National Research Institute of Oncology. It is not possible to integrate national data into the programme due to national legal and IT infrastructure constraints and at a local level Showcase the novel integration of the local PROMs data with the clinical data to provide a Ongoing throughout the programme duration. Integration of existing AYA cancer fertility PRO data into Vantage6 Node at Warsaw budgets, capability and resource capacity limitations. So only core data from the centre can be provided prospectively. There is not a culture in Poland of collecting and using patient reported outcomes in clinical practice. template of what is realistic and achievable for other centres to follow in the future. This could be done through presentations on Strong-AYA at national conferences , leaflets promoting the work of Strong-AYA and cooperation with advocacy and professional groups. data can or cannot be used for federated learning or analyses. There is an expectation that not all countries and institutions will be able to fully meet all requirements (technical, administrative etc) and there will be gaps in policies. The ethos of a collaborative pan-European ecosystem is that these gaps can be bridged as part of the working practices. A task within the ecosystem needs to be the transparent communication and discussion of these potential differences and gaps to resolve them or to develop secondary plans to achieve explicit goals. 3.7 Methodology: 3.7.1 Stakeholder consultations: Stakeholder consultations have taken place within the community of our ecosystem to answer any questions and to seek input on how to enhance the value of the following: Table 4 – Stakeholder Consultations Study aspect Stakeholders consulted Date of interaction Use cases Patient representatives Healthcare Professionals Project Partners Consortium members Policy Makers Researchers December 2023 through online surveys IT and Data management local support None required based upon September 2024 actions required. NA Data flows into the data node NKI CLB IGR UoM NIO UoL INT March 2023 February 2024 April 2024 May 2024 July 2024 January 2025 To follow Healthcare Professional and Patient portals Healthcare Professionals Patient representatives Consortium members September 2024 through an online workshop Ethical guidelines and code of conduct Strong-AYA partners Up to September 2023 and fed into D2.2 submitted September 2023 Table 3. Details of stakeholder consultations on the status of local and national ecosystems for D4.9. 3.7.2 Stakeholder interviews and surveys: Ecosystem centres were provided with a questionnaire and followed up with a virtual meeting to provide a comprehensive overview of the progress that has been achieved in the past 12 months and to discuss the barriers and challenges still present. The questionnaire and discussions focused on the following questions: 1. Do you have the relevant, correct and ethically approved consents in place in your organisation for the ecosystem? 2. What type of cultural/language based challenges have you identified for people accessing information about Strong-AYA? 3. The aim is to have retrospective data available in the ecosystems by the end of September 2024. Has this data been identified and ready to be made available to Strong-AYA? 4. Has the electronic location where the data you collect will be stored been set up? Is this available for data to be input? 5. Do you have the relevant people in place to manage the ecosystem – data mangers, technical staff? 6. Do you have the staff in place to recruit new patients for the PRO’s and have you identified where you will recruit the patients from? 7. Have local consent forms to recruit the PRO patients been drafted or approved? 8. Do you have patient information sheets available for patients about the study? 9. What actions do you believe are outstanding to have the best local ecosystem in place? 10. Do you believe you are lacking any skills or capacity to set up the ecosystem? 11. What risks of delay or non-completion can you see locally, that might be relevant to StrongAYA and do you have mitigation plans in place? Based upon the questionnaires, a summary of responses has been tabulated to provides an overview of the milestones achieved across the centres towards achieving the three levels of ecosystem (local, national and international). This is detailed below and includes the next steps required which will form the basis of the actions that will be reported on in D4.10. Table 5 – Details of centre-specific milestones achieved in the past 12 months Ecosyste m partner Ethical approvals retrospecti ve data Ethical approvals prospectiv e data Infrastructure agreement Availability of retrospecti ve data in data node Research staff capacity Data node available Dedicated IT hardware IT Staff Outstanding tasks and methods Lead and Timelines Lyon No retrospecti ve PRO data so ethical approvals not needed Awaits the completed CoS Translatio n The data processing agreement is being circulated at present. No retrospecti ve PRO data available. Identified that more staff are needed to enter the data. AYA nurses are available Vantage6 node has been connecte d to the central Vantage6 server at Medical Data Works A dedicated server has been identified with support av ailable and up to date in terms of IT security standards. Project Managers are in place. Patient information sheets need to be made available in the local language. Ethics application for prospective PROs. Questionnaire s and Protocol need to be submitted in local language. No PRO collection solution currently in clinical use in AYA – will use an existing local in-house platform for data collection. PIS expected to be in draft in Winter 2024/25 Ecosyste m partner Ethical approvals retrospecti ve data Ethical approvals prospectiv e data Infrastructure agreement Availability of retrospecti ve data in data node Research staff capacity Data node available Dedicated IT hardware IT Staff Outstanding tasks and methods Lead and Timelines Instituto nazional e dei tumori Meeting with the DPO have been conducted and agreement s set for the project. Awaits the CoS translation The data processing agreement is being circulated at present. No retrospecti ve PRO data is available. PRO data will not be collected prospective ly. No new staff have been recruited until now that the COS is defined. Not at present. Clinical data collection will be performed in RedCap and transcoded in OMPO an internal server. Staff are not in place and has been identified as an outstandi ng action. IT and human resources staff are needed at clinical and IT level. Staff recruitme nt expected Data manager starting to work from January 2025 Gustave Roussy IRB approval has been obtained however DPO needs final COS to implement the data. No. Cannot progress without the COS translation and list of covariables. This is now in place (December 2024) but work The data processing agreement is being circulated at present. Some retrospecti ve data has been identified but final COS is needed for further progress A clinical research associate has been recruited for 24 months to consent the patients Vantage6 node has been connecte d to the central Vantage6 server at Medical Data Works A dedicated server has been identified with support available and up to date in terms of IT security standards. IT teams are in place to extract clinical data items automatic ally. Quality checked will be complete d by a Patient information sheets need to be made available in the local language. Obtaining CPP approval for the prospective data collection PIS expected in Spring 2025 following it is delayed Data Manager. There is a lack of an existing technical infrastructur e to collect the PRO data. A platform exists to collect breast cancer data which may be adaptable. A new platform will be costly and will require substantial resource to set up. Warsaw Yes the consents are in place for the retrospecti ve sexNo. Ethics approval can not be applied for until the COS is in The data processing agreement is being circulated at present. Source data has been identified. Two clinics have been identified from which to recruit patients. Staff are in Vantage6 node has been connecte d to the central Vantage6 Need to esta blish a secure research data environment to put the local data. Staff are in place at present to manage the ecosyste m. There Patient information sheets and questionnair e need to be made availabl e in Ethical approval is being applied for as COS is in place – health project. the local language. place to do this. server at Medical Data Works is no budget to sustain the ecosyste m after the project ends the local language. Support is requires from the wider consortium’s expertise to collect the data in a very specific structure NKI Yes the consents are in place and data has started to be moved into the StrongAYA node. The application was sent in July 2024. The MDW agreement has been signed. The data processing agreement is being circulated at present for all parties to sign. Retrospecti ve data from COMPRAYA and SURVAYA have been identified and work has started to move COMPRAYA data into the node Access to patients is in place due to the COMPRAYA study. Applying for secondary use of the COMPRAYA data for the prospective patients. Vantage6 node has been connecte d to the central Vantage6 server at Medical Data Works Yes a server has been identified. However a dedicated server is being sourced for the longer term. All staff in place except a Data Manager who will be recruited imminentl y. Submit amendment for retrospectiv e data to include prospective data. Draft Patient Information Sheets Seek longterm server solution for FL projects. Since the COS is defined December 2024 this is nearing completio n Early 2025 No available timeline Recruit A Data Manager Add data to lakehouse at and ensure the data is made FAIR for interoperabilit y Early 2025 Early 2025 The Christie Not yet, UNIMASS are in discussions about the metadata structure. The work is being led by UoL and is ongoing. The infrastructure is already in place and has been used on another federated data analysis project previously. The data processing agreement is being circulated at present. The retrospecti ve data has not yet been made available. Discussions are ongoing with UNIMASS. Data Managers are in place however staff to recruit patients will be put in post when the HRA approval has been granted. Vantage6 node has been connecte d to the central Vantage6 server at Medical Data Works An existing UKCAT infrastructure is in place. IT staff are in post. HRA approval has not been given yet for the prospective data collection. National Data ecosystem ongoing discussions. Recruitme nt will be in place once HRA approval given HRA approval awaited January 2025 Septembe r 2027 Universit y of Leeds The retrospecti ve clinical Work is ongoing to finalise the The data processing agreement is The retrospecti ve data is The staff are all in place for Vantage6 node installed Dedicated IT staff are in place. A manual check of the retrospectiv HRA approval is expected data has the approvals in place and the retrospecti ve PRO data is being individually checked to ensure secondary use of data permission has been given. document ation needed to submit the UK-specific ethics application . Sponsor approval sought, and comments responded to September 2024. Some elements are outstandin g for the relevant approvals awaiting the final CoS being circulated at present. not available in the node however the retrospecti ve data has been identified and work is underway to move it across. the manageme nt of the ecosystems and the recruit new patients. January 2025 e PRO studies to ensure ethics approval present for secondary use of the data. Ethics approval for prospective data collection. The final migration to the TRE server. Further investigation into how to reach less well served populations through translated questionnair es and availability in alternate formats in January 2025 Table 6 Infrastructure status update Gustave Roussy Centre Leon Berard Instituto nazionale dei tumori University of Leeds The Christie Hospital Manchester Netherlands Cancer Institute Narodowy Instytut Onkologii Nation France Italy UK Netherlands Poland Data station created Expected 2025 Service user agreement with secure connectivity provider (Medical Data Works) Expected 2025 Tested end-to-end encrypted federated query Expected 2025 Tested connectivity with secure connectivity provider passed Expected 2025 Tested interoperability/FAIRification tool functionality Expected 2025 Table 6. Technological infrastructure status at national levels, building towards a pan-European infrastructure. 25 4. The Research Priorities for our ecosystems Key questions that participants wish to use the Strong-AYA data for were identified during the initial consultations during 2023. These research priorities have progressed to form the initial Use Cases. The top three use cases have been identified as: 1. What proportion of AYA patients had a Multi-Disciplinary Team clinical evaluation of their case prior to commencing clinical treatment/management 2. Did the patient experience psychological problems during and after treatment? 3. What proportion of AYA cancer patients survive at 1, 3, 5 years post-treatment? The methods and processes used to establish the use cases were outlined previously (Deliverable 4.8.). In the next 12 months, with WP2, we propose to test these and other use cases against novel ethical data frameworks, to ensure our work is delivered with optimal proactive consideration of data security and ethical pitfalls. 5. Plans to expand our ecosystems from 2025 onwards The Cancer Mission 01-05 call was for survivorship research when cancer is diagnosed in AYA aged 15-39 years. Two proposals were submitted led by STRONG-AYA beneficiaries, which included creating new datasets for use within the Strong-AYA ecosystem. These applications for the call would seek to build upon and expand the existing STRONG AYA