D1.2 - Report on the initial Clinical Outcome Set

1 A new, interdisciplinary, multi-stakeholder European network to improve healthcare services, research and outcomes for Adolescents and Young Adults with cancer. STRONG-AYA – No. 101057482 – D1.2 Deliverable Report WP1 – Report on the initial Core Outcome Set Deliverable 1.2 Report on the initial Core Outcome Set Due date of deliverable: 30/09/2023 Actual submission date: 28/03/2024 Project: STRONG-AYA Lead Contributor UOSKirsty Way, Anne-Sophie Darlington, Nicole Collaço Email [email protected] Other Contributors NKI – Silvie Janssen Emails [email protected]l Due date 30 09 2023 Delivery date 28 03 2024 Deliverable type R Dissemination level PU Description of Work Version Date V1.0 27/03/2024 STRONG-AYA – No. 101057482 – D1.2 Description: This report contains the initial Core Outcome Set, including the methodology that took place for its development. Results from the literature review, qualitative interviews, Delphi survey rounds as well as the consensus meeting have been reported to show the pragmatic decision-making processes behind the development of the Initial COS summary (max ½ page) The aim of Work Package 1 (WP1) of STRONG AYA is to develop a Core Outcome Set (COS) for adolescents and young adults (AYAs) with cancer. There is specific scientific methodology that must be followed to develop a COS [1-4], as well as identify the measurements instruments to measure the items in the COS [5]. The stages for making decisions on the final outcomes to be included in a COS are three-fold: (1) A review of the literature to discover which outcomes are important for AYAs with cancer, (2) qualitative interviews with stakeholders (AYA patients, their caregivers, and healthcare professionals) to hear their perceptions of what outcomes are important for AYAs with cancer, (3) consensus methodology, known as a Delphi study. Following a review of the literature and qualitative interviews, a total of 129 outcomes were accumulated and considered important for AYAs with cancer. These 129 outcomes were then categorised in accordance with Dodd’s 38-item item taxonomy for outcomes in medical research [6], which formed the skeleton of the Delphi survey. The Delphi survey has a total of three rounds, which aims to come to a consensus between stakeholder groups on which outcomes are the utmost important to measure for AYAs with cancer. Following the first two rounds of the Delphi survey, a total of 61 items met the inclusion criteria (outcomes that received ≥70% respondents rating it as a 7-9 on the Likert-scale) and were taken forward to a consensus meeting, which involved 12 members of the STRONG AYA consortium forming an expert panel to make pragmatic decisions on the final outcomes to be included in the Initial COS. The Initial COS consists of 21 domains; there are 15 CORE domains, 3 ON TREATMENT domains and 3 OFF TREATMENT domains. The development of the Core Measurement Set to supplement the initial COS is still being developed. Round 3 of the Delphi survey is still ongoing; results from the third and final Delphi round will be used to inform the Final COS. In the meantime, the Initial COS will be implemented throughout the healthcare research ecosystem in future work packages in the STRONG AYA project STRONG-AYA – No. 101057482 – D1.2 1 Table of Contents PUBLISHABLE SUMMARY (MAX ½ PAGE) .............................................................................................................. 3 1 TABLE OF CONTENTS ..................................................................................................................................... 4 2 DEFINITIONS ................................................................................................................................................. 5 3 ABBREVIATIONS............................................................................................................................................ 6 4 INTRODUCTION ............................................................................................................................................ 7 PROJECT BACKGROUND ...................................................................................................................................... 7 DELIVERABLE INTRODUCTION .............................................................................................................................. 9 5 DEVELOPING THE INITIAL CORE OUTCOME SET............................................................................................ 10 BACKGROUND ............................................................................................................................................................. 10 METHODS .................................................................................................................................................................. 10 Phase 1: Scope definition .................................................................................................................................... 11 Phase 2: Establishing the need for a COS for AYAs ............................................................................................. 11 Phase 3: Composition of AYA Working Group .................................................................................................... 11 Phase 4: Study protocol development ................................................................................................................ 12 Phase 5: Determine “what” to measure – the outcomes in the COS .................................................................. 12 Phase 6: Determine “how” to measure the COS ................................................................................................. 21 Determine “case-mix variables” ......................................................................................................................... 23 RESULTS .................................................................................................................................................................... 23 The Literature Review ......................................................................................................................................... 23 Qualitative Interviews ........................................................................................................................................ 24 The Delphi survey ............................................................................................................................................... 25 FUTURE WORK ............................................................................................................................................................ 30 6 CONCLUSION ............................................................................................................................................... 31 7 REPOSITORY FOR PRIMARY DATA (ANNEX) ................................................................................................. 31 STRONG-AYA – No. 101057482 – D1.2 2 Definitions STRONG AYA consortium members are referred to as following within this text: 1. NKI-AVL – Stichting het Nederlands Kanker Instituut – Antoni van Leeuwenhoek Ziekenhuis (NL) 2. YCE – Youth Cancer Europe (RO) 3. INT – Fondazione IRCCS Instituto Nazionale dei Tumori (IT) 4. FFUND – FFUND BV (NL) 5. CLB – Centre de Lutte Contre le Cancer Leon Berard (FR) 6. ECO – European Cancer Organisation (BE) 7. UNIMAAS – Universiteit Maastricht (NL) 8. IKNL – Stichting Integraal Kankercentrum Nederland (NL) 9. EORTC – European Organisation for Research and Treatment of Cancer AISBL (BE) 10. IGR – Institut Gustave Roussy (FR) 11. MSCNRIO – Narodowy Instytut Onkologii im. Marii Sklodowskiej-Curie – Panstwowy Instytut Badawczy (Marie Sklodowska-Curie National Research Institute of Oncology) (PL) 12. UOM – University of Manchester (UK) 13. UOL – University of Leeds (UK) 14. LTHT – Leeds Teaching Hospitals National Health Service Trust (UL) 15. SOUTHAMPTON – University of Southampton (UK)  Grant Agreement (including its annexes and amendments): the agreement signed between the beneficiaries of the HORIZON Research and Innovations Actions (hereafter referred to as Horizon) and the European Health and Digital Executive Agency (hereafter referred to as HADEA) for the undertaking of the STRONG AYA project (Grant Agreement no. 101057482).  Beneficiary: Signatories of the Grant Agreement  Associated Partner: Entities which participate in the action but without the right to charge costs or claim contributions.  Project: the sum of all activities carried out in the framework of the Grant Agreement.  Consortium: the STRONG AYA consortium, including all the aforementioned partners.  Consortium Agreement: The agreement made between STRONG AYA members for the implementation and execution of the action outlined in the Grant Agreement. The agreement shall not affect the parties’ obligations to HADEA on behalf of the European Union, and/or to one another arising from the Grant Agreement. STRONG-AYA – No. 101057482 – D1.2 3 Abbreviations Acronym/Abbreviation Meaning HCP Health Care Provider PRO Patient Reported Outcome PROM Patient Reported Outcome Measure COS Core Outcome Set WP Work Package WPL Work Package Lead(s) WP1 Work Package 1 (Development Core Outcome Set AYA with cancer & data collection) WP2 Work Package 2 (Governance, Data Security and Ethics) WP3 Work Package 3 (Infrastructure and Interoperability) WP4 Work Package 4 (Operation of STRONG AYA ecosystems, stakeholder and patient involvement, dissemination, exploitation, communication) WP5 Work Package 5 (Scientific coordination and project management) KPI Key Performance Indicator OA Open Access PAB Patient Advisory Board EC European Commission HADEA European Health and Digital Executive Agency SC Steering Committee MT Management Team STRONG-AYA – No. 101057482 – D1.2 4 Introduction Project background Cancer at adolescent and young adult (AYA) age is rare, although 4-6 times more frequent than paediatric cancer (i.e. prepubescent period). However, this rarity does not reflect the significant personal and societal costs of cancer in this population, as reflected in the potential years of life lost or saved, the decreased productivity and quality-of-life due to the impact of the disease during formative years, and the long-term complications or disabilities1. AYAs with cancer form a unique group; they face age-specific issues (e.g. Infertility, unemployment, financial problems) and decreased quality of life due to cancer and its treatment. Unlike dedicated healthcare and trials for paediatric cancer patients, AYA-specific healthcare services are scarce and vary across Europe. AYAs who are at the core of society and economy need access to age-appropriate and high-quality healthcare. Defining AYAs with cancer as 15 to 39 years at initial cancer diagnosis2, their annual cancer incidence is 42.2/100.000, with 156.431 cases in Europe and 1.231.007 cases worldwide reported in 2018 (together 6.8% of all cancers)3. Population-based data from 27 European countries supports that AYAs have lower survival than children but higher than adults affected by cancer. Advances in cancer treatment have led to increased survival rates for AYAs with cancer, improving by 82% for all cancers between 1990 and 20074. However, survival improvement in AYA is more challenging than for children and older cancer survivors, which might be due to the fact that AYA have the highest absolute excess risk of second primary malignant neoplasms5. AYA face some distinct challenges given that they do not belong to neither paediatric nor adult oncology groups. Characteristic features of this population group include unique spectrum of cancer types, different tumour biology, unique complex psychological needs, distinct late sequelae, including impaired fertility, and palliative care. These traits imply that clinical management, treatment, diagnosis, psychological support will need to be designed and developed for AYA’s specific needs. For example, AYAs diagnosed with breast and prostate carcinomas have worse survival than older patients because of the biological differences between them, highlighting the need to target screening methodologies, treatment and policies to their needs6. AYAs with cancer also face significant psychological challenges, including substance abuse, mental health issues, suicidal ideations and increased emotional burden from cancer and cancer-related morbidity. Finally, tailoring cancer care to AYA’s needs is difficult and due to many different complex factors including the low rate of participation by AYAs in clinical trials and cancer research7. 1 Stoneham SJ. AYA survivorship: The next challenge. Cancer 2020; 126: 2116-2119. 2 Adolescent and Young Adult Oncology Review Group. Closing the gap: Research and Care Imperatives for Adolescents and Young Adults with Cancer. National Institute of Health, National Cancer Institute, and Livestrong Young Ault Alliance: Bethesda, MD, USA, 2006. 3 Trama A, Botta L, Steliarova-Foucher E. Cancer Burden in Adolescents and Young Adults: A Review of Epidemiological Evidence. Cancer J 2018; 24: 256266 4 Trama A, Botta L, Foschi R et al. Survival of European adolescents and young adults diagnosed with cancer in 2000-07: population-based data from EUROCARE-5. Lancet Oncol 2016; 17: 896-906. 5 Keegan THM, Bleyer A, Rosenberg AS et al. Second Primary Malignant Neoplasms and Survival in Adolescent and Young Adult Cancer Survivors. JAMA Oncol 2017; 3: 1554-1557. 6 Stark D, Bielack S, Brugieres L et al. Teenagers and young adults with cancer in Europe: from national programmes to a European integrated coordinated project. European Journal of Cancer Care, 25(3), 419–427. 7 Hayashi RJ. Adolescent and young adult cancer survivorship: The new frontier for investigation. Cancer 2019; 125: 1976-1978. STRONG-AYA – No. 101057482 – D1.2 Despite the increasing awareness and a growing body of the scientific literature, these unique issues remain to be fully recognised and addressed by the European health systems and AYAs with cancer are frequently underserved. In part, this may have resulted from the traditional dichotomy between the integrated paediatric (“patient/family-centred”) care services versus dispersed (“diseasecentered”) adult oncology services8,9,10. Up to half of AYAs with cancer report unmet informational and service needs, impacting their direct (survival rates) and indirect (long term effects and mental health) recovery to participate in society11. Furthermore, aligned with this barrier is the low rate of health care utilisation by AYAs, especially primary care, given their challenges to sustain health insurance coverage12. According to a survey conducted through the networks of the AYA Working Group of the European Society for Medical Oncology (ESMO) and the European Society for Paediatric Oncology (SIOP Europe), 67% of practitioners do not have access to specialised centres for AYA with cancer, 67% had no access to a specialist cancer service for late effects management and 38% had no access to fertility specialists. Under-provision and inequality of AYA cancer care is common across Europe and especially in the Eastern and Southern-East. Furthermore, the Working Group also reported an absence of outcome measures for monitoring and evaluating AYA cancer care programs and control13. Among the recommended future steps, it has been identified that one of the most important contributions to AYA research would be to pool data (e.g. patient-reported outcomes, clinical and treatment data) across institutions and countries and create large cohorts for researchers to Address the burden of cancer in AYA14. There is a lack of data standardization, data interoperability and (prospective) collection of outcomes of relevance for AYAs with cancer. The STRONG-AYA project aims to tackle the underrepresentation of AYA’s experiences and outcomes when navigating the healthcare system and in clinical care by developing national infrastructures for outcome data management and clinical decision-making within a pan-European ecosystem and establishing communication feedback for AYAs with cancer and the healthcare systems. This will be key to improving healthcare services, research, outcomes and policies for AYAs and to ultimately better cancer care for this patient group. To this aim, STRONG-AYA brings together an international multi-disciplinary consortium across seven European countries, led by the Netherlands Cancer Institute (NKI) and composed of academic research organisations (European Organisation for Research and Treatment of Cancer (EORTC), University of Southampton, University of Leeds, University of Manchester, Maastricht University, Netherlands Comprehensive Cancer Organisation (IKNL)), clinical partners (Italian National Tumour Institute, Léon Bérard Centre, Gustave Roussy Institute, Maria Sklodowska-Curie National Research Institute of Oncology, the Leeds Teaching Hospitals National Health Service Trust), stakeholder and patient organisations (Youth Cancer Europe, European Cancer Organisation) and a consulting company (FFUND). 8 Ferrari A, Stark D, Peccatori FA et al. Adolescents and young adults (AYA) with cancer: a position paper from the AYA Working Group of the European Society for Medical Oncology (ESMO) and the European Society for Paediatric Oncology (SIOPE). ESMO Open 2021; 6: 100096. 9 Osborn M, Johnson R, Thompson K et al. Models of care for adolescent and young adult cancer programs. Pediatr Blood Cancer 2019; 66: e27991. 10 Fardell JE, Patterson P, Wakefield CE et al. A Narrative Review of Models of Care for Adolescents and Young Adults with Cancer: Barriers and Recommendations. J Adolesc Young Adult Oncol 2018; 7: 148-152. 11 Keegan TH, Lichtensztajn DY, Kato I et al. Unmet adolescent and young adult cancer survivors information and service needs: a population-based cancer registry study. J Cancer Surviv 2012; 6: 239-250. 12 Hayashi RJ. Adolescent and young adult cancer survivorship: The new frontier for investigation. Cancer 2019; 125: 1976-1978. 13 Saloustros E, Stark D, Michailidou K et al. Report on ESMO/SIOPE European Landscape project key results: Mapping the status and needs in AYA cancer care. Late-breaking and deferred publication abstracts public health 2017; 28, 5: V643. 14 Smith AW, Seibel NL, Lewis DR et al. Next steps for adolescent and young adult oncology workshop: An update on progress and recommendations for the future. Cancer 2016; 122: 988-999. STRONG-AYA – No. 101057482 – D1.2 Building on previous initiatives, a STRONG-AYA data ecosystem will be set up for value-based care, research and policy for AYA with cancer by: 1. Developing a Core Outcome Set (COS) specifically for AYAs with cancer, via a participative consensus process defining most important aspects for those directly affected by AYA cancer, including patients and healthcare professionals. 2. Implementing the COS across several national European healthcare systems. Data will be collected at local level and will then be included in a data integration platform. An overall ecosystem framework for data analytics and output will be created supporting federated analyses and the creation of reports across clinical and patient-reported data and making national repositories of (patient-reported) health data, available to individual patients, patient organisations, regulatory authorities, as well as the patients’ health care providers to inform clinical decision-making. The five resulting national ecosystems will be connected to each other into the pan-European ecosystem using a federated approach also utilizing the overall ecosystem framework. 3. Disseminating the COS to a wide range of local as well as pan-European stakeholders, in particular by developing analytical tools to process and present patient outcome data and establish feedback loops that inform patients and clinicians. Implementing these project objectives are five work packages within the STRONG-AYA project:  WP1: Development Core Outcome Set AYA with cancer and Data Collection (Lead: SOUTHAMPTON)  WP2: Governance, Data Security and Ethics (Lead: EORTC)  WP3: Infrastructure and Interoperability (Lead: UNIMAAS)  WP4: Operation of STRONG-AYA ecosystems, Stakeholder and Patient involvement, Dissemination, Exploitation, Communication (Lead: E.C.O.)  WP5: Scientific Coordination and Project Management (Lead: NKI) STRONG-AYA will enable AYA care and research to benefit from collection and pooling of patient-centered data and collaboration among all stakeholders: patients, healthcare professionals, scientists, and policymakers. More widely, the project will leverage the network of interested organisations and networks established under STRONG-AYA for long-term strengthened promotion of the necessary implementation of specialist AYA cancer services across Europe. This will ultimately bring novel insights into AYA cancer care, research and policy, contributing to the long-term improvement of outcomes for people with AYA cancer. Deliverable introduction WP1 (developing a Core Outcome Set): The overall aim of WP1 is to develop a Core Outcome Set (COS) for AYAs with cancer. We aim to assess which outcomes, including patient-reported outcomes, clinical, and treatment related outcomes, are most important to measure for AYAs with cancer, from the point of view of AYAs with cancer, their caregivers, healthcare professionals, researchers and policymakers. The process for developing a COS is outlined in several guidance documents, and follows a clear process [1-4]. This includes adhering to the 11 minimum standards of Core Outcome Set-STAndards for Development (COS-STAD) [2] and the 13 items considered essential documentation as described in the Core Outcome Set-STAndardised Protocol Items (COS-STAP) [3]. STRONG-AYA – No. 101057482 – D1.2 in which outcomes were extracted and categorized into relevant domains. Outcomes were counted to identify weight of domains. The Delphi process The Delphi technique is a multistage process, designed to transform opinion into consensus by iterative rounds of questionnaires, interspersed with controlled feedback [14]. Anonymity of individual panel members within specific stakeholder groups removes inherent bias, such as group conformity, as participants do not interact directly with each other, so situations where the group is dominated by the views of certain individuals can be avoided [15]. Delphi studies are typically 2-3 rounds, although aspects of its methodology can be interpreted in a variety of ways. Delphi survey development The outcomes identified as relevant for AYAs with cancer via the review of the literature (step 1) and the qualitative interviews (step 2) formed an initial list, which would be taken forward to the first round of the Delphi survey. The initial list of outcomes were categorized using Dodd’s 38-item taxonomy for outcomes in medical research [6], and formed the skeleton of the Delphi survey (Table 3). The taxonomy was adapted, in terms of order of outcomes and specific wording, to adjust for the lay audience of this Delphi survey. The ordering of domains in the taxonomy was also adjusted to improve the coherence of the survey, for the purposes of the Delphi. Table 3: The skeleton of the Delphi survey, using an adapted version of Dodd’s 38-item taxonomy for outcomes in medical research [6] Dodd’s 38-item taxonomy for outcomes in medical research Outcomes of disease Mortality/survival Physiological/clinical Outcomes relating to neoplasms Cardiac outcomes Circulation (blood, vascular and lymphatic system) outcomes Respiratory system (chest and lungs) outcomes Stomach (gastrointestinal) and digestion outcomes Nervous system outcomes Kidney and urinary system outcomes Liver and pancreas (hepatobiliary) outcomes Immune system outcomes Infection outcomes Ear, hearing, and balance outcomes Eyes and vision outcomes Skin and subcutaneous tissue outcomes Musculoskeletal and connective tissue outcomes Nutrition and metabolism outcomes Injury and poisoning outcomes STRONG-AYA – No. 101057482 – D1.2 Musculoskeletal and connective tissue outcomes Hormones (endocrine system) outcomes Reproductive system outcomes Pregnancy and childbirth (before, during and after childbirth) outcomes Genetic outcomes Psychiatric outcomes General outcomes Functioning Physical functioning Social functioning Role Functioning Emotional and Psychological Functioning / Wellbeing Cognitive Functioning Global Quality of Life Perceived Health Status Delivery of Care Personal Circumstances Resource Use Economic Hospital Need for further intervention Societal and carer burden Adverse events / effects Draft versions of the survey items were evaluated by 8 members of the STRONG AYA consortium, including work package leads (n = 2), researchers (n=2) and HCPs (n = 4). This was to ensure all outcomes were: (1) phrased in layman’s terms to ensure participants could accurately interpret them, and (2) categorized correctly according to the taxonomy. The length of the survey was highlighted as an overarching concern; therefore, efforts were made to eliminate any potential repetitiveness in outcome items and thus reduce the overall number of items to be included. The final list of outcomes, as well as the survey instructions, were pilot tested by three researchers within the STRONG AYA consortium, and feedback was provided. This was to test the usability of the survey platform, as well as the interpretations of each outcome. Recruitment for the Delphi survey The Delphi process was comprised of three expert stakeholder panels: (4) AYAs (aged 15-39 at diagnosis) and caregivers (including parents / guardians, family members, partners and caregivers). (5) Healthcare professionals who provide care for AYAs with cancer. (6) Researchers and policymakers who are active in the field of AYAs with cancer. STRONG-AYA – No. 101057482 – D1.2 The recruitment target for the Delphi survey was 100 participants per stakeholder group (300 in total). This would allow for drop-out rates moving into subsequent Delphi rounds. Recruitment materials were developed, including a 1-Page recruitment letter outlining the study, email templates, and social media adverts. Participants for each stakeholder group were recruited in the following ways: Table 4: Description of the recruitment strategies for each stakeholder group in the Delphi surveys (1) AYAs and Caregivers (2) Healthcare professionals (3) Researchers and policymakers AYAs and caregivers that took part in the qualitative interviews were asked if they would also like to take part in the Delphi survey. Healthcare professionals that took part in the qualitative interviews were asked if they would also like to take part in the Delphi survey. Researchers were contacted via their email addresses which were found in recent publications in AYA journals (e.g. Journal of Adolescent and Young Adult Oncology). Youth Cancer Europe are a charity made up of youth cancer organisations across Europe. A contact at the charity contacted potential participants to invite them to take part in the Delphi survey. They also advertised through social media channels (LinkedIn, Facebook). The European Society for Paediatric Oncology (SIOPE) is a pan-European organisation representing all professionals working in the field of childhood and AYA cancers. They shared the recruitment letter with all members of the European Network for Teenagers and Young Adults with Cancer (ENTYAC). Researchers provided contact details for other researchers who may wish to take part. The Youth and Cancer Foundation (Stichting Jongeren en Kanker) is a patient organisation for AYAs with cancer. They shared the social media adverts via their Instagram and Facebook stories. The European Cancer Organisation (ECO) are network of oncology professionals and cancer patients. They advertised the Delphi survey in their monthly newsletter. The European Cancer Organisation (ECO) are network of oncology professionals and cancer patients. They advertised the Delphi survey in their monthly newsletter. The European Cancer Organisation (ECO) are network of oncology professionals and cancer patients. They advertised the Delphi survey in their monthly newsletter. Participants shared with their networks of HCPs at local hospitals. Participants directed other AYAs and caregivers to take part, through word of mouth HCPs alerted their AYA patient community, via community patient groups and local charities. STRONG-AYA – No. 101057482 – D1.2 or advertisements in their local cancer communities. Survey data collection procedure The online survey platform used to collect responses was Qualtrics (https://www.qualtrics.com/uk/). The participant information sheet was provided at the start of the survey, as well as a checkbox for participants to provide their consent for taking part. Participants were reminded of their right to withdraw from the study at any time, privacy and accessibility information, and how their data will be used. Initial, generic demographic questions (relevant to all stakeholder groups) were included at the start of the survey to collect information, such as: age, gender, ethnicity, country of residence, education, and socioeconomic status. Specific demographic questions relevant to separate stakeholder groups were then asked. For example, AYAs were asked about their diagnosis type, age at diagnosis, treatment status, and clinic-type. Caregivers were asked the same questions, in relation to the AYAs they care for. HCPs, researchers and policymakers were asked questions relating to their profession, years’ experience, and their expertise within their profession. Our Delphi process included three survey rounds: (1) Initial item rating, with all 129-outcomes (2) Item re-rating, with feedback from previous survey round (3) Item ranking, choosing a top 15 outcomes for on and off treatment* * Round 3 of the Delphi survey is currently underway at the time of this report Round 1 of the Delphi Survey Following demographic questions, the survey asked participants to rate the importance all 129 outcomes on a 9-point Likert Scale, as per recommendations outlined by the GRADE working group [16]. A rating scale of “not important” (rating of 1–3), “important, but not critical” (rating of 4–6), or “extremely important / critical” (rating 7–9) was provided. An option for “unable to answer” was also provided. It was explained to participants that they should not rate too many outcomes as “extremely important”, as the end-goal was to have only approximately 10-15 outcomes that are kept for inclusion in the final COS. Participants were reminded on each page of the survey that they should rate the outcomes with emphasis on the AYA (aged 15-39) demographic, to avoid resulting in a COS that is not specific to the AYA age-group. Participants were required to provide a rating for every item in the survey. At the end of the survey was an open-text box for participants to provide any outcomes they thought were missing in the survey, that were important to measure for AYAs with cancer. Translations of the survey were provided for the following languages: Dutch, French, Italian, German, Spanish and Polish. This was to increase the reach and cultural diversity of the Delphi survey. Round 2 of the Delphi Survey STRONG-AYA – No. 101057482 – D1.2 The second survey round included the same initial demographic questions as round 1. Participants were provided with a personalised URL, therefore their responses to the demographic questions were saved and carried forward from round 1 and they were able to make changes (if necessary). The concept of a second Delphi survey round is to provide feedback to participants of the results from round 1, and allow them to potentially change their rating on the 9-point Likert scale, to begin a convergence of opinion (consensus) and come to an agreement on the most important outcomes to measure for AYAs with cancer. Feedback from round 1 was provided as a percentage above each number on the Likert scale (i.e. a percentage of how many other participants in their stakeholder group rated the outcome as X level of importance). Participants were also provided with their original rating from round 1 as a pre-filled dot on the Likert scale. See Figure 3. All items listed in round 1 of the Delphi survey were carried forward to round 2. Participants were also asked to rate six additional outcomes that participants had listed as important yet missing from round 1 (using the open text box). Figure 3: A snapshot of round 2 of the Delphi survey, evidencing the use of feedback (%) and previewing participant’s round 1 responses. Consensus meeting In order for an Initial COS to be developed in time to begin implementation of data collection throughout the healthcare research ecosystem for the STRONG AYA project, a consensus meeting was scheduled prior to conducting Round 3 of the Delphi survey. This was to allow decisions to be made regarding the outcomes to be included in the initial COS. This meeting took place on 18/01/2024 and attendees included 12 participants from the STRONG AYA consortium, who formed an expert panel. All outcomes that received ≥70% respondents rating it as a 7-9 on the Likert-scale, across all stakeholder groups, were presented to the expert panel. Based on discussions regarding the difficulty of distinguishing STRONG-AYA – No. 101057482 – D1.2 the most important outcomes for AYAs with cancer, it was decided that the initial COS would identify outcomes for both ON and OFF treatment phases. The definitions for ON and OFF treatment were as follows: On treatment = a cancer patient receiving curative treatment, including maintenance treatment, who is not believed to be disease-free. Off treatment = a cancer patient treated with curative intent, who is disease-free, or receiving lifeextending or palliative treatment / care An online poll took place (using a polling platform, Vevox) whereby participants were asked to select the most important 15 outcomes to measure for AYAs with cancer for both ON and OFF treatment. The number “15” was decided following exploration of the literature and averaging the number of COS items of cancer COS published since 2020. Results from this meeting were used to inform the first draft of the Initial COS. This draft of the Initial COS was circulated to the STRONG AYA consortium, plus members of the public advisory board for feedback. Feedback was provided from 7 researchers and HCPs in the STRONG AYA consortium and 4 patient advisory board members. Round 3 of the Delphi Survey Round 3 aims for participants to be more succinct with their decision making. Any outcome that received ≥70% respondents rating it as a 7-9 on the Likert-scale was carried forward to round 3 of the Delphi process, per stakeholder group. For example., the AYA and parent/caregiver stakeholder panel were only presented with the list of outcomes that received ≥70% respondents rating it as a 7-9 on the Likert-scale by their stakeholder group (see Table 6) . Similar to the consensus meeting, participants were asked to identify the top 15 outcomes for AYAs with cancer, whilst both ON and OFF treatment. Additionally, they were asked to then rank the top 15 that they chose, in order of “most important” and “least important”, for both ON and OFF treatment. It is hoped that this methodology will elicit greater consensus from stakeholder groups, thus resulting in the Final COS. Round 3 is currently ongoing at the time of this report, and is due to close within the upcoming weeks. Results from this survey round will then inform the Final COS. Phase 6: Determine “how” to measure the COS A joint initiative between the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative [5] and the COMET initiative [1] have developed a guideline on how to select measurement instruments for outcomes included in a COS (Figure 4). STRONG-AYA – No. 101057482 – D1.2 Figure 4: Flowchart for the selection of outcome measurement instruments for COS [17] Step 4: Select one instrument for each outcome in the COS Use a consensus procedure to agree on the instruments for each outcome included in the COS Evaluate the methodological quality of the studies on measurement properties Evaluate the quality of the methodological measurement properties Combine the evidence on the methodological quality of the studies with the quality of the measurement properties Evaluate the feasibility of the instruments Perform a new literature search Update the literature search Step 3: Evaluate the quality of the instruments No, not up-to-date No, of poor quality No, quality assessment not performed Step 1: Conceptual considerations Define construct to be measured Define target population Step 2: Find all existing instruments Is a good quality, up-to-date systematic review of outcome measurements available? STRONG-AYA – No. 101057482 – D1.2 To develop the Core Measurement Set for the Initial COS, the guidelines proposed in Figure 4 were followed, with the exception of updating / performing a literature search in the absence of a good quality, up-to-date review. The consensus process (step 4) will be in the form of a meeting due to take place on 26th March 2024, whereby members of the STRONG AYA consortium from the University of Southampton and NKI will meet to make pragmatic decisions on which are the most suitable measurement tools, in terms of their psychometric properties, accessibility, translation availability, and number of items/subscales. Once the Final COS has been developed, the scientific process of measurement instrument selection detailed in Figure 4 will be followed extensively. Determine “case-mix variables” As part of the data extraction process for the literature review (see section 5.5.1), case mix variables published in the literature were collected. These will then be pooled and analysed, ready for implementation alongside the Initial COS in future work packages in the STRONG AYA project. This process is ongoing at the time of this report. Results The Literature Review A total of 14 reviewers from the STRONG AYA consortium conducted title and abstract screening, and full text screening. 15 reviewers conducted data extraction. The initial search identified 37127 records through the five databases (Figure 5). After duplicates were removed, the search returned 19826 records. Following screening 17301 records on title and abstracts, 11727 records were excluded. This resulted in 5574 recorded being assessed for eligibility, from which 2980 were excluded. In total, 2594 studied were included in the reviews; outcomes from these records were collated and informed the development of the Delphi survey. Importantly, refinement of the literature search is still ongoing at the time of this report. The number of articles reported din Figure 5 were accurate for developing an initial list of outcomes for the Delphi survey, however more articles have been excluded since this point, and moving forwards to the write up of the Literature Review. STRONG-AYA – No. 101057482 – D1.2 Figure 5: Flow chart of study screening and inclusion process, according to the PRISMA Framework. Qualitative Interviews A total of 56 interviews were conducted between April 2023-June 2023; 27 AYAs, 4 parents / caregivers, and 25 HCPs. Demographic information and results relating to the production if a digital tool for the STRONG AYA project are outlined in Deliverable 1.11. A thematic content analysis resulted in 22 outcome domains that were perceived as important for AYAs with cancer. Table 5 summarises the domains identified in the interviews, including which stakeholder group identified what as important. These 22 domains, plus the individual outcomes within each domain, were then to inform the production if the Delphi survey. Identification of studies via databases and registers Records identified through database searching (n = 37127): Medline ALL (n = 13088) Embase (n = 12039) Web of Science Core Collection (n = 11146) Cochrane Central Register of Controlled Trials (n = 654) Google Scholar (n = 200) Duplicate records removed before screening (n = 19826): Medline ALL (n = 119) Embase (n = 10089) Web of Science Core Collection (n = 9021) Cochrane Central Register of Controlled Trials (n = 420) Google Scholar (n = 177) Records screened on title and abstract (n = 17301) Records Excluded (n = 11727) Full text records assessed for eligibility (n = 5574) Records excluded (n = 2980): Studies included in review (n = 2594): Identification Screening Inclusion STRONG-AYA – No. 101057482 – D1.2 Table 5: Domain results from the thematic content analysis of the qualitative interviews, per stakeholder group Domain AYAs HCPs Parents / caregivers Fertility X X X Relationships/sexual intimacy X X X Changes to family dynamics/roles X X X Mental health/psychological X X X Identity X X Social functioning (excl. romantic relationships) X X X Connecting to other peers with cancer of the same life stage X Education X X X Work (positive) X X Work (negative) X X Financial implications X X X Body image (self-esteem) X X Survival/mortality X X X Loss of control X Loss of independence X X X Long term effects of cancer and treatment/affects post treatment X X Living with and beyond cancer X X X Managing impacts on daily routines/lives X Behaviour X X Healthcare X X X Physical X X Cognitive X The Delphi survey Generating the initial list of outcomes