1 Summary of findings - Beyond Open Research (phase II)

Developing a proof-of-concept for quality and reliability Summary of findings Imperial College London Work Package 1 Data Transparency and Quality in the Context of Team Science Workshop 1: Embedding Transparency and Quality in Research Data at the LMS, 28 April 2025 15/12/2025Beyond Open Research project (phase II) 2 "Highway to hell" and "Stairway to heaven" exercise: what holds you down (rocks) and what lifts you up (balloons) in the different stages of the research data management lifecycle? Example practical activity from the Reproducibility by Design toolkit: participants identified the qualitycritical steps in their research process and how to manage risks to quality at those stages Imperial College London 15/12/2025 3 Barriers and incentives to research quality and transparency in the different stages of the research data management lifecycle Planning and project set up Barriers: Lack of knowledge, experience and confidence; resistance to data sharing; discipline-specificities and time constraints Enablers: More training opportunities for students and staff; a scheme of data management champions; shared college-wide infrastructure and a standard directory Data collection Barriers: Technical and digital obsolescence and data storage capacity; lack of standards (e.g. file naming) and untrialled/unchecked data quality Enablers: More human resources (permanent lab manager); procedures for handing over data when someone leaves the post; shared folder best practices for trustworthiness and reliability of data Data processing and analysis Barriers: Paper-based record keeping practices; lack of confidence when dealing with large datasets Enablers: Peer-reviewing; domain-specific data stewards Data archiving and sharing Barriers: Password protected data, corrupted data, inaccessible data; the cost of data maintenance; not knowing the provenance of the data Enablers: Accountability; transparency in the University policies for data archiving and sharing Data discovery and reuse Barriers: Balancing the need to release the data in a reasonable amount of time and following protocols and best practices; keeping data annotated, using URI and/or universally recognised ID’s. Enablers: Paying for patients’ data copyright. Imperial College London 15/12/2025 Feasibility of Adopting Data Management Plans (DMPs) for All New Starters (2025/26) •Adoption is feasible , provided there is strengthened institutional support, consistent training and standardised procedures and necessary due to gaps in knowledge, experience and confidence among new researchers •Time constraints were identified as a barrier and should be factored into implementation planning •Embedding peer-review into the DMP process would allow to both develop skills and enhance research quality Training, Support and Infrastructure for Data Transparency and Openness •A robust support ecosystem, encompassing technical, human and governance, is essential to foster research quality, transparency and openness •Training must be ongoing and tailored to disciplinary needs, particularly where confidentiality and privacy requirements affect data sharing •Emphasis on developing clear protocols (e.g. file naming conventions, data handover procedures and metadata annotation) Imperial College London 15/12/2025 Scrutiny of Data Types and Workflow Integration •Effective scrutiny relies on transparent protocols and methodologies as well as trustworthy systems •Well-trained data stewards with domain expertise can help address challenges such as data readiness and unclear processing strategies •Peer review is recommended as a quality assurance measure Recognition and Incentivisation of Good RDM Practice •Incentives must reflect the diverse roles, including technical and support staff •Additional human resources are essential to ensure research quality and transparency. This includes funding for roles such as data stewards and peer reviewers and, in some cases, covering copyright costs for data owners •Accountability for both Principal Investigators and those involved in data archiving is key for good RDM practice Imperial College London 15/12/2025Beyond Open Research project (phase II) 6 The workshop was an effective learning platform for the FAIR principles, successfully raising awareness of the benefits of transparent research in enhancing research quality. However, the extent of improvement observed highlights the importance of continuing these conversations with already engaged participants, while also expanding the dialogue to include new and less-engaged audiences. Imperial College London Work Package 2 Public and Patient Involvement in the Context of Fundamental Research and Team Science Workshop 3: Improving Quality and Reliability by Involving the Public in Fundamental Research, 10 July 2025 15/12/2025Beyond Open Research project (phase II) 7 Icebreaker: Turn to the person next to you, introduce yourself, explain the reason why you came to this workshop and tell them an interesting fact about yourself. Visually represent what your partner says. Imperial College London 15/12/2025 Motivations for public involvement in fundamental research Impact Public partners emphasised the aspirational, altruistic side of impact: “Helping others is rewarding on itself.” Imperial community wants to be sure that research matters for their patients, to understand the audience better and enhance representativeness/representation. Curiosity and learning To understand research at Imperial and how public money is being used. The opportunity to ask questions directly to experts. Bringing the public in is an opportunity to improve research by learning from lived experience, to understand ethical issues and public perception of “how acceptable the things we do in research are to the public” and to practice means of making data and outputs accessible to non-expert audiences. Collaboration and communication Public members noted that meaningful involvement can change the researcher’s approach to the project. Imperial staff values the possibility of getting new ideas from the public as well as more knowledge about their needs and how to reach them. Public engagement will also help in fine tuning research question and prevent from getting bogged down in narrow pathways. No. of occurrences Motivations’ codes 13Collaboration and communication 14Curiosity and learning 4Enjoyment and creativity 16Impact 11Involvement and engagement 8Personal benefit 7Representation and inclusion 9Transparency and trust Beyond Open Research project (phase II) Imperial College London Disseminate Ensuring participants receive results of research Ensuring results are in clear, lay language for non experts Taking advices from participants where to disseminate results/findings ‘Question time' for science Understand barriers to sharing that may be unethical Recognition for those involved Create forums like this workshop Design Ensuring inclusivity during the project Understanding what public/patient consent to Making sure the study is relevant Go to public communities, explain in lay terms and accessible images Understand public views on proposed design and question other ways to improve methods Public contribution: their thoughts and priorities Create vested interest early will help on dissemination “Speed-dating" for science-public relationships Understand what part of a condition to focus on Gather lived experience Act Policy making Participants to evaluate their experience Determine what happened as a result of dissemination Learning how to do things better next time Improved feedback Create more interest for further research Partner New outreach areas (libraries, community leaders, etc.) Finding funds from public sector Finding partners for the next phase to design the project Enhance diversity of voices Community members could help identifying involvement groups Analyse Having good reliable info from patients will help to understand data Members of public in Data Monitoring Committees Preventing bias (parity in terms of collection + analysis) Transparency: outreach activities to try data analysis Collect Transparency is important for the public to know What/why/when/who has access (e.g., insurance companies, GDPR) Go to the public and consider their individual needs Advertising effectively and via relevant routes Providing training to gather data (qualitative and quantitative) Designing questionnaire Bringing those communities to the lab to understand the context Data collection from as diverse parts of population as possible Public involvement in different stages of the research lifecycle Beyond Open Research project (phase II) Imperial College London 15/12/2025 15 Methodology Workshops format: Building on the BOR I workshop model, with well-briefed introductory talks from subject experts and facilitated breakout discussions, BOR II further explored participants involvement and agency. The use of toolkits and frameworks, as well as the tailoring of fit-to-purpose engagement activities for each workshop, contributed to adding a practical, co-creative approach to the discussion led format. Active participation in developing the tools, such as process mapping or values sitting and probing, paired collective critical thinking with horizontal decision making and resulted in actionable outputs. Workshop evaluation: Built-in, ongoing evaluation (4 L’s – liked, lacked, learned, longed template) proved effective and constructive. By allowing participants to give immediate feedback during the workshop, it was possible to have a more granular and comprehensive understanding of the participants’ needs and to respond to their requirements as the workshop unfolded. Record keeping, data analysis and sharing findings: Changes in record keeping methods, from notetaking in BOR I to collective documentation in BOR II, promoted dynamic participation and tangible results. Although the data collected is less structured, these materials enable more engaging ways of sharing the project findings. Imperial College London Final notes Proof of concept for quality and reliability Is feasible and necessary Requires ongoing, tailored, discipline-specific training Benefits from peer review to foster critical thinking and transparency Public Involvement in Fundamental Research There are ideas to explore (e.g. question time for science), but it is necessary to further investigate the specificities of involvement in fundamental research What do we value in Team Science? Diversity Knowledge sharing Recognition Training Methodological learnings Discussion-driven, practical approach: frameworks and templates such as Reproducibility by design, ALCOA and SCOPE are effective and appreciated. The impact of the process: ongoing community building/strengthening around Open Research quality and reliability create the necessary systemic changes in Research Culture. 15/12/2025Beyond Open Research project (phase II) 17 Options? Diversity & Recognition: Track contributors and their roles using ORCID Training & Knowledge sharing: Cross training demands with available skills Imperial College London Next steps •Maintain BOR community involved and engage with new stakeholders, by developing a communication and engagement plan •Engage with Principal Investigators and Heads of Departments •Embed DMP peer-review as both training tool and quality and reliability assurance mechanism •Collaborate with on-going projects at LMS that can provide a concrete context to apply and test public involvement practices 15/12/202518Beyond Open Research project (phase II)