From Reproducible Research to Open Science Dissemination: A Computing Platform-Centric Approach
Abstract
Transparency and reproducibility requirements in computationally intensive scientific research demand novel solutions that integrate rigorous research practices with open science dissemination. This showcase will highlight a scientific computing platform designed for big data life science research that treats the open sharing of reproducible findings as a natural and efficient extension of the research process itself. By embedding a computational reproducibility framework directly within the platform, researchers can proactively capture a complete trace of their analysis, including data, methods, and executable tools, as their investigation unfolds. This approach empowers result verification and re-execution during the study. It also provides the essential components for transparent open science publication of the study findings through the release of the reproducible trace, granting access to all relevant data and the ability to re-run analysis steps within a compute environment mirroring the original infrastructure. Furthermore, this approach to research provenance offers a powerful mechanism for contextual data governance, moving beyond traditional IT-centric metrics to policies informed by the actual use and significance of specific data sets and tools. This enables organizations to create precise policies for data archival and tool discontinuation, which in turn reinforces the platform's long-term sustainability, ensuring its continued existence and impact on scientific discovery.
Full text
From Reproducible Research to Open Science Dissemination Paul Meijer Senior Director, Scientific Software Engineering A Computing Platform-Centric Approach
immunology.alleninstitute.org | 2 Large Scale Research Presents Transparency and Reproducibility Challenges 1. Store, categorize and find large amounts of data 2. Prepare data for transformation and analysis 3. Keep track of complex multi-step analyses 4. Support interdisciplinary team collaboration 5. Enable transparent research for ongoing review 6. Share the data, analysis, and results to the open science community 7. Ensure analysis reproducibility of shared results 8. Keep research available, affordable, and sustainable
immunology.alleninstitute.org | 33 Use Case: Large-Scale Human Immunology Research… Longitudinal studies Multi-omic data Wet-bench (validation, follow-up, etc.) Scientific Computing
immunology.alleninstitute.org | 4 Generates Lots of Research Assets… Large-Scale Human Immunology Research ~5 petabytes of assay data Flow Cytometry ~2.5B cells single cell RNAseq ~55M cells A large concurrent computing footprint 2500+ collection kits 350+ subjects
immunology.alleninstitute.org | 5 Across Intricate, Multi-Step Workflows Large-Scale Human Immunology Research
immunology.alleninstitute.org | 6 Our Solution: A Comprehensive Platform Built for Reproducibility and Openness complex multi-step analyses →capture research as it unfolds transparent ongoing review →enable exact re-execution of steps team collaboration →trace all tools and transformations share data, analysis and results →directly from the computing platform analysis reproducibility →share the research trace of the results available, affordable, sustainable →ongoing usage guides governance research governance reproducibility traceability
immunology.alleninstitute.org | 77 A Computing Platform for Large, Complex Research… 7 teamwork data search analysis visualization processing pipelines
immunology.alleninstitute.org | 8 …and Open Science Dissemination and Interaction explore.allenimmunology.org
immunology.alleninstitute.org | 9 How it Works: Trace-Driven Architecture Tracks all Data and Transformations An analysis platform designed for traceability Register data, code, and analysis environment details Publish data, code, and tools for interactive inspection and reproducibility JupyterLab Input files IDE Conda environment Output files Jupyter Notebook Incrementally build a graph showing each step
immunology.alleninstitute.org | 1616 Certificate trace from the Human Immune Health Atlas 92 automated pipeline runs 47 analysis steps 182 output files results for 108 wet lab samples Certificates of Reproducibility: The Reality More in our paper in Royal Academy Open Science: Meijer P, Howard N, Liang J, Kelsey A, Subramanian S, Johnson E, et al. Provide proactive reproducible analysis transparency with every publication. R Soc Open Sci. 2025;12: 241936. doi:10.1098/rsos.241936 https://tinyurl.com/repro-article
immunology.alleninstitute.org | 17 Our Solution: A Comprehensive Platform Built for Reproducibility and Openness complex multi-step analyses →capture research as it unfolds transparent ongoing review →enable exact re-execution of steps team collaboration →trace all tools and transformations share data, analysis and results →directly from the computing platform analysis reproducibility →share the research trace of the results available, affordable, sustainable →ongoing usage guides governance research governance reproducibility traceability
immunology.alleninstitute.org | 18 Available, Affordable, and Sustainable Research What Does That Mean? Data Tools Available Keep data that is used for new studies or by the open science community Available Containerize tools and algorithms so they continue to run Affordable Archive all other data - abandoned analysis paths, data that has lost interest Affordable Discontinue tools along abandoned research traces Sustainable Evaluate data retention against regeneration via modern methods Sustainable Manage and upgrade containerized tools and algorithms with continued usage
immunology.alleninstitute.org | 19 Not All Research Traces Are Published incomplete data data quality issues mistakes in analysis tool execution failures abandoned data explorations unclear results archive data archive data archive data discontinue tool
immunology.alleninstitute.org | 20 Not All Data Releases Remain Relevant evaluate cost of tool maintenance consider data archival savings
immunology.alleninstitute.org | 2121 Research Traces Inform Sophisticated Data and Tool Governance Policies collection data ingest data transformation tool generation storage data persistence tool containerization utility data usage data archival data deletion tool execution tool upgrade tool removal Data age and size are arbitrary metrics for governance Research traces reveal the true relevance of data and tools
immunology.alleninstitute.org | 22 A Holistic View Of Sustainable Research Enable Original Research New hypotheses Innovative data generation techniques Novel algorithmic approaches Support Open Science Reproducibility and verification Availability of reference data sets Tool democratization A Sustainable Approach To Science Requires balancing financial health, infrastructure, and the expertise vital to both launch new research and promote robust open science engagement More in our paper in the Harvard Data Science Review: Meijer, P. et al. (2025). Research Lifecycle Management: Using Analysis Reproducibility Research Software to Define Contextual Data Governance Policies. Harvard Data Science Review, 7(3). doi:10.1162/99608f92.08da1513. https://tinyurl.com/reproGov
immunology.alleninstitute.org | 23 In Summary: Transparent Interdisciplinary Analysis Drives Reproducible and Sustainable Research Large scale, compute intense interdisciplinary research inherently requires... Proactive traceability of analysis as it unfolds. Capturing every step enables… Releasing data with its research trace to ensure reproducibility, a cornerstone of scientific validity. Understanding the usage patterns of trace data and tools informs retention governance, optimizing resource allocation, allowing for more….
THANK YOU We wish to thank the Allen Institute founder, Paul G. Allen, for his vision, encouragement, and support.