No Knowledge without Source: Collecting, Preserving and Sharing Software in a Risky World

ERCIM meeting, Rennes, October 30th, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ No Knowledge without Source Collecting, Preserving and Sharing Software in a Risky World Roberto Di Cosmo Image created with DALL-E Inria and Université Paris Cité Director, Software Heritage Co-chair, Software College, French Open Science Committee https://dicosmo.org @rdicosmo 1 ERCIM meeting, Rennes, October 30th, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ Open Source Software: “data altruism” everywhere Industry Academia Open Source Security/Risk Analysis 2024 ●OSS in 96--97% audited commercial codebases ●~77% of code within those codebases is OSS ●Avg. ~900+ OSS components/app ERCIM meeting, Rennes, October 30th, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ Open Source Software: global and growing ERCIM meeting, Rennes, October 30th, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ It is a risky (digital) world Digital fragility (one root of non reproducibility) HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ The largest open source code archive Unique digital common good built in France since 2015 with - 50 × 10⁹ nodes - 800 × 10⁹ edges ~ 2 PB of storage 5000+ platforms All versions, all history development in a single graph } ensures availability guarantees integrity allows traceability of source codes A unique infrastructure to preserve the knowledge embedded in software source code 5 HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ More risks and challenges Collateral damage from ●EU Copyright Directive ●GDPR Legal uncertainty: vast majority of publicly available code is shared to be reused, but has no licence information New obligations e.g. Cyber resilience act (ongoing, you should look at it) HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ Coping with the (Gen)AI tidal wave Looking for founding principles at Software Heritage Question: can one build a model this way? 7 Findings from BigCode: The Stack v2 and StarCoder2 Yes, it’s possible but it’s hard to do it well Dataset Model HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ Lessons learned Transparency is easy: use SWHID (under ISO/IEC standardisation) and Software Heritage (Re)use of the data is tricky: who is the real owner of a source code file? What are the real use rights? + ●Building a qualified training set is expensive (includes license detection at massive scale) Need a coordinated effort to address these issues ●Attribution on model output is challenging (800 billion edges, and counting!) 8 Software Heritage SCAI members can ●deposit source codes ●publish their SWHID today! It’s time to build a Code Commons HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ CODE COMMONS Gather data Data Sources Clean and Process Raw Data Train the model Trainin g Data Test the model Model Production Qualifie d Model Availability: Easy access to all relevant data for software (source code, PR, issues, discussions, etc.) Shared harvesting: we do it only once! Efficiency: Facilitate the extraction of qualified datasets to build high-performance models. Structuring: Organize and connect the various data sources to create a coherent training set. Ethics: Provide tools to verify the provenance and attribution of generative AI outputs Traceability: Identify and make available the data used for training Solutions 5Me FR funding 30 Months HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ Integrity and identification of 50B+ artifacts in the archive 16 Traceability AI anecdote HiPEAC Bruxelles Meeting on Open Source and AI, October 22, 2025 CC-BY 4.0 Roberto Di Cosmo https://codecommons.org/ 17 Related projects feeding the archive expansion SoFAIR SWH-Security