JOSS: A journal for open source software that is an open source software project

https://doi.org/10.5281/zenodo.17288084 JOSS: A journal for open source software that is an open source software project Daniel S. Katz Chief Scientist, NCSA Research Professor, CS, iSchool University of Illinois at Urbana-Champaign [email protected], @[email protected] (contributors/community on next slide) US-RSE’25, 7 October 2025 Patrick Diehl Los Alamos National Laboratory William Gearty Syracuse University 2 https://doi.org/10.5281/zenodo.17288084 Editors (contributors) •JOSS track editors: Warrick Ball, Samuel Forbes, Daniel S. Katz, Rachel Kurchin, Kevin M. Moerman, Kyle Niemeyer, Arfon Smith (Editor-in-Chief), Kristen Thyng, Chris Vernon •JOSS editors: Fruzsina Agocs, Gabriela Alessio Robles, Stefan Appelhoff, Jack Atkinson, Mojtaba Barzegari, Johanna Bayer, Juanjo Bazán, Sophie Beck, Sebastian Benthall, Monica Bobra, Frederick Boehm, Sébastien Boisgérault, Josh Borrow, Gabriele Bozzola, Jed Brown, Philip Cardiff, Taher Chegini, Pi-Yueh Chuang, Beatriz Costa Gomes, Pierre de Buyl, Patrick Diehl, Axel Donath, Elizabeth DuPre, Matthew Feickert, Julia Ferraioli, Henrik Finsberg, Vissarion Fisikopoulos, James Gaboardi, William Gearty, Nikoleta Glynatsi, Nicl Golding, Jeff Gostick, Rohit Goswami, Jayaram Harihan, Susan Holmes, Adam R. Jensen, Mark A. Jensen, Prashant K. Jha, Christoph Junghans, Sehrish Kanwal, Ujjwal Karn, Vincent Knight, Olexandr Konovalov, Vangelis Kourlitis, Paul La Plante, Oskar Laverny, Hugo Ledoux, Richard Littauer, Owen Lockwood, Michael Mahoney, Brian McFee, Rocco Meli, Sarath Menon, Tristan Miller, Ivelina Momcheva, Yasmin Mzayek, Kanishka B. Narayan, Lorena Pantano, AHM Mahfuzur Rahman, Blake Rayfield, Julia Romanowska, Kelly Rowland, Neea Rusch, Anjali Sandip, Mehmet Hakan Satman, Jonny Saunders, Fabian Scheipl, Jacob Schrieber, Hauke Schulz, Adi Singh, Claudia Solis-Lemus, Charlotte Soneson, Øystein Sørensen, Evan SpotteSmith, David Stansby, Andrew Stewart, Marcel Stimberg, Fabian-Robert Stöter, Fei Tao, George K. Thiruvathukal, Abhishek Tiwari, Ana Trisovic, Adam Tyson, Mauricio Pacha Vargas Sepulveda, Anastassia Vybornova, Yuanqing Wang, Rachel Wegener, Britta Westner, Lucy Whalley, Ethan White, Frauke Wiese, Mengqi Zhao, Bonan Zhu •JOSScast: Abigail Cabunoc Mayes & Arfon Smith As of 25 September 2025 3 https://doi.org/10.5281/zenodo.17288084 Software isn’t a creditable research activity 4 https://doi.org/10.5281/zenodo.17288084 How to better recognize software contributions 1. Find some way to fit software into current (paper/book-centric) system 2. Evolve beyond one-dimensional credit model We want to do 2, but it’s a long struggle, so let’s start with 1 5 https://doi.org/10.5281/zenodo.17288084 What if we just wrote papers about software? 6 https://doi.org/10.5281/zenodo.17288084 Journal of Open Source Software (JOSS) •Gives us something easy to cite 👍 •No changes required to existing infrastructure 👌 •Publishing in existing journals raises profile of software in a community 🤘 But •Writing another paper can be a ton of work 😅 •Many journals don’t accept software papers 🤬 •For long-lived software packages, static authorship has major issues 😕 •Many papers about the same software may lead to citation dilution 👊 7 https://doi.org/10.5281/zenodo.17288084 Journal of Open Source Software (JOSS) •Solution –Make it as easy as possible for authors to write software papers –Paper preparation (and submission) for well-documented software should take no more than an hour –Make the system developer-friendly for authors, reviewers, editors –The primary purpose of a JOSS paper is to enable citation credit to be given to authors of research software –Accepted software equivalent to accepted paper 8 https://doi.org/10.5281/zenodo.17288084 Journal of Open Source Software (JOSS) •JOSS uses open-source practices –GitHub issues for reviews –GitHub notifications to invite reviewers –Open collaborative discussion in reviews –Editors are a community, make high-level decisions collectively –Bot-driven commands, usable by authors, reviewers, editors, track editors –Paper is written in source (markdown), compiled to other formats (PDF, JATS) 9 https://doi.org/10.5281/zenodo.17288084 16 https://doi.org/10.5281/zenodo.17288084 JOSS as a community •JOSS practices (these levels) have influenced reviewers and developers in terms of what's good and what's minimally acceptable –Similar to rOpenSci's influence in the R community •Cultures change based on rules and incentives •JOSS provides rules, and at a high-level, tries to nudge incentives •Over time, community changes, and JOSS levels also change (stricter) •If software was cited directly, JOSS papers wouldn't be needed, but JOSS reviews and JOSS community would still be important in shaping values 17 https://doi.org/10.5281/zenodo.17288084 JOSS communication •Functional: GitHub issues (public), Slack (mostly private) •Blog: calls for editors & general news about the journal •Outreach: new papers & general news posted on Mastodon, Bluesky, X (until Jun 2023) •JOSSCast podcast (coming back from summer break soon) –Each episode features an interview with different authors of published papers in JOSS –Aimed at researchers & the public, to demonstrate the value of open-source research software, the latest developments, and how they’re changing the way research is conducted –Hosted by Arfon Smith and Abby Cabunoc Mayes –New episodes every other Thursday. –Available via Apple, Spotify, YouTube, RSS 18 https://doi.org/10.5281/zenodo.17288084 Observations on working openly •Reviewers often volunteer into a JOSS reviewers pool +Generally need relatively small number of invites to identify reviewers (~2-3 invites per reviewer) –Semi-regular emails from people annoyed they haven’t been asked to review yet –Vanity software package ‘pile-on’ •for high-profile open source projects, often have many reviewers volunteering –Good reviewers can become well known quickly, potentially leading to overuse and reviewer burnout •Zero privileged information in the system: Reviewer reports, editorial decisions available to all •Increases transparency: +Public declarations of potential conflicts +Editorial decisions documented in the open +Clear expectations of authors +Reduces complexity of infrastructure +People can link to their reviews –However, sometimes authors chase reviewers, editors, etc. –Potential cultural barriers to entry for some and negative dynamics for junior staff 19 https://doi.org/10.5281/zenodo.17288084 Scaling JOSS •Most of our challenges are about scaling people processes: –AEiC/managing editor rotations –More editors –Term limits for editors (to avoid burnout) •Technology improvements: –Smarter reviewer assignments –Better monitoring tools for editors –Tools to help authors prepare their submissions https://blog.joss.theoj.org/2019/07/scaling 20 https://doi.org/10.5281/zenodo.17288084 Reviewer feedback 21 https://doi.org/10.5281/zenodo.17288084 Appreciation of the bot * @whedon is now @editorialbot 22 https://doi.org/10.5281/zenodo.17288084 JOSS costs •JOSS depends on volunteers •Actual costs we pay: –Annual Crossref membership: $275/year –JOSS paper DOIs: $1/accepted paper –JOSS website hosting: $19/month –JOSS domain name registration: $10/year •~$2.50/paper, at 350 papers/year •Doesn't include $50k infrastructure development paid by Sloan grant, GitHub usage, user donations, AAS fees •Calculated and documented mid-2019 –https://blog.joss.theoj.org/2019/06/cost-models-for-running-an-online-open-journal •Mostly still correct, except for we didn’t plan for legal issues –Because the system is open, people can see submissions by others –They can complain: e.g., this was based on my work; I should be credited; I should be an author; this shouldn’t be allowed; don’t go forward with this or I will sue you –Maybe it’s better to have these issues arise before publication 23 https://doi.org/10.5281/zenodo.17288084 JOSS collaborations •When AAS articles include new software, authors can jointly submit –Science paper to AAS –Software paper to JOSS •Reviews done in parallel; published papers cite/linked via DOIs; AAS pays JOSS $50/paper •JOSS infrastructure also used by Journal of Open Source Education (JOSE), JuliaCon Proceedings, EuroPar artefact publication (& open to more) https://blog.joss.theoj.org/2018/12/a-new-collaboration-with-aas-publishing 24 https://doi.org/10.5281/zenodo.17288084 Citing JOSS articles •About 3200 pages published to-date •About 800 have 0 citations –Note: citations lag publication •About 500 have 20+ citations •Five most cited papers –Welcome to the Tidyverse, 16289 citations –UMAP: Uniform Manifold Approximation and Projection, 7529 citations –seaborn: statistical data visualization, 5294 citations –An R Package for Assessment, Comparison and Testing of Statistical Models, 3507 citations –ggeffects: Tidy Data Frames of Marginal Effects from Regression Model, 2039 citations Thanks! [email protected] https://joss.theoj.org/ @[email protected]