The power of open polyglot plain text tooling for reproducible AI research

Outlook • main limitation: all three tools have a steep learning curve and familiarity with the command line is useful ◦ thus, we assume the main audience to be data scientists or tech-savvy researchers • we are currently implementing practical demonstrations - follow our work for updates in this regard • I am very interested in talking about open science in clinical research as well as implementing reproducible research projects to reduce inequality in healthcare - find me at the conference or contact me regarding these topics: • what is it? ◦ organizes files as datasets, based on git and git-annex • solves: ◦ version control of all your code/data/etc ◦ recording provenance of files, including which code was used to generate them ◦ use git branches or tags as entrypoints to reproduce experiments • alternatives: ◦ git (problem: no tracking of arbitrarily large files, no direct link from code to data) datalad Box 2: literate programming instead of code with comments, write prose interjected with code (focus on humans) related concepts: • tangling - automatic extraction of source code from literate programming document into source files • transclusion - opposite approach, include viewable references to (parts of) source files into the literate programming document • what is it? ◦plain text-based tool for outlining, note-taking, spreadsheets, project planning, ... ◦ polyglot (see Box 1) and literate (see Box 2) programming via notebook-like environment for >70 languages ◦one-click publishing as LaTeX, ODT, a modern website, ... • solves: ◦ human-readable "glue" for different reproducibility components ◦ integrate typical human readable texts like research hypotheses, ethics declarations, etc. into reproducible pipelines • alternatives: ◦ not plain-text, limited languages: Polyglot Notebooks in VS Code Emacs org-mode Limitations Github Mastodon Bluesky The power of open polyglot plain text tooling for reproducible AI research Alva Seltmann1, 2, Christian Eggeling1, 2 1Institute for Applied Optics and Biophysics, Friedrich Schiller University Jena, Jena, Germany, 2Leibniz Institute of Photonic Technology, Jena, Germany Introduction flake.nix { description = "A very basic flake"; inputs = { nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-25.05"; }; outputs = { self, nixpkgs }: let pkgs = nixpkgs.legacyPackages."x86_64-linux"; in { devShells."x86_64-linux".default = pkgs.mkShell { packages = with pkgs; [ ((emacsPackagesFor emacs).emacsWithPackages(epkgs: [ epkgs.emacs-jupyter ])) datalad python313 R ] ++ (with pkgs.python313Packages; [ seaborn jupyterlab ]) ++ (with pkgs.rPackages; [ gtsummary ]); shellHook = '' echo "welcome to the shell!" ''; MY_ENVVAR = "custom_envvar"; }; }; } labbook.org Some plain text description, with typical markup features like *bold* or /italic/ #+begin_src sh datalad create --force datalad save -m "save initial status" #+end_src #+RESULTS: add(ok): labbook.org (file) add(ok): flake.nix (file) save(ok): (dataset) action summary: add (ok: 2) save (ok: 1) #+begin_src R :session r1 library("gtsummary") trial |> tbl_summary(include = c(age, response), by = trt) |> as_tibble() #+end_src #+RESULTS: | Age | 46 (37, 60) | 48 (39, 56) | | Unknown | 7 | 4 | | Tumor Response | 28 (29%) | 33 (34%) | | Unknown | 3 | 4 | #+begin_src jupyter-python :session /jpy:localhost#8888:py1 import seaborn as sns df = sns.load_dataset("penguins") sns.swarmplot(data=df, x="body_mass_g", y="sex", hue="species") #+end_src #+RESULTS: [[file:./.ob-jupyter/98fd83785222d93 b70dc4691db24c0d50af5a992.png]] #+begin_src sh datalad save -m "Add table and image" #+end_src #+RESULTS: etc... here, we connect to R directly, but for Python we connect to a jupyter session (via header arguments) record datalad commands and output together with R and Python scripts output plot is saved as file, inline image is displayd by orgmode plain text is interpreted by Emacs org-mode, making the document interactive shell script to execute after environment creation; set custom environment variables. nix package manager • goal: methods reproducibility = ability to record and implement all experimental and computational procedures with the same data and tools, obtaining the same result • several sub-problems have to be solved: setting up a reproducible environment, tracking and versioning generated code and data, ... ◦ currently, no one best practice tool exists ◦ the problem gets harder for polyglot programming (see Box 1) • we propose the combination of three existing tools, each solving specific sub-problems of methods reproducibility ◦ all tools are popular in certain communities - here we highlight their synergies • what is it? ◦declarative environment (see example on the right) ◦ here: using the flakes feature → creates a lock file for all dependencies of the project, making it reproducible • solves: ◦ no more "it works on my machine" ◦ easy and deterministic set up of more complicated tools (looking at you, Emacs!) ◦ no more dependency hell (looking at you, Python!) ◦ multiple package version for different projects are possible ◦ sharing projects - very easy to run just from e.g. a Gitlab repository link • alternatives: ◦ only Python: uv, poetry, venv ◦ only Python and R: conda • existing wrappers for ease of use: ◦ rix https://github.com/ropensci/rix declare system packages, Emacs packages, Python packages, R packages, etc... declare system - works for all UNIX systems and using WSL2 on windows declare package source (nixpkgs has >120.000 software packages - custom sources possible) Box 1: polyglot programming use multiple programming languages in one software / data science project to leverage their strengths for different tasks example scenario: train a classifier of two groups → use R for creating reproducible summary tables of the group variables (e.g. via gtsummary) → use Python for reproducible model training (e.g. via tensorflow.keras and mlflow) Synergies easy setup of Emacs org-mode and datalad when reproducing projects or experiments reproducible polyglot environment (hard to achieve otherwise!) Synergies plain text files (and code) lend themselves to git-based versioning - a struggle for e.g. jupyter notebooks - while being feature-rich when viewed     inside Emacs Find me: LinkedIn