Full text
R E SILIENT T E CHNOLOGIES WHY DECADES-OLD TOOLS DEFINE THE ROOT OF MODERN RESEARCH DATA MANAGEMENT LUKAS C. BOSSERT bosser[email protected]wth-aachen.de �0000-0003-3076-3968 IT Center @ RWTH AACHEN UNIVERSITY Department: Research Process & Data Management Group: Technical Training & Consulting Version: 1.0.1 cba LuaHBTeX, Version 1.22.0 (TeX Live 2025) GNU Emacs 30.2 | org-mode: 9.7.34 DOI: 10.5281/zenodo.17157588 Powered by: Org mode r s y n c s e d t a r d i f f f i n d p e r l S Q L i t e g r e p a w k c u r l L a T e X c r o n m a k e e m a c s / o r g - m o d e P l a n n i n g P r o d u c t i o n A n a l y s i s A r c h i v i n g A c c e s s R e - u s e Research data management (RDM) today relies on a growing landscape of platforms and specialized tools. While these innovations advance research, they also bring risks: short life cycles, proprietary dependencies, and limited sustainability. In contrast, long-established tools have proven to be remarkably resilient— distinguished by openness, interoperability, and active communities. This poster demonstrates how such “resilient technologies”— despite their age—remain central to sustainable, transparent, and reproducible research data management today. emacs (1976) / org-mode (2003) planning, production, analysis, archiving, access, re-useplanning, production, analysis, archiving, access, re-use Emacs is not just an editor but a fully extensible research and productivity environment. It can be adapted to cover the entire workflow—from notes and coding to analysis and publishing—without switching applications. The real strength unfolds with org-mode: plain text enriched with semantic markup and lightweight metadata. An Org file can act simultaneously as notebook, task manager, analysis script, and manuscript— a robust foundation for reproducible research. Org-mode supports structured metadata at the top of every file, also for export/provenance tracking. Org-mode has semantic markup, similar but more extensible than markdown. Personal knowledge management and linking by creating a network of thoughts, snippets and information with orgroam. With org-babel you have embedded, executable code blocks for transparent analyses, best way to do literate programming. org-mode provides a lightweight macro system that lets you define reusable text snippets with parameters. Tables are plain-text, too, however you have full control over formulas and calculations. curl (1997) planning, access, re-useplanning, access, re-use curl is one of the most universal and resilient tools for data transfer. Unlike graphical or browser-based downloads, curl can be scripted and parameterized for reproducibility, transparency, and automation—core principles of FAIR and Good Scientific Practice. The minimal usage pattern: snippet (bash) curl [OPTIONS] [URLs] sed (1974) productionproduction sed is a stream editor for transforming text—fast, scriptable, and reliable even on very large files. It excels at data cleaning tasks: normalizing metadata, harmonizing formats, and fixing systematic errors. snippet (bash) sed [FLAGS]'s/pattern/replacement/g' [FILE] The leading s stands for substitute. It tells sed to search for the given pattern and replace it with the specified replacement. The trailing g flag applies the substitution globally across the line, so every occurrence of the pattern is replaced, not just the first. grep (1973) production,analysisproduction,analysis When it comes to searching and validating data, grep is one of the oldest and most resilient tools. It is extremely fast, works on arbitrarily large files, and supports /regular expressions/ for flexible pattern matching. The basic syntax is: snippet (bash) grep [FLAGS]PATTERN [FILE ...] diff (1974) analysisanalysis When it comes to verifying reproducibility and provenance, diff is one of the simplest yet most powerful verification tools. It compares files line by line and reports any deviations. This allows researchers to document data cleaning steps, validate outputs, or monitor versioned transformations. After applying transformations (e.g. normalization with sed ), diff can confirm the expected changes and provide a readable audit trail. The basic syntax looks like: snippet (bash) diff [FLAGS] [FILE1] [FILE2] make (1976) production,analysis,archiving,accessproduction,analysis,archiving,access AMakefile (without suffix) is an integral part of every project. It serves as a Standard Operating Procedure (SOP) that defines how individual tasks depend on each other and how they are executed. In the context of research data management, make functions as a lightweight yet powerful workflow engine. Each rule represents not just a command, but a clearly defined dependency chain that transparently records how results are generated from inputs. This approach ensures traceability,reusability, and automation—core principles of Good Scientific Practice and reproducible research. Here is the basic design of a makefile: snippet (make) VAR =|?=|+=|:=VALUE .PHONY:TARGET TARGET TARGET all:TARGET TARGET:PERREQUISITE(S) COMMAND cron (1975) production, analysis, archivingproduction, analysis, archiving When reproducibility meets time, cron provides the temporal backbone of automated research workflows. It executes predefined tasks at regular intervals — hourly, daily, or monthly — ensuring that updates, validations, and synchronizations happen automatically. cron thus complements make: the former manages when, the latter how. Here is the explanation and a basic pattern of a cron-file. snippet (bash) # |----------- 1: minute (0–59) # | |--------- 2: hour (0–23) # | | |------- 3: day of month (1–31) # | | | |----- 4: month (1–12) # | | | | |--- 5: day of week (0–6, Sunday=0) #12345 * * * * * command_to_execute awk (1977) analysis, productionanalysis, production When it comes to data processing and analysis, awk is one of the most remarkably enduring tools in computational research. It combines the features of a streaming text processor, a pattern matcher, and a lightweight programming language. Its concise syntax allows complex data operations to be written in just a few lines of code. Here is the basic pattern: snippet (awk) awk -F',' 'BEGIN { init } pattern { action } END { summary }' [FILE],→ tar (1979) archivingarchiving The command-line tool tar—short for tape archive—has been the backbone of data packaging and long-term archiving for nearly half a century. Originally designed to write sequential data streams to magnetic tape, it remains one of the most widely used and trusted archiving tools in computing today. Here you see the basic pattern snippet (bash) tar [FLAGS] [FILEs/PATTERN] [DIRECTORY] Not all tools of lasting relevance can be introduced in depth here. The following list presents additional resilient technologies that remain fundamental to reproducible and sustainable research workflows. find (1974) BSD/GPL; Recursively searches complex directory structures to locate files, match patterns, or filter metadata. Essential for navigating large research repositories with nested folders or evolving data versions. LaTeX (1984) LPPL; The enduring standard for scientific writing and documentation— used for DMPs, reports, posters, and publications. Combines typographic precision with reproducibility and platform independence. perl (1987) Artistic/GPL; A versatile scripting language for text manipulation, pattern recognition, and metadata transformation. Ideal for parsing logs, transforming XML/JSON, or validating research data. rsync (1996) GPL; Synchronizes and mirrors directories across systems efficiently. Commonly used for backups, replication, and maintaining mirrored repositories. Transfers only differences, saving time and bandwidth. SQLite (2000) Public Domain; A serverless SQL database stored in a single file—lightweight yet powerful. Excellent for metadata catalogs, portable archives, and small-scale analytical datasets.
