NFDI4Earth IG High-Performance Computing in Earth System Sciences - Activity Report 2023

IG High-Performance Computing in Earth System Sciences (HPC in ESS): Activity Report by June 2023 for the IG: Stephan Frickenhaus, Alfred Wegener Institute (AWI, Bremerhaven & U. Bremen) Stephan Hachinger, Leibniz Supercomputing Centre (LRZ, Garching b.M.) 1 2 IG HPC in ESS: Motivation Why FAIR is easier said than done in HPC Observation: Repositories are perfect ... but: ... most “average” repositories cannot hold huge datasets as produced in HPC (in particular in ESS). ... marrying RDM and FAIR principles with computational workflows is not easy. FAIR „vs.“ HPC in ESS: Lots of challenges! ▪Technical: Datasets produced too large for many repositories – satellite data & global/large-scale high-resolution datasets ▪Methodical: Reproducibility –simulation/analytics output depends on compiler switches, libraries, etc. “Wild” community software usage: netCDF, eccodes, … ▪Administrative: Computing needs grants and accounting, but researchers want to quickly work where their data are. ▪Practical: “Everyone talks while I have 1PB to handle!” Grouping challenges: Metadata & Findability –Access –Interoperability –Federation & Compute-to-Data 3 IG HPC in ESS: Motivation Activity report for the last year 4 12 meetings since 1st Plenary + 3 side-meetings for Pilot proposal Online-meeting participants ~12 distinct persons ~6 participants every time on average Institute participation (number of people on mailing list): AWI (4) DKRZ (3) DLR (2) FZJ-JSC (1) GEOMAR (1) HEREON (1) HLRS (2) KIT (1) LRZ (4) MPI-BGC (1) TU Berlin (1) TUDD (2) TUDD-ZIH (3) TUM (1) ▪Output: Pilot CAPICE ▪Networking: 10.11.22 workshop with NHR „Federated and FAIR Data in HPC“ Activity report (contd., for year before) 5 •Output: Publicly available concept paper: •Networking: contribution to NHR Worskhop on RDM@HPC https://tu-dresden.de/zih/hochleistungsrechnen/nhr-events/rdm-nhr-beyond https://doi.org/10.5281/zenodo.6565405 Plans: Concept Paper Update 6 ▪Concept Paper had presented 17 recommendations ▪General ▪On „(Meta-)data Formats, Interoperability and Reproducibility” ▪On „Federated Access, Findability and Accessibility“ ▪Concept Paper ToDo‘s every ~18 (12-24) months ▪Status survey w/r/t recommendations ▪Concept paper update (adapting to what has happened) (new versions of same Zenodo document) It means: We have work for the end of 2023 :) Plans: Pilot Proposal „CAPICE“ & Collaboration on Concepts with NHR & GCS 7 ▪Distributed data/modelling workflow for ice sheet modeling (CAPICE – Combined Analysis and Publication of Ice-Sheet Data) as a concept test ▪Data Projects with NHR and Gauss Alliance ▪Interaction with InHPC-DE project (GCS centres) With our collaborations, we seek to maximise synergies towards our goals. Thank You ! Want to get active with us ? Send a mail to stephan.frickenhaus [@] awi.de and/or stephan.hachinger [@] lrz.de ! ▪https://nfdi4earth.de/2participate/get-involved-by-interest-groups/ig-high-performance-computing-in-earth-system-sciences ▪Every 2nd Monday in the month we meet at 9:00 am