Introduction to OME-ZARR, S3 and transformations

ome-zarr and transformations Johannes Soltwedel 09.12.2025 Gerbi Teammeeting 1 09.12.2025 Gerbi Teammeeting 2 Slide shared under CC-BY 4.0 https://zenodo.org/records/17609551 09.12.2025 Gerbi Teammeeting 3 Why do we need a new (next-gen) file format? Shared under CC BY-NC 2.5 https://creativecommons.org/licenses/by-nc/2.5/ So far, OME, bioformats have your back: Read more: https://www.openmicroscopy.org/bio-formats/ 09.12.2025 Gerbi Teammeeting 4 NGFF/OME/ZARR? ngff.openmicroscopy.org 09.12.2025 Gerbi Teammeeting Chunked file formats Def. (n) Directory-like layout of data in subfiles rather than a single file, header information available in formats like JSON or YAML Pros: •natively supports parallel read/write access to chunks •avoid costly downloads or transfer of large data on cloud infrastructure (e.g., S3) •web-/cloud-ready Cons: •A lot of files on disk Image „Monolithic vs. chunked“ by Henning Falk https://github.com/zarr-developers/zarr-illustrations-falk-2022 (c) NumFOCUS (CC-BY-4.0) Open image 09.12.2025 Gerbi Teammeeting 6 Pyramidal layout Typical WSI image: ~30k x 30k pixels Typical monitor: 2550 x 1440 pixels H&E hnSCC tumor section 09.12.2025 Gerbi Teammeeting 7 S3: Simple Storage Solution – first introduced by AWS S3 cloud storage Local S3 Where‘s the data? Local drive Remote cloud Data is stored as Files Objects with metadata Hierarchy Folders None (Data lake) Data can be accessed through File browser (loading) Webbrowser (streaming) Sharing Copy data Via URL Scalability Buy more harddrives Scales virtually indefinitely Permissions Per network-drive, per user locally Data in buckets Image „Clara shares“ by Henning Falk https://github.com/zarr-developers/zarr-illustrations-falk-2022 (c) NumFOCUS (CC-BY-4.0) 09.12.2025 Gerbi Teammeeting 8 Becoming OME-ZARR Human-readable metadata header file contains all important information about image (axis ordering, pixel sizes, etc) This is what „the spec“ defines: [..] "multiscales" contains a list of dictionaries where each entry describes a multiscale image. [...] Each "multiscales" dictionary MUST contain the field "datasets", which is a list of dictionaries [...] [item1, item2..]: Array of stuff {“key“: “value“}: Dictionary of stuff 09.12.2025 Gerbi Teammeeting 9 RFCs 09.12.2025 Gerbi Teammeeting 16 Acknowledgements