Digital Porous Media Portal: Specialized Features for Porous Media Research - Demo

Digital Porous Media Portal: Specialized Features for Porous Media Research Shayan Khan Texas Advanced Computing Center The University of Texas at Austin Austin, USA [email protected]xas.edu John Gentle Texas Advanced Computing Center The University of Texas at Austin Austin, USA [email protected] Jake Rosenberg Texas Advanced Computing Center The University of Texas at Austin Austin, USA [email protected] Maria Esteva Texas Advanced Computing Center The University of Texas at Austin Austin, USA [email protected] Masa Prodanovic The University of Texas at Austin Austin, USA [email protected] Abstract - This paper introduces the Digital Porous Media Portal, a specialized science gateway which extends TACC's Core Experience Portal (CEP) codebase to address HPC and data management challenges for porous microstructure image analysis. This demo submission highlights the portal’s advanced features which includes robust data curation with Pydantic validation, automated image processing workflows, and a streamlined publishing pipeline. These capabilities provide researchers with a more efficient, reproducible, and accessible environment for managing and analyzing scientific data. Keywords – core portal, tapis, science gateways, open-source I. INTRODUCTION Scientific research has been increasingly relying on HighPerformance Computing (HPC) resources for data processing, simulation, and analysis. However, direct interaction with complex HPC environments and management of vast scientific datasets often poses significant technical and logistical challenges for many researchers. To bridge this gap, the Web and Mobile Applications Team at the Texas Advanced Computing Center (TACC) has developed the Core Experience Portal (CEP) [1]. CEP serves as a foundational, highly configurable online platform, designed to simplify researcher engagement with TACC's diverse HPC systems. Its primary objective is to provide an "out-of-the-box" framework for rapidly deploying science gateways, ensuring pre-configured common capabilities that follow TACC's current best practices. CEP enables students and researchers to access supercomputers remotely, offering web-based tools for intuitive data file management, streamlined HPC job submission, and comprehensive system monitoring. The Digital Porous Media Portal (DPMP) [2] is built as a specialized instance of CEP. It is designed to address the unique needs of researchers in the fields of Petroleum, Civil and Environmental Engineering, and Geology. DPMP specifically focuses on the fast storage, retrieval, sharing, organization, and analysis of images of varied porous micro-structures and their accompanying measurements [2]. DPMP inherits all the functionalities of CEP and introduces a suite of advanced, domain-specific features that enhances the entire research workflow. This paper serves as an overview for an accompanying live demo, which will showcase how DPMP extends CEP's capabilities by providing solutions for data curation and validation, integrated automated image processing, and a streamlined data publishing pipeline. II. CORE FUNCTIONALITY Since DPMP is built on CEP it comes with standard functionality that is included on all gateways. CEP utilizes Tapis [3] and its Python library, tapipy [4], to handle all interactions with the HPC resources. The gateways’ Dashboard offers an at-a-glance view of user-accessible HPC systems, their status, available applications, and executed jobs. User storage is seamlessly mapped to a folder on TACC’s highperformance storage system, allowing easy management of personal, community, public, and shared workspace data. The gateway also features System Monitoring, which displays the status of available HPC systems and detailed queue information to aid in job submission. It supports hundreds of applications like MATLAB, Jupyter Lab, RStudio, and Paraview, facilitating the launch of workflows on HPC resources using researchers' data. A key aspect of CEP is its feature development and deployment model: new or enhanced features developed for CEP are shared across all sibling gateway projects, and useful capabilities from specific gateways can be incorporated back into CEP and disseminated to others. These fundamental tools provide a consistent way for researchers to handle their essential HPC needs, laying the groundwork for DPMPs’ specialized features. III. DIGITAL POROUS MEDIA PORTAL FUNCTIONALITY While CEP provides a solid foundation, DPMP introduces specialized features that enhance the research lifecycle for those working with porous microstructure images and associated data. During the demo, we'll dive into these unique capabilities. DPMP features a system for data curation with metadata and its validation. The gateway provides mechanisms for users to attach metadata to their images and measurements. A python library called Pydantic [5] is used to define and validate strict metadata schemas, ensuring that uploaded data and its associated metadata meet predefined standards. This approach significantly improves data organization, searchability, and overall data quality, reducing errors and enhancing scientific rigor. The gateway also offers automated image processing workflows. Tailored specifically for porous microstructure analysis, DPMP integrates powerful capabilities built on Python libraries like NumPy, Matplotlib, and Pillow. These workflows handle various image formats, which enables tasks such as image interpretation, basic corrections, and automated generation of visualizations like thumbnails, histograms, and animated GIFs from 3D image stacks. This feature saves researchers significant time and computational effort, provides standardized analysis, and allows for immediate generation of derived data products within the portal workflow. Furthermore, DPMP features a streamlined publishing pipeline. This is an automated process designed to simplify sharing research outputs with external repositories. This pipeline efficiently bundles data with its metadata, creates an easily downloadable archive of the entire dataset, and assigns persistent identifiers like DOIs before pushing it to be publicly available via the gateways’ public interface as well as DataCite [6]. This streamlined approach simplifies open scientific data publishing, promoting open science, reproducibility, and broader researcher impact. Screenshots and descriptions of the user interface can be seen in Fig. 1. IV. CONCLUSION DPMP stands as a prime example of how specialized science gateways, built upon the flexible foundation of the Core Experience Portal, can significantly enhance domain-specific research workflows. By abstracting the complexities of HPC interaction and integrating advanced features tailored to the needs of Petroleum, Civil and Environmental Engineering, and Geology researchers, the portal empowers users to focus more on their science. The demo for DPMP will provide a clear overview of how the gateway accomplishes this. It will showcase the data curation process with metadata validation, the automated image processing workflows, and the streamlined data publishing capabilities. The demo aims to illustrate how these features collectively create a more efficient, reproducible, and accessible research environment for managing and analyzing porous microstructure data. ACKNOWLEDGMENT The Digital Porous Media Portal project is supported by a National Science Foundation grant NSF GEO OSE 2324786. Initial development was funded by NSF EAR CAREER Grant 1255622 and EarthCube Grant 1541008. REFERENCES [1] “Core Portal Home,” Utexas.edu, Feb. 03, 2021. Available: https://cep.tacc.utexas.edu/. [Accessed: Jun. 12, 2025] [2] “DPM Home,” Digitalporousmedia.org, May 07, 2025. Available: https://digitalporousmedia.org/. [Accessed: Jun. 12, 2025] [3] “Tapis Project,” Tapis-project.org, Jul. 25, 2019. Available: https://tapis-project.org/. [Accessed: Jun. 13, 2025] [4] tapis-project, “GitHub - tapis-project/tapipy: Python lib for interacting with an instance of the Tapis API Framework,” GitHub, 2020. Available: https://github.com/tapis-project/tapipy. [Accessed: Jun. 13, 2025] [5] “Pydantic,” GitHub, Jun. 18, 2023. Available: https://github.com/pydantic/pydantic [6] “Welcome to DataCite,” datacite.org. Available: https://datacite.org/ Fig 1. User interface for DPMP, from top to bottom: (1) DPMP Dashboard, (2) DPMP Published Datasets, (3) DPMP Home Page