Leveraging Jetstream2's Multi-Region Cloud Infrastructure to Develop and Support a Science Gateway

Leveraging Jetstream2’s Multi-Region Cloud Infrastructure to Develop and Support a Science Gateway 1st Sean B. Cleveland University of Hawaii -System Information Technology Services Honolulu, USA [email protected] 2nd Jared McLean University of Hawaii -System Information Technology Services Honolulu, USA [email protected] 4th Matthew P. Lucas University of Hawai‘i at M¯ anoa Water Resource Research Center Honolulu, USA [email protected] 4th Keri Kodama University of Hawai‘i at M¯ anoa Water Resource Research Center Honolulu, USA [email protected] 5th Ryan Longman East West Center Water Resource Research Center Honolulu, USA [email protected] 6th Thomas W. Giambelluca University of Hawai‘i at M¯ anoa Water Resource Research Center Honolulu, USA [email protected] Abstract This paper explores the experience and benefits of leveraging the multi-region cloud infrastructure of Jetstream2 [1] to develop and support science gateways, with a specific focus on the Hawaii Climate Data Portal (HCDP). As climate change impacts intensify, access to up-to-date climate information and data is critical for researchers, decision-makers, and communities to facilitate appropriate preparation and response. The HCDP, a web-based portal, provides access to historical and near-real-time gridded monthly and daily climate products for Hawaii, enabling applications such as drought planning and wildfire management. A key challenge for such gateways is ensuring the consistent acquisition of transient datasets that are only available for limited time windows, sometimes less than 24 hours. This document details how deploying the HCDP’s data acquisition and processing workflows across multiple Jetstream2 regions addresses this challenge, offering improved availability, failover capabilities, and data backup. We discuss the architecture, benefits, and inherent challenges (such as cost and complexity) associated with this multi-region deployment approach, providing an experiential case study of its implementation for a critical climate science gateway. Index Terms I. INTRODUCTION The escalating impacts of changes in temperature, precipation and extreme weather events necessitate immediate access to current information and data for researchers, decision-makers, and communities to enable effective preparation and response. In Hawaii, a sustainable supply of clean water is paramount, facing threats from decreasing rainfall, rising sea levels, and increased demand due to population growth and development. Recognizing this critical need, the University of Hawai‘i Information Technology Services Research Cyberinfrastructure (UH ITS RCI) team, in collaboration with the Hawai‘i Established Program to Stimulate Competitive Research (EPSCoR) Change-HI project and the University of Hawai‘i Water Resource Research Center (WRRC), has dedicated several years to developing gridded monthly and daily climate products, which were previously unavailable for the state of Hawaii. To make these vital datasets accessible, the Hawaii Climate Data Portal (HCDP) was developed as a science gateway [?], [2]. This portal provides access to both historical and new near-real-time analyses, thereby enabling additional applications, such as drought planning and wildfire management. Science gateways are fundamentally webbased portals designed to provide researchers with streamlined access to computing resources, data, and tools. They typically feature a user interface for submitting jobs to High Performance Computing (HPC) resources, browsing and downloading data, and running scientific tools. A significant challenge for gateways like the HCDP is their reliance on data sources or processes that require acquisition at specific times to ingest necessary datasets. In many cases, these datasets become unavailable or exceedingly difficult to recover if not harvested within their limited availability window. To ensure continuous data ingestion, a robust solution is to deploy processes on redundant hardware. However, hardware within the same data center or even the same geographic region remains susceptible to localized power or network outages, which can prevent successful data acquisition. This paper highlights that a more comprehensive solution involves deploying processes across multiple geographic regions. This multi-region strategy offers substantial benefits, including improved availability and built-in backup of raw data. This work presents the organizational structure and architecture employed across two Jetstream2 regions to orchestrate the daily data acquisition and ingestion for the HCDP. II. BACKGROUND A. Science Gateways and Cloud Infrastructure Science gateways are integral in providing access to and lowering the barrier of entry to research infrastructure, accelerating scientific discovery. The orignal NSF Jetstream [3] and current Jetstream2 [1] clouds, have been key enablers for this initiative, as the National Science Foundation (NSF) federated cloud that integrates multiple regions from different institutional cloud providers. This distributed architecture can be advantageous, as it allows the deployment of science gateways across various cloud geographic regions. A cloud region is defined as a geographical area containing one or more data centers. Commercial cloud providers often offer multiple regions to enhance availability and performance. The University of Hawaii (UH) initially utilized the original Jetstream cloud for the development and testing of the Hawaii Climate Data Portal’s code. Upon the funding of Jetstream2, UH became a regional partner, gaining access to local Jetstream2 resources. The development Jetstream virtual machines (VMs) at Indiana University (IU) were then transitioned to the Jetstream2 infrastructure at IU, as the main Jetstream2 deployment was in production before the regional sites came online. During this transition period, UH successfully released the HCDP into production, leveraging Jetstream2 for both data acquisition and the daily and monthly climate data processing and product generation. B. The Hawaii Climate Data Portal The HCDP serves as a crucial resource, providing researchers and community stakeholders with access to climatological data and resources, with a current focus on the state of Hawai‘i. The portal offers interactive access and visualization of hosted historical and near-real-time gridded maps alongside aggregated sensor station observational data. Currently, precipitation and temperature data from sensor stations are collected, quality controlled, and processed daily to produce high-resolution gridded climate maps. A publicly available web application allows users to navigate, visualize, and generate time series from this data (https://hawaii.edu/hcdp) The portal is designed to host and disseminate any climatological variables that can be processed into observational time-series data and gridded or raster value maps, with an extensiblity for additional variables. The gridded monthly rainfall products, in particular, are highly useful for water resources analysis, ecological modeling, and other applications. Historically, such maps were produced irregularly and were often not available for recent months. By automating and streamlining data acquisition, quality assurance and quality control (QA/QC), gap filling, interpolation, and dissemination, the HCDP produces regularly updated maps, with preliminary products released within one day of the end of the most recent month. This ensures more up-to-date analyses, facilitating applications like drought planning and wildfire management. Historical monthly gridded data are available from 1920 and daily data back to 1990, with current production focusing on daily and monthly rainfall from 1990 onward. The workflow for these near-real-time products requires daily execution. This daily run is essential not only for generating near-real-time gridded and tabular climate products but also for ensuring the collection of transient climate data that becomes unavailable from some resource providers if not harvested within a specific window of availability. C. Workflow Process Containerization The process of collecting and processing climate data and producing products, such as precipitation and air temperature, involves diverse domain expertise, software engineering and orchestration. The codes developed for these various workflows were often created by different researchers or groups using a variety of programming languages. Given the multitude of development groups and the need for seamless deployment across local and shared infrastructures, the adoption of software containers, such as Docker, proved ideal for ensuring portability and consistency of the execution environment across different resources. Containers facilitate the distributed execution of workflow processes over local or national resources, including High Performance Computing clusters or cloud environments. The development of these workflows specifically leveraged Jetstream and Jetstream2 virtual machines (VMs) that had the Docker runtime environment pre-installed. Researchers and research software engineers (RSEs) developed the codes and workflows for data acquisition, quality analysis and control, and the generation of monthly and daily climate products on these development VMs as separate container image artifacts. III. APPROACH: MULTI-REGION DEPLOYMENT FOR HCDP A. Rational for Multi-Region Deployment A fundamental consideration for any continuously operating system, regardless of its scale, is its susceptibility to outages, whether planned or unexpected. For the HCDP, a key use case is the consistent acquisition of data from various sources and providers that often have limited availability windows, sometimes as short as 24 hours or less. In such scenarios, even a brief outage in a single data center or region could result in the permanent loss of critical transient climate data. To mitigate this risk and ensure uninterrupted data ingestion, there was a clear need for replication of at least the acquisition services into another geographic region. Deploying processes across multiple regions offers several distinct advantages beyond just redundancy. It provides improved availability, ensuring that if one region experiences an outage, operations can seamlessly continue in another. It also enables failover, where the gateway workflows can switch to a different region if one becomes unavailable. Furthermore, this approach inherently offers data backup, as data can be replicated across multiple regions for outages or disaster recovery purposes. B. Jetstream2 Multi-Region Architecture The Jetstream2 cloud infrastructure is a National Science Foundation (NSF) federated cloud, distinguished by its composition of multiple regions hosted by different institutional cloud providers [1]. This federated nature is a critical aspect that enables the operation of science gateways across multiple cloud regions without the common constraints of vendor lock-in. The University of Hawai‘i (UH) has been a significant beneficiary of this architecture. Initially, UH leveraged the original Jetstream cloud [3] for the development and testing of the HCDP code. With the advent of Jetstream2, UH evolved into a regional partner, establishing local Jetstream2 resources. The development of the HCDP transitioned from Jetstream virtual machines (VMs) at Indiana University (IU) to the Jetstream2 infrastructure, also at IU, as the main Jetstream2 deployment became operational before the regional sites. Critically, during this period, UH launched the HCDP into production, leveraging Jetstream2 not only for data acquisition but also for the gateway application programming interfaces (APIs) to serve the production data products, to reduce latency to Hawaii stakeholders. This established UH’s multi-region operational model for HCDP, utilizing both the central IU region and eventually the University of Hawaii regional Jetstream2 resources for replication and production services. C. Containerized Workflow Deployment Across Regions A significant factor contributing to the successful multi-region deployment of the HCDP’s acquisition processes was the adoption of containerization for its workflows. The HCDP team found it remarkably easy to deploy replicate acquisition processes within the UH cloud region of Jetstream2. This was primarily achieved by modifying some simple configurations for the existing containerized workflows. This approach highlights the inherent portability and environment consistency benefits offered by software containers like Docker and Apptainer. Moreover, access to persistent storage within Jetstream2 is a key enabler for the HCDP’s multi-region strategy. This feature allows the HCDP to maintain copies of the Docker container instances, which encapsulate all acquisition codes, dependencies, and their generated outputs. This capability has proven invaluable for troubleshooting: in instances of network transfer issues, the team can revert to these stored artifacts to access the data or examine problems with outputs. This ability to inspect the output alongside the exact code and dependencies used to generate it significantly aids in the continued development and improvement of the HCDP’s climate products. The streamlined management of these containerized workflows across distinct regions underpins the HCDP’s enhanced resilience when outages do occur due to planned maintenance or unexpected issues at the VM, physical server, or larger data-center scale. D. Benefits of Mulit-Region Science Gateways Operating science gateways across multiple cloud regions, as demonstrated by the HCDP on Jetstream2, yields several critical benefits: •Improved Availability: If one region experiences an outage, users can simply switch to another region to continue utilizing the gateway, minimizing downtime for essential data acquisition and access. This is particularly vital for systems like HCDP that rely on time-sensitive data. •Failover: In the event that one region becomes unavailable, the science gateway can failover to another operational region. This transition ensures continuous service and data processing, crucial for near-real-time climate products. •Backup: Data can be comprehensively replicated across multiple regions, providing a robust strategy for disaster recovery. This means that even if a catastrophic event affects an entire region, the critical climate data and derived products are securely backed up elsewhere. IV. DISCUSSION: CHALLENGES AND RECOMMENDATIONS While the benefits of running science gateways across multiple cloud regions are substantial, particularly for applications requiring high availability like the Hawaii Climate Data Portal, it is equally important to acknowledge the inherent challenges associated with this approach. A. Challenges of Multi-Region Deployment The primary challenges identified when implementing a multi-region science gateway are: •Cost: Operating a gateway across multiple regions typically incurs higher expenses compared to hosting it in a single region. This increased cost stems from the need for duplicated infrastructure, data transfer fees between regions, and potentially more complex licensing or resource allocation models that include staffing time and effort. •Complexity: Managing a science gateway distributed across multiple regions is inherently more complex than managing one hosted in a single location. This complexity extends to aspects such as data synchronization, network configuration, load balancing, security protocols across different geographical zones, and monitoring systems to ensure consistent performance and availability across all regions. The orchestration of workflows and ensuring data consistency across distributed environments requires careful planning. B. Recommendations For researchers and institutions considering the deployment of a science gateway across multiple cloud regions, careful consideration of the costs and benefits is paramount before making a decision. Based on the HCDP’s experience with Jetstream2, we offer the following recommendations: Thoroughly assess the cost implications: Understand the ongoing financial commitment required for duplicated resources, data egress charges, and potentially increased personnel needs for managing a more complex infrastructure. •Evaluate the management complexity: Be prepared for the increased operational overhead. This may necessitate a stronger cyberinfrastructure team or specialized tools for distributed system management. •Choose a cloud provider that offers a federated cloud: Opting for a platform like Jetstream2, which is explicitly designed as a federated cloud composed of multiple institutional providers, can simplify aspects of multi-region deployment by enabling the same IAM across regions. •Consider vendor lock-in: Clouds like Jetstream2’s support for virtual machine image snapshots and exports, container runtime environments and ingress/egress-free data movement. This enables moving applications to another cloud if necessary. Leveraging cloud specific offerings (e.g. function runners, cloud specific databases, etc.) in the application architecture can led to migration challenges and increased costs. V. CONCLUSION The experience of the Hawaii Climate Data Portal demonstrates that running science gateway processes across multiple cloud regions offers significant advantages, particularly improved availability, failover potential, and critical data backup for disaster recovery. This strategy is invaluable for ensuring the continuous acquisition and processing of time-sensitive climate data, which is fundamental to supporting climate research, drought planning, and wildfire management in Hawaii. However, this advanced deployment model is not without its challenges, primarily related to increased cost and operational complexity. Therefore, for researchers and organizations contemplating a multi-region science gateway, it is strongly recommended to carefully weigh these costs against the unique benefits to make an informed decision that aligns with their project’s requirements and resources. The successful implementation of HCDP on Jetstream2 serves as a compelling case study, highlighting the feasibility and strategic value of leveraging federated cloud resources for highly available and reliable scientific cyberinfrastructure and science gateways. ACKNOWLEDGMENT This work was made possible by grant funding from the National Science Foundation (NSF) OIA-2149133 RII Hawaii EPSCoR: Change Hawaii and Project Tapis: Next Generation Software for Distributed Research NSF CSSI #1931439, #1931575 and NSF OAC-2232862 CC* Data Storage: KoaStore. This work used Jetstream2 at Indiana University through allocation allocation ASC190026 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants 2138259, 2138286, 2138307, 2137603, and 2138296 REFERENCES [1] D. Y. Hancock, J. Fischer, J. M. Lowe, W. Snapp-Childs, M. Pierce, S. Marru, J. E. Coulter, M. Vaughn, B. Beck, N. Merchant, E. Skidmore, and G. Jacobs, “Jetstream2: Accelerating cloud computing via jetstream,” in Practice and Experience in Advanced Research Computing, ser. PEARC ’21. New York, NY, USA: Association for Computing Machinery, 2021. [Online]. Available: https://doi.org/10.1145/3437359.3465565 [2] R. J. Longman, M. P. Lucas, J. Mclean, S. B. Cleveland, K. Kodama, A. G. Frazier, K. Kamelamela, A. Schriber, M. Dodge, G. Jacobs, and T. W. Giambelluca, “The Hawai‘i Climate Data Portal (HCDP),” Bulletin of the American Meteorological Society, vol. -1, no. aop, Apr. 2024. [Online]. Available: https://journals.ametsoc.org/view/journals/bams/aop/BAMS-D-23-0188.1/BAMS-D-23-0188.1.xml [3] C. A. Stewart, G. Turner, M. Vaughn, N. I. Gaffney, T. M. Cockerill, I. Foster, D. Hancock, N. Merchant, E. Skidmore, D. Stanzione, J. Taylor, and S. Tuecke, “Jetstream,” in Proceedings of the 2015 XSEDE Conference on Scientific Advancements Enabled by Enhanced Cyberinfrastructure - XSEDE '15. ACM Press, 2015. [Online]. Available: https://doi.org/10.1145/2792745.2792774