D2.3 HPC and AI Maturity Survey

LUMI AI Factory Service Center Empowering Europe’s AI Ecosystem D2.3 HPC and AI Maturity Survey 2 D2.3 HPC and AI Maturity Survey D2.3 HPC and AI Maturity Survey 3 Project Title LUMI AI Factory Service Center Project Acronym LUMI-AIF Project Number 101234208 Type of Action HORIZON-JU-RIA Topic HORIZON-JU-EUROHPC-2025-AI-01-IBA-01 Starting Date of Project 01.03.2025 Ending Date of Project 29.02.2028 Duration of the Project 36 months Website lumi-ai-factory.eu Work Package WP2 Customer Engagement Task Task 2.3. Customer needs analysis and facilitating access to AI supercomputers Lead Authors Outi Tasala (CSC) Contributors Susanne Sjöblom (CSC), Markus Koskela (CSC), Heidi Laine (CSC), Aleksi Kallio (CSC) Peer Reviewers Marta Maj (Cyfronet) Morthen Mathisen (CSC) Version 1.0 Due Date 31.7.2025 Submission Date 28.7.2025 Dissemination level x PU: Public SEN: Sensitive – limited under the conditions of the Grant Agreement EU-RES. Classified Information: RESTREINT UE (Commission Decision 2005/444/EC) EU-CON. Classified Information: CONFIDENTIEL UE (Commission Decision 2005/444/EC) EU-SEC. Classified Information: SECRET UE (Commission Decision 2005/444/EC) D2.3 HPC and AI Maturity Survey 4 Version History Revision Date Editors Comments 0.1 26.6.2025 Outi Tasala, Susanne Sjöblom, Markus Koskela Submitted for internal review 0.2 1.7.2025 Outi Tasala Submitted to SMB for comments. 0.3 2.7.2025 Outi Tasala, Susanne Sjöblom Sumitted to PMO for final quality check. 1.0 28.7.2025 Anna Luoma Final quality check performed by the PMO, version submitted to official review. Glossary of Terms Item Description Access mode One of the available ways to access the LUMI-AI supercomputer: batch tenant access, interactive access, or exclusive access. Access model One of the available methods for getting access to the LUMI-AI supercomputer for different customer groups such as SMEs, start-ups, and large industries: e.g. national and European calls, grants, challenges, and pay-per-use. Customer An organization that uses LUMI AI Factory services. Digital Support Systems Collection of digital tools used in the Customer Process, such as CRM system and ticketing tool for customer inquiries, desribed in detail in deliverable 2.1 Customer Process Description. Exclusive Access Mode Exclusive access to a number of nodes for a given period of time granted for a project. In this way, a project may have a private cluster within LUMI-AI, with a tailored computing environment if needed. HPC and AI Maturity evaluation Action that is either automated or carried out by the AI Factory staff to identify the entry points and access models for the customer’s maturity level with HPC and AI. Evaluation is based on the survey and the following discussion. HPC and AI Maturity survey A survey carried out either independently by the customer (online form) or with the assistance of AI Factory staff (early discussion) to assess the HPC and AI readiness as indication and a base for maturity evaluation. Maturity Maturity refers to the depth, sophistication, and integration of HPC and AI capabilities within an organization or system over time. Maturity D2.3 HPC and AI Maturity Survey 5 indicates how far the organization has come and how well they’re leveraging HPC and AI in practice. Readiness The current capability and preparedness of an organization to adopt and deploy HPC and AI technologies. Readiness indicates the starting point – how equipped the organization is to begin to use AI or scale up the use – and reflects the organizations maturity. User Person with a user account in LUMI or LUMI-AI, obtained either from the customer portal or via the EuroHPC Federation Platform. D2.3 HPC and AI Maturity Survey 6 Executive Summary This report presents the HPC and AI Maturity survey developed by the LUMI AI Factory as a strategic tool to support customers in assessing their potential to adopt high-performance computing (HPC) and artificial intelligence (AI) technologies. The survey plays a central role in guiding customers through the process of defining project scope, understanding available resources, and preparing successful applications for compute capacity through LUMI AI Factory and EuroHPC JU. The primary objective of the survey is to evaluate an organization’s current capabilities, identify technical and skills gaps, and recommend suitable entry points and access models for compute resources. It is an integral part of the AI Factory’s customer process, ensuring that customers are wellinformed and positioned for success in leveraging advanced computing technologies. In addition to its direct benefits to individual customers, the survey enables LUMI AI Factory to collect and analyze data across organizations, uncovering broader trends and informing future support strategies. This dual function enhances both individual project outcomes and the overall effectiveness of AI Factory’s services. Looking ahead, the report outlines a roadmap for reviewing and developing the survey. Key recommendations include incorporating user feedback, modularizing the survey for domain-specific relevance, and integrating it with AI Factory’s digital support systems. These enhancements will ensure the survey remains a dynamic, user-centered tool that evolves alongside technological advancements and customer needs. D2.3 HPC and AI Maturity Survey 7 Table of Contents 1. Introduction ............................................................................................................... 8 2. Objective .................................................................................................................... 8 3. Scope ....................................................................................................................... 10 4. Method for reaching customers ................................................................................ 11 5. Criteria for scoring .................................................................................................... 12 6. HPC and AI Maturity Survey template structure ........................................................ 13 A) Questions to estimate the readiness level 14 B) Questions to estimate the need category 19 7. Requirements for the survey tool used for HPC and AI Maturity evaluation ............... 22 8. Future review and development of the HPC and AI Maturity Survey .......................... 23 9. Conclusions .............................................................................................................. 23 D2.3 HPC and AI Maturity Survey 8 1. Introduction This document describes the comprehensive support actions by the LUMI AI Factory to customers applying or considering applying for HPC and AI computing resources, ensuring successful project acquisition and customer satisfaction. It also provides a classification for HPC and AI readiness and introduces a template for the HPC and AI Maturity Survey, the main tool used in the evaluation. HPC and AI Maturity Survey helps the customer in defining the project scope, objectives, and timelines, as well as helping them understand and navigate the available HPC and AI resources, tools, and services. For completing the evaluation, the survey is supplemented by an in-depth discussion. This deliverable introduces the template used for the survey and the methods for assessing customer’s HPC and AI maturity. It also makes explicit the planned actions for developing the HPC and AI Maturity Survey and the analysis based on it further. 2. Objective The objective for the HPC and AI Maturity evaluation is to understand customers’ current capabilities for leveraging AI and HPC technologies to form a coherent picture of the customers’ maturity for themselves and for the LUMI AI Factory. The analysis does this by covering key dimensions related to HPC capabilities, access and ownership of datasets and experiences with AI applications. It describes the organization’s technical readiness for AI and HPC development and shows possible skills gaps and competence needs. Readiness is used here to reflect the organization's current capability to adopt and scale HPC and AI technologies, serving as a practical indicator of the organizations overall digital maturity in HPC. The main purpose of the HPC and AI maturity evaluation is to help the customer navigate within the compute services available through LUMI AI Factory and EuroHPC JU. HPC and AI maturity evaluation is an integral part of the customer process in LUMI AI Factory. Customer journey is described in more detail in deliverable 2.1 Customer Process Description and only the simplified graph of the customer journey is presented here for reference: D2.3 HPC and AI Maturity Survey 9 Figure 1 shows a simplified customer journey with the different steps along the path, the LUMI AI Factory services and the customer needs analysis and the HPC and AI maturity analysis as part of the process. The HPC and AI maturity evaluation aims to: • Implement the understanding of the customer’s current state and need, gained from the Customer Needs Analysis • Determine the customer’s potential to apply for resources If the results show that the project needs compute resources, the aim is to: • Get the customer acquainted with the requirements for applying for resources and the different access models • Aid the customer in writing a successful proposal in applying for HPC resources suitable to their needs The objective of HPC and AI maturity survey is to give customers an informed picture of their HPC and AI maturity and aid them on their journey towards getting access to compute resources. The assessment provides recommendations for the most suitable entry point and access model for that customer to apply for HPC and AI capacity. The aspects included in the survey are relevant in the application process for the resources, and thus this phase in the customer process prepares the customer for applying for the resources. D2.3 HPC and AI Maturity Survey 16 Data collection and preparation 2 Model selection and development 3 Deployment 3 Have you previously proven in some environment that the application or code is scalable? Score range: 0-3. Weight: 3. Response Score No scalability testing or validation has been done. 0 Limited scalability testing; the application has been run on a small cluster or with a modest number of cores/nodes, but no formal scalability analysis. 1 Moderate scalability proven; the application has been tested on mid-sized HPC systems with some performance metrics or scaling behavior observed. 2 Scalability has been thoroughly demonstrated; the application has been benchmarked or profiled on large-scale HPC systems, with documented strong scaling or weak scaling results. 3 What types of workloads have you run on HPC systems (e.g., simulations, AI/ML training, data analytics)? Score range: 0-3. Weight: 2. Response Score No HPC workloads run yet. 0 Basic or single-domain workloads (e.g., simple simulations, batch processing, or basic data analysis). 1 Moderate diversity or complexity (e.g., multi-domain simulations, AI/ML training, or large-scale data analytics). 2 Advanced and varied workloads (e.g., tightly coupled simulations, hybrid AI/HPC workflows, real-time data processing, or workflows requiring GPUs, high memory, or I/O optimization). 3 What job schedulers or resource managers are you familiar with (e.g., Slurm, PBS, LSF)? Score range: 0-3. Weight: 3. D2.3 HPC and AI Maturity Survey 17 Response Score No familiarity with job schedulers or resource managers. 0 Basic awareness or limited experience (e.g., submitted jobs using a template but not familiar with scheduler options or commands). 1 Comfortable using one or more schedulers (e.g., Slurm, PBS, LSF); can write and modify job scripts, use job arrays, and monitor jobs. 2 Advanced proficiency; experienced with multiple schedulers, understands resource allocation, job dependencies, and can optimize job scheduling or manage queues. 3 Are your applications optimized for parallel or distributed computing? Score range: 0-3. Weight: 3. Response Score No optimization for parallel or distributed computing; applications run serially or on a single core. 0 Basic parallelization implemented (e.g., multithreading), but limited scalability or efficiency. 1 Applications are reasonably optimized for parallel or distributed environments; demonstrate good performance on moderate core counts or nodes. 2 Applications are highly optimized for HPC; include advanced parallelization strategies (e.g., 3D parallelism) efficient memory and I/O usage, and proven scalability on large systems. 3 Do you use containers or workflow managers (e.g., Singularity, Nextflow, Snakemake)? Score range: 0-3. Weight: 3. Response Score No use of containers or workflow managers; all work is done manually or in ad hoc scripts. 0 Basic awareness or occasional use of containers or workflow tools, but not integrated into regular workflows. 1 D2.3 HPC and AI Maturity Survey 18 Regular use of containers (e.g., Singularity, Docker) or workflow managers (e.g., Snakemake, Nextflow) to improve reproducibility and automation. 2 Advanced use of containers and workflow managers; workflows are fully containerized, portable, and automated, with support for scalability, reproducibility, and version control. 3 Are you using any version control or CI/CD tools in your workflows? Score range: 0-3. Weight: 2. Response Score No version control or CI/CD tools are used; code and workflows are managed manually. 0 Basic use of version control (e.g., Git) for code tracking, but no structured workflow or automation. 1 Regular use of version control and some CI/CD practices (e.g., automated testing, deployment scripts, GitHub Actions). 2 Fully integrated version control and CI/CD pipelines; includes automated testing, deployment, documentation, and reproducibility across environments. 3 Do you have a dedicated team or person managing HPC resources? Score range: 0-3. Weight: 3. Response Score No dedicated person or team 0 One person has partial responsibility for HPC tasks, but it's not their primary role. 1 A dedicated person manages HPC resources, supports users, and handles basic maintenance or job troubleshooting. 2 A dedicated HPC team is in place, with clear roles for system administration, user support, performance tuning, and workflow optimization. 3 Do you hold the necessary data you need for your analysis? Score range: 0-3. Weight: 3. Response Score D2.3 HPC and AI Maturity Survey 19 No relevant data is currently available; data collection has not started or is a major blocker. 0 Some data is available, but it is incomplete, outdated, or not in a usable format or the quality of the data is unknown. 1 Most of the required data is available and usable, though some gaps or quality issues may remain. 2 All necessary data is available, well-organized, and ready for analysis; includes proper documentation, formats, and access controls. 3 Do you have a data management or backup strategy in place? Score range: 0-3. Weight: 2. Response Score No data management or backup strategy. 0 Basic strategy in place. 1 A consistent data management and backup process exists; includes versioning, regular backups, and some documentation. 2 A comprehensive and automated data management strategy is in place; includes regular backups, metadata, access control, data lifecycle policies, and compliance with best practices or standards. 3 B) Questions to estimate the need category How much computing capacity does your work require? Score range: 0-3. Weight: 1. Response Score None 0 Small scale <5000 GPU hours 1 Medium scale, 10 000 - 50 000 GPU hours 2 Large scale, > 50 000 GPU hours 3 How long do your jobs usually run? Score range: 0-3. Weight: 1. D2.3 HPC and AI Maturity Survey 20 Response Score Jobs are very short (e.g., a few minutes); typically test runs or lightweight tasks. 0 Jobs run for up to a few hours. 1 Jobs typically run for several hours to a day. 2 Jobs regularly run for multiple days or require checkpointing. 3 Do you anticipate scaling your workloads significantly in the near future? Score range: 0-3. Weight: 1. Response Score No plans to scale workloads; current usage is expected to remain the same. 0 Minor scaling anticipated (e.g., slightly larger datasets or more frequent runs). 1 Moderate scaling expected (e.g., new projects, increased data volume, or more users). 2 Significant scaling planned (e.g., major increase in compute demand, transition to large-scale simulations, AI/ML model training, or multi-node workflows). 3 Do you have your own application or self-developed codebase? Score range: 0-3. Weight: 1. • Own application refers to a complete software product that you or your organization has created, a finished product. • Self-developed codebase refers to code written by you or your team that may be part of an application, library, or tool. Response Score No self-developed code; only uses third-party application or tools. 0 Minor modifications to existing codebases or scripts; limited original development. 1 Custom scripts or applications tailored to specific tasks or workflows. 2 Fully self-developed, complex codebase or application; actively developed, versioncontrolled, and optimized for HPC use. 3 Does your company use open-source or proprietary software for AI and HPC? Score range: 0-3. Weight: 1. D2.3 HPC and AI Maturity Survey 21 • Proprietary software is owned by an individual or company. Source code is closed to the public. • Open-source software is freely available for anyone to use, modify, and distribute. Source code is open and accessible. Response Score No clear strategy; limited or no use of specialized software for AI or HPC. 0 Uses only proprietary or only open-source software, with limited flexibility or integration. 1 Uses a mix of open-source and proprietary tools, but with limited customization or optimization. 2 Actively uses and integrates both open-source and proprietary software strategically; contributes to open-source projects or customizes tools for performance and scalability. 3 Does your work possess/generate/use large datasets? Score range: 0-3. Weight: 1. Response Score No or < 10 GB 0 Small-scale, 10 GBs - 100 GB 1 Medium-scale, 100 GBs - 1 TB 2 Large-scale, > 1 TBs 3 Is the data in the project sensitive or confidential? Score range: 0-3. Weight: 1. • Sensitive data is such that, if exposed, could cause harm to individuals, organizations, or systems. This includes personal data, health records, financial details, or proprietary business information. • Confidential data is restricted to authorized users only, often protected by legal, contractual, or organizational policies. Disclosure could violate agreements or regulations. Response Score No 0 D2.3 HPC and AI Maturity Survey 22 Yes, confidential 2 Yes, sensitive 3 Is the data in the project dynamic or streaming? Score range: 0-3. Weight: 1. • Dynamic data changes over time, it can be updated, modified, or replaced periodically. • Streaming data is continuously generated and delivered in real time or near-real time. Data is streamed into the environment from the outside. Response Score No 0 Yes, dynamic 2 Yes, streaming 3 7. Requirements for the survey tool used for HPC and AI Maturity evaluation At the beginning, a simple survey tool, such as Typeform or similar, is implemented to get the maturity surveys running. However, some needs have been identified that demand further investigation into the selection of the tool. To enhance the functionality and user experience of the survey tool, it should include a mechanism for scoring user responses in the background. This scoring process must be automated and invisible to the user, allowing for real-time analysis of input data. Based on the calculated scores, the system should be capable of generating personalized suggestions or recommendations that are relevant to the user's responses. This feature aims to provide immediate, context-aware feedback, thereby increasing the tool’s interactivity and value. Requirements: • The system shall assign scores to user responses using predefined logic or algorithms. • The scoring process shall be executed in the background without requiring user intervention. • The system shall use the scores to generate context-relevant suggestions or recommendations. • The suggestions shall be presented to the user in a clear and actionable format. • The scoring logic shall be configurable to support different survey types or use cases. D2.3 HPC and AI Maturity Survey 23 8. Future review and development of the HPC and AI Maturity Survey To ensure the continued relevance and effectiveness of the HPC and AI Maturity Survey, a regular review cycle for the survey is established. This includes feedback collection from both internal stakeholders and customers who have completed the survey. Their insights can highlight areas where the survey may be too complex, too simplistic, or missing critical dimensions of HPC and AI readiness. Incorporating this feedback will help refine the questions, improve clarity, and ensure the survey remains aligned with evolving technologies and customer needs. Another area for development is modularization and customization of the survey. As AI and HPC use cases diversify, a one-size-fits-all approach is not desired. Developing modular sections tailored to specific domains (e.g., healthcare, manufacturing, academia) or organization types could make the survey more relevant and actionable. This customization could be supported by an adaptive digital interface that adjusts the survey flow based on initial responses. Another avenue for future development is the integration with AI Factory’s broader customer support systems. Integration with CRM system or project tracking tools would enable a seamless experience from initial assessment to resource application and project execution. 9. Conclusions The HPC and AI maturity survey is designed to be a valuable tool in supporting customers on their journey toward effectively utilizing high-performance computing and AI resources. By assessing an organization’s current capabilities, technical readiness, and potential skills gaps, the survey enables tailored guidance and strategic alignment with available compute services. It not only facilitates successful project planning and proposal development but also enhances the overall customer experience within the AI Factory framework. Furthermore, the aggregated insights from the survey contribute to a broader understanding of trends across industries and domains, allowing AI Factory to continuously refine its support mechanisms. As the survey evolves, it will play an increasingly critical role in empowering customers to make informed decisions and maximize the impact of HPC and AI technologies in their operations.