Description of the Use of the HTCondor Cluster by the GUAIX Group

Description of the Use of the HTCondor Cluster by the GUAIX Group Sergio Pascual <[email protected]> October 2025 This document describes the usage of the HTCondor Onix cluster, built using resources from the GUAIX group, part of the Department of Earth Physics and Astrophysics at the Complutense University of Madrid. The use of high-performance computers for intensive computations is common across various scientific disciplines. In our research group, the strategy adopted to provide computational capacity has been the acquisition of high-performance workstations, funded through different projects. Over time, these machines become outdated or are considered obsolete, although many remain fully functional and could still be leveraged for computational tasks. To reuse these computers and meet the computational needs of the group, a cluster has been built based on HTCondor1 [1], a job management system that enables workload distribution across multiple nodes. The technique known as High Throughput Computing (HTC) allows for the processing of large volumes of tasks via distributed nodes, thereby facilitating efficient reuse of existing resources. The Onix cluster of the GUAIX group consists of between six and nine Linux nodes, depending on availability, running operating systems such as Rocky Linux 9, RHEL 9, and Alma Linux 10. RAM across the nodes ranges from 16 to 128 GB. The cluster architecture includes an orchestrator node, an entry point (pollux.fis.ucm.es), and several compute nodes, each being an independent workstation. The HTC cluster operation mode involves dividing the workload into tasks as small as possible that can be executed independently. Each task is described using .sub files, specific to HTCondor. The orchestrator is responsible for assigning jobs to available nodes, managing their execution, and collecting results. Since tasks are executed on different nodes, both the application and the required data must be sent to each node individually. However, some nodes share a folder via NFS, which 1 https://htcondor.org/ 1 simplifies this process. Additionally, HTCondor supports running containerized applications, compatible with Apptainer and Docker formats. Most of the computers used in the cluster are dedicated machines that were not being used for other tasks prior to their integration. Two of the cluster nodes were funded by the European project AC3[2]. One of the nodes corresponds to a personal machine temporarily provided by a group researcher. Tests have also been conducted to integrate personal workstations into the cluster during specific hours or when their workload is low. To use the cluster, the only technical requirement is to have a user account on pollux.fis.ucm.es, which must be requested from the system administrator. Currently, the Onix cluster is in an experimental phase, and no usage policies regarding compute time or storage capacity have been defined. If the Onix cluster has been useful in your research, we kindly ask you to include the following acknowledgment: “This work was partially carried out using the computing resources of the GUAIX group https://guaix.ucm.es/ from the Department of Earth Physics and Astrophysics at the Complutense University of Madrid.” References [1] Thain, D., Tannenbaum, T., & Livny, M. (2005). Distributed computing in practice: The Condor experience. Concurrency and Computation: Practice and Experience, 17(2‐4), 323–356. https://doi.org/10.1002/cpe.938 [2] European Commission. (2022). Agile and Cognitive Cloud edge Continuum management (AC3). Horizon Europe Programme. Grant Agreement No. 101093129. https://cordis.europa.eu/project/id/101093129 2