Performance-Guided Evaluation of Clustering Strategies for Single-Cell RNA Sequencing in Cancer Research within HPC Environments

The benchmarking of clustering strategies in scRNA-seq and ADT data demonstrated that Leiden was the most robust, scalable, and consistent method for identifying cellular subpopulations. This approach clearly overcomed Louvain, K-means, GMM/EM, and HDBSCAN in accuracy and stability. In contrast, dimensionality reduction combinations such as UMAP+PCA or PCA+UMAP did not improve performance and often degraded clustering quality, highlighting their inadequacy to capture stable biological structures. Therefore, our findings provide actionable guidance for constructing scalable, accurate scRNA-seq analysis pipelines, emphasizing the importance of aligning algorithmic choices with biological context and computational constraints in cancer research and beyond. Motivation Performance-Guided Evaluation of Clustering Strategies for Single-Cell RNA Sequencing in Cancer Research within HPC Environments Welinton Barrera Mondaca1*, Joaquín Araya Bustos1, Renato Álvarez Ramos1, Claudia Cancino Quiroz1, Raúl Caulier-Cisterna2, Ana Moya-Beltrán2 1Escuela de Informática, Facultad de Ingeniería, Universidad Tecnológica Metropolitana, Santiago, Chile. 2Departamento de Informática y Computación, Facultad de Ingeniería, Universidad Tecnológica Metropolitana, Santiago, Chile. Single-cell RNA sequencing (scRNA-seq) has transformed the study of cellular heterogeneity, enabling detailed characterizations of biological systems. Yet, these data are high-dimensional, sparse, noisy, and rapidly growing in scale, creating persistent computational challenges. Despite this, clustering is often applied through default pipelines, with limited scrutiny of how methodological choices shape biological conclusions or computational feasibility. Such practices risk obscuring true biological signals or inflating technical artifacts, particularly in High-Performance Computing (HPC) settings where efficiency is critical. This raises a main question: which combinations of dimensionality reduction (PCA, UMAP, PCA+UMAP) and clustering strategies best capture biologically meaningful groups in complex cancer scRNA-seq data while remaining computationally feasible at scale? Motivated by this challenge, our work systematically examines these methodological interactions across both RNA and ADT modalities, where panel integration and normalization add further layers of complexity, seeking to establish transparent, consistent, and reproducible practices that enhance biological fidelity, computational scalability, and interpretability in large-scale single-cell studies. To identify the best clustering algorithm for recovering immune sub-lineages in scRNA-seq by benchmarking five methods (Leiden, Louvain, K-means, EM/GMM, and HDBSCAN) on the expression matrix of a large dataset (91 samples, using sub_lineage as reference) and confirming its transferability on amultimodal ADT subset (40/91 samples). The core aim is to conclude which algorithm performs best and provide a clear recommendation that other teams can plug into their pipelines, reducing methodological ambiguity and speeding subsequent studies. The curated single-cell NSCLC dataset is the reference resource used in this study (published in Cancer Cell, 2021; publicly available via GEO: GSE15482). The final dataset comprises 361,929 cells × 33,660 genes across 91 samples. In addition, a 40/91-sample subset includes ADT (CITE-seq), which we used for multimodal validation. Leiden Acknowledgments: Leiden is a graph community-detection algorithm: build a kNN graph, optimize a quality function (e.g., modularity/CPM), refine to ensure well-connected groups, then aggregate communities; repeat until stable. Louvain Louvain maximizes modularity in two phases: locally moves nodes to improve it, then aggregates communities into supernodes;rebuilds the graph and repeats until convergence. K-means HDBSCAN GMM/EM K-means is an iterative partitioning method (Lloyd) that initializes Kcentroids with k-means, alternates nearestcentroid assignment and mean updates until inertia barely improves; assumes compact clusters and benefits from feature scaling. HDBSCAN is hierarchical, density-based, and noise-aware; builds mutual-reachability distances and an MST, condenses the cluster tree using min_cluster_size/min_samples, and selects the most stable clusters by persistence while labeling the rest as noise, handling variable densities and shapes. GMM/EM fits a Gaussian mixture with soft memberships; starts with μ, Σ, π, computes responsibilities in the E-step, updates parameters in the M-step, and stops when the loglikelihood converges; supports full or diagonal covariances to model elliptical, unequal-size clusters. Macro-F1 (Hungarian): Average F1 score across classes after matching clusters and true labels with the Hungarian algorithm. Hungarian Accuracy: Overall accuracy after aligning predicted clusters with ground-truth labels using the Hungarian algorithm. AMI: Mutual information between clusters and true labels, adjusted for chance. ARI: Pairwise similarity between partitions, corrected for chance. V-measure: Harmonic mean of homogeneity and completeness. Homogeneity: Each cluster contains only members of a single class. Completeness: All members of a given class are assigned to the same cluster. Goals Ground Truth Results Data Clustering Strategies Methodology Clustering Evaluation Metrics Best-performing method —Leiden clustering Protein-space clustering (ADT): how it lines up with RNA subtypes Sub-lineage landscape (30 groups) in the scRNA-seq dataset The radar charts (absolute 0–1 scores) show Leiden with the most balanced, high-area profile, slightly ahead of Louvain, indicating the best overall agreement with the sub_lineage reference. K-Means and GMM/EM were competitive but consistently lower, while HDBSCAN was the most variable and strongly embedding-dependent (UMAP > PCA+UMAP > PCA), with higher Completeness often driven by many small clusters. Complementing accuracy, the runtime benchmark showed KMeans as fastest (ms seconds), followed by Leiden (ten seconds), Louvain (2.5 minutes), and HDBSCAN/GMM the slowest. Overall, Leiden offers the best accuracy–runtime tradeoff across embeddings. Leiden treats data as a k-NN graph and optimizes communities on that graph, so it doesn’t require fixing kand handles nonspherical, imbalanced groups well. Compared with Louvain, it yields well-connected clusters and scales to hundreds of thousands of cells. On our dataset it matched the biological reference best, achieving Hungarian Accuracy = 55.2% (with strong AMI/V) while keeping runtime in the tens of seconds, the best overall accuracy, runtime trade-off. To provide an orthogonal validation of the RNAbased benchmark, we ran Leiden on the multimodal subset comprising 40 samples with ADT (CITE-seq), standardized to the intersection of 82 surface proteins. Preprocessing mirrored the RNA workflow (normalization, dimensionality reduction, kNN graph), and evaluation used the curated sub_lineage labels from the repository. The ADT analysis captures proteomic structure consistent with known cell identities and shows qualitative concordance with the RNA landscape, while also revealing expected transcriptome-to-proteome differences (e.g., post-transcriptional regulation and membrane kinetics). Overall, these results confirm the robustness of the Leiden-based pipeline and provide independent, protein-level support for the subpopulation separation. The sub_lineage is the curated ground truth (published in Cancer Cell, 2021; publicly available via GEO: GSE15482), used independently of our clustering to avoid circularity and capture fine-grained biology (~30 subtypes). Because the data are strongly imbalanced, we emphasized Macro-F1 (Hungarian) and also reported ARI/AMI/V. For context, we included a UMAP of ~361,929 cells from PCA colored by sub_lineage; this 2D view is provided only for visualization and summarizes neighborhood structure. Throughout the study, sub_lineage was consistently used as the reference to evaluate all clustering algorithms. Leiden consistently overcomed Louvain, K-means, GMM/EM, and HDBSCAN, achieving the best balance among accuracy, runtime, and scalability. Its robustness was confirmed on ADT data, where it reliably recovered biologically relevant subpopulations. In contrast, combining PCA with UMAP did not improve results and often reduced resolution, underscoring the need for careful pipeline design in large-scale scRNA-seq clustering. Discussion Dot plot of marker genes (columns) across immune sub-lineages (rows). Size = % cells expressing; color (viridis_r) = mean expression. Columns are grouped by lineage; the dendrogram orders rows by similarity. Bright blocks show expected patterns (B/plasma in B, mono/mac in MNP, cytotoxic in T/NK). Contact: wbarrerautem.cl [email protected] Cell 1 Cell 3 Cell 2 Cell n … Genes ADT Cell 1 Cell 2 Cell 3 … Cell n Conclusions 1.- Maier B, Leader AM, Chen ST. A conserved dendritic-cell regulatory program limits antitumour immunity. Nature. 2020;580:257–262. 2.- Qi R, Ma A, Ma Q, Zou Q. Clustering and classification methods for single-cell RNA-sequencing data. Brief Bioinform. 2020;21(4):1196-1208. References Departamento de Informática y Computación, UTEM; Escuela de Informática, UTEM; Laboratorio de Investigación Aplicada, Departamento de Informática y Computación, UTEM. This work was supported in part by “Competition for Research Assistant Funding UTEM”, year 2024, code AI23-11,and in partby the “Scientific and Technological Equipment Projects Competition, year 2024, code LE24-03”.