Biostatistical Assessment of Paradoxical Patterns in Psychiatric Outcome Data Related to COVID-19 Vaccination

1 Biostatistical Assessment of Paradoxical Patterns in Psychiatric Outcome Data Related to COVID-19 Vaccination Marco Roccetti Department of Computer Science and Engineering University of Bologna, 40126, Italy [email protected] tel. +39 3920271318 Abstract Background: The use of Big Data and computational methods has enabled large-scale epidemiological studies using administrative health databases. Psychiatric research, however, is particularly susceptible to methodological distortions, including outcome misclassification and selection bias, which can render findings misleading and potentially harmful for public health policy. Objective: This study aims to critically evaluate the bio-statistical validity and epidemiological plausibility of reported associations between COVID-19 vaccination and incident psychiatric disorders emerging from a recent research with an experimental cohort based on administrative data from a large database. 2 Methods: We conducted a methodological re-analysis using the published summary statistics. Reported incidence rates and Hazard Ratios (HRs) were benchmarked against independent national epidemiological data and assessed for logical and mathematical consistency, considering fundamental constraints such as incidence being lower than prevalence and expected directionality across risk groups. We also examined potential bias mechanisms, including the impact of selection bias, surveillance bias, and misclassification of psychiatric outcomes, to determine whether the reported associations could plausibly reflect true biological effects. Results: Three epidemiological paradoxes were identified. First, an implausibly low incidence of Schizophrenia in the older, less healthy vaccinated cohort (HR = 0.231) contradicted national incidence data. Second, the reported incidence of Bipolar Disorder in the unvaccinated group exceeded the annual prevalence of related chronic conditions, indicating misclassification. Third, simultaneously deflated HRs for chronic disorders and inflated HRs for common psychiatric conditions reflected uncorrected baseline disparities and surveillance bias. Conclusions: The associations reported in the scrutinized study are primarily methodological artifacts rather than true biological effects. Large administrative datasets require rigorous study design, careful cohort construction, and validation against external epidemiological benchmarks to produce reliable results. This analysis underscores the necessity of methodological rigor in psychiatric and vaccine safety research to ensure that findings accurately reflect biological reality and guide public health decisions. 3 Keywords: Biostatistical Computing; COVID-19 Vaccines; Psychiatric Adverse Events; Epidemiology; Selection Bias; Health Administrative Data Introduction The integration of Big Data analytics, computational statistics, and algorithmic modeling has transformed contemporary medical research. Retrospective analyses of large administrative health databases, such as national insurance registries, allow investigators to assess retrospective population-scale associations between exposures (e.g., pharmaceuticals or vaccines) and clinical outcomes in real-world settings. While this approach offers unprecedented statistical power and breadth, large datasets do not inherently produce reliable evidence [1]. Their interpretability and validity depend critically on adherence to foundational epidemiological principles, including the use of gold-standard benchmarks, external validity checks, and rigorous control of selection bias. These challenges are amplified in psychiatry. Unlike disorders with objective biomarkers, psychiatric diagnoses rely on clinical assessment and can vary across time, institutions, and diagnostic cultures. Administrative databases are therefore particularly prone to misclassification, both under-ascertainment of true cases and over-ascertainment due to miscoded or chronic conditions being labeled as “incident.” Additional distortions arise from surveillance bias, whereby groups with higher health- 4 seeking behavior (HSB) appear to have inflated incidence simply due to more frequent medical encounters. These vulnerabilities, when uncorrected, often generate epidemiological paradoxes: findings that contradict established incidence rates, violate basic epidemiological constraints, or produce hazard ratios incompatible with clinical reality. As documented in the causal-inference literature (e.g., Hernán & Robins, 2006), failure to address selection bias, confounding, and outcome misclassification can transform real-world data into artifacts of study design rather than accurate representations of biological risk [2-5]. The stakes of methodological rigor are particularly high when analyses touch on matters of major public health importance, such as mass vaccination campaigns. Flawed conclusions based on administrative Big Data have the potential to misinform policy, undermine public trust, and generate unnecessary alarm or reassurance. In this context, a recent study [6] on the association between COVID-19 vaccination and incident psychiatric diagnoses in a South Korean cohort offers a recent and illustrative example of these methodological challenges. Unfortunately, as we show in the following Section, the study’s construction of the unvaccinated control cohort failed to correct for major differences in age, comorbidities, and baseline health-seeking behavior. Combined with substantial misclassification of psychiatric outcomes, this led to three striking and mutually incompatible statistical paradoxes. These paradoxes are not biologically interpretable; rather, they arise directly from uncorrected selection bias and errors in outcome classification from the sourced data. The specific nature and quantitative details of these paradoxes are presented below in the Results Section. 5 Methods This work presents a methodological and epidemiological re-evaluation of the results reported in [6] regarding the incidence of psychiatric adverse events following COVID19 vaccination within a South Korean National Health Insurance Service (NHIS) cohort. Because the original patient-level data were not publicly available, our analysis focuses on assessing the internal coherence, epidemiological plausibility, and methodological validity of the study’s published incidence rates and Hazard Ratios (HRs). Our approach consisted of three complementary analytic strategies: Benchmarking Against Established National Epidemiology Incidence and prevalence values reported in [6] were compared with reference estimates derived from large, independent South Korean epidemiological surveys and NHIS-based studies. These benchmark sources provide stable national estimates for disorders such as Schizophrenia, Bipolar Disorder, and Anxiety Disorders [7-9]. For each condition, we evaluated whether the reported quarterly incidence in the vaccinated and unvaccinated cohorts fell within the mathematically plausible range derived from: i) annual national incidence (converted to quarterly values), ii) annual prevalence (used as an upper bound for incidence in chronic disorders), iii) diseasespecific clinical properties (e.g., chronicity, typical onset patterns). 6 Logical and Biostatistical Consistency Checks To determine whether the reported HRs could represent valid causal estimates, we examined: a) the expected directionality of incidence across risk groups (e.g., older and less healthy groups should not exhibit markedly lower incidence of chronic psychiatric disorders); b) internal coherence between HRs and raw incidence values; c) compatibility with fundamental epidemiological constraints (e.g., incidence ≤ prevalence; chronic disorders cannot show sudden collapse of incidence without mechanistic justification); d) ratio tests and incidence-fraction comparisons to identify implausible or contradictory patterns. This included evaluating whether the study’s results violated basic population-level laws—for example, the impossibility of a shortterm incidence exceeding an annual prevalence benchmark for a chronic illness. Assessment of Potential Bias Mechanisms We evaluated the likely impact of three key biases known to affect psychiatric epidemiology in administrative datasets, specifically: Selection bias, arising from major baseline differences between vaccinated and unvaccinated groups (age, comorbidities, health-seeking behavior), especially given the authors' use of a random sample of unvaccinated individuals to construct the control cohort; Outcome misclassification, including under-ascertainment of chronic cases, temporal diagnostic drift, and heterogeneity in ICD-10 coding practice; Surveillance bias, in which differential healthcare utilization leads to artificially elevated incidence in one exposure group. By mapping these mechanisms onto the patterns observed in the published 7 results, we assessed whether the paradoxical findings could arise from biological effects or were more parsimoniously explained by methodological flaws. Outcome of the Methods Through this structured tri-level evaluation, that is benchmark comparison, mathematical consistency analysis, and bias-mechanism mapping, we identified three epidemiological paradoxes that cannot be reconciled with established national data or with fundamental epidemiologic principles. These paradoxes, detailed in the Results section, point conclusively to selection bias and misclassification as the primary drivers of the reported associations. Results: Identified Statistical Paradoxes We present the results obtained in a narrative manner, albeit supported by numbers, illustrating the paradoxes that emerged from our analysis, along with the corresponding statistical anomalies. Paradox 1: Implausible Hazard Ratio for Schizophrenia Spectrum Disorders A major inconsistency [6] concerns the reported Hazard Ratio (HR) for Schizophrenia (ICD-10 F20–F29), presented as a strong protective association (HR = 0.231, 95% CI: 0.164–0.326). This finding is unexpected because, according to Table 1 of the same study, the vaccinated cohort is approximately 10.5 years older thus implying a naturally less favorable baseline health profile compared to the unvaccinated group. Historical incidence estimates from the Korean National Health Insurance Service (NHIS) 8 indicate annual rates of 77.6–88.5 per 100,000 individuals [9], corresponding to an expected quarterly incidence of approximately 2.0–2.2 per 10,000. This range is in line with the incidence observed in the unvaccinated group (1.98 per 10,000), but sharply contradicts the value reported for the high-risk vaccinated group (0.51 per 10,000). The latter is about four times lower than both the control group’s incidence and established national benchmarks. Given that Schizophrenia is a chronic disorder with stable epidemiological dynamics, such a marked reduction in incidence within an older and less healthy group is statistically implausible. The pattern is best explained by a misclassification bias, whereby chronic schizophrenia cases within the vaccinated cohort were systematically excluded or incorrectly categorized as “non-incident” thereby distorting the numerator and invalidating the resulting HR. Paradox 2: Bipolar Disorder Incidence Exceeding National Epidemiological Limits A second anomaly arises from the incidence of Bipolar Disorder (BD) in the unvaccinated group, reported as 1.39 per 10,000 over a three-month period [6]. This figure exceeds the annual prevalence of a severe, chronic, and clinically related condition, i.e., the Borderline Personality Disorder, reported as 1.06 per 10,000 in the NHIS data [7]. A fundamental epidemiological principle dictates that the short-term incidence of a chronic condition cannot exceed the annual prevalence of a comparable chronic disorder within the same population. Thus, the BD incidence reported in the 9 control group is mathematically incompatible with established national prevalence estimates. This discrepancy indicates that the algorithm used in [6] to identify “new cases” of BD is methodologically flawed. Consequently, the associated HR (0.672; Abstract of [6]) lacks statistical validity. Paradox 3: Simultaneously Deflated HRs for Chronic Diseases and Inflated HRs for Common Disorders The third paradox stems from the coexistence of (a) extremely low HRs for chronic disorders (Schizophrenia, BD) and (b) elevated HRs for highly prevalent conditions such as Anxiety, Dissociative, Stress-Related, and Somatoform Disorders (F40–F48). Authors of [6] report an HR of 1.439 (95% CI: 1.322–1.568) for this category, based on an incidence of 28.41 per 10,000 in vaccinated individuals versus 20.27 per 10,000 in unvaccinated individuals. Yet national data indicate a 12-month prevalence of Anxiety Disorders of 3.1% (310 per 10,000) [8]. Given that incidence over a threemonth period must be below the quarterly fraction of this prevalence (approximately 77.5 per 10,000), both values reported in [6] fall substantially below epidemiological expectations. This under-ascertainment signals a severe misclassification bias affecting both vaccinated and unvaccinated cohorts. In essence, the simultaneous presence of: i) artificially low HRs for chronic, persistent conditions, and ii) artificially high HRs for common conditions with high baseline prevalence, within the same unadjusted cohort is incompatible with any plausible 16 National Health Insurance Service Customized Database. Yonsei Med J. 2023;64(9):566–72. doi: 10.3349/ymj.2023.0071. 8. Rim SJ, Hahm B-J, Seong SJ, Park JE, Chang SM, Kim B-S, et al. Prevalence of Mental Disorders and Associated Factors in Korean Adults: National Mental Health Survey of Korea 2021. Psychiatry Investig. 2023;20(3):262–272. doi: 10.30773/pi.2022.0307 9. Cho SJ, Kim J, Kang YJ, Lee SY, Seo HY, Park JE, et al. Annual Prevalence and Incidence of Schizophrenia and Similar Psychotic Disorders in the Republic of Korea: A National Health Insurance Data-Based Study. Psychiatry Investig. 2020 Jan 25;17(1):61–70. doi: 10.30773/pi.2019.0041 10. Chemaitelly H, Ayoub H, Coyle P, et al. Assessing healthy vaccinee effect in COVID-19 vaccine effectiveness studies: a national cohort study in Qatar. eLife. 2025;14:e103690. doi: 10.7554/eLife.103690 11. D’Agostino RB Jr. Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med. 1998;17(19):2265–81. doi: 10.1002/(sici)10970258(19981015)17:19<2265::aid-sim918>3.0.co;2-b 12. Roccetti M, Cacciapuoti G,. Beyond the Gold Standard: Linear Regression and Poisson GLM Yield Identical Mortality Trends and Deaths Counts for COVID19 in Italy: 2021–2025. Computation, 2025;13(10):233. doi: 10.3390/computation13100233 17 Author Information Marco Roccetti: Department of Computer Science and Engineering, University of Bologna, 4016 Bologna, Italy, [email protected]. ORCID: 0000-0003-12648595 Contributions MR conducted the data analysis, conceptualized the statistical arguments, and wrote the paper. Corresponding author Correspondence to Marco Roccetti: [email protected]. Ethics approval This study does not involve humans, animals, plants. It uses only publicly available, aggregated data that contains no private information. Therefore, ethical approval is not required Competing interests The author declares no competing interests. Funding This research received no external funding 18 Data Availability The data presented here is either included directly or was extracted from the referenced documents. All calculations are easily reproducible based on the definitions provided