Quantifying Structural Selection Bias in Observational Cohort Data: A Ponderation Analysis of Age- Specific Incidence Rates to Inform Vaccine Safety Verification

1 Original Research Quantifying Structural Selection Bias in Observational Cohort Data: A Ponderation Analysis of Age - Specific Incidence Rates to Inform Vaccine Safety Verification Marco Roccetti Department of Computer Science and Engineering University of Bologna, 40126, Italy [email protected] ORCID: 0000-0003-1264-8595 Abstract Background: A recent nationwide cohort study reported an unadjusted Hazard Ratio (HR) of 2.714 for Vitiligo incidence following COVID-19 vaccination, indicating a major safety concern. This finding was based on cohorts with an ≈ 11-year age difference, immediately raising critical concerns regarding extreme structural selection and detection bias. Objectives: We hypothesize that this extreme association is an artifact of a fatal methodological flaw, challenging the study's internal validity and subsequent external validity. We aim to quantitatively separate the HR attributable to the structural age imbalance (HR Structural) from the residual HR (HR Residual), which quantifies the uncorrected methodological failure. We further perform a plausible recalculation of risk to demonstrate the complete collapse of the risk signal upon correcting the methodological failure in the baseline cohort. Methods: We performed a stratified ponderation analysis using the age distribution of the scrutinized study’s cohorts (Vaccinated, mean age=56.32 years vs Non-Vaccinated, mean age=45.51 years) and applied established national age-specific Vitiligo incidence rates (IR) from external epidemiology. 2 Results: The HR Structural was calculated to be 1.211. The remaining HR Residual of 2.24 quantifies the uncorrected methodological failure. The NV cohort's observed incidence rate (0.67/10,000) was found to be nearly 70% lower than the expected rate (2.2133/10,000), providing quantifiable evidence of profound noncomparability. The subsequent recalculation of risk, correcting for this baseline failure, reduces the observed HR of = 2.714 to an HR Corrected of alomost a unity, thus completely annulling the signal of risk due to vaccination. Discussion: The HR= 2.714 of the scrutinized study is an unstable statistical artifact. The overwhelming majority of the observed association is a consequence of a fatal design flaw. The HR Corrected of almost 1 confirms that correcting the methodological error eliminates the risk signal, demonstrating a severe lack of internal and external validity of the original study. Keywords: Covid-19 vaccine safety, Structural selection bias, Ponderation analysis, Hazard ratio decomposition, Internal/External validity, Detection bias 1. Introduction Observational studies utilizing national registries, such as those conducted in South Korea [1], represent a critical resource for post-marketing surveillance and vaccine safety verification. However, the reliance on pre-existing data necessitates strict adherence to established methodological standards, notably the STROBE (STrengthening the Reporting of Observational studies in Epidemiology) guidelines [2]. The primary goal is to ensure internal validity, that the observed association is real within the study context, which is a prerequisite for achieving external validity (that is generalizability to the broader population). The recent study published in [1] reported a strikingly high, unadjusted Hazard Ratio (HRGross) of 2.714 for Vitiligo following COVID-19 vaccination, based on a comparison between a Vaccinated (V) cohort (mean age=56.32 years) and a Non-Vaccinated (Non-Vaccinated) cohort (mean age=45.51 years). This ≈11-year age difference immediately flagged critical concerns regarding confounding by indication and immortal time bias [3]. The sheer magnitude of the ≈11-year age difference, coupled with the cumulative incidence rates observed (2.22 vs 0.67 per 10,000), strongly suggests that the cohorts were inherently non-comparable. 3 Our analysis posits that the reported HR=2.714 is not a reflection of a robust biological signal but rather a quantitative measure of a fatal design flaw. We hypothesize that an Extreme Structural Selection and Detection Bias was introduced by defining the cohorts in a manner that artificially minimized the baseline risk in the NV group, while simultaneously maximizing the detection and prevalence risk in the V group. We present a rigorous, quantitative method, that is a stratified ponderation analysis using external South Korean national age-specific incidence data, to decompose the observed HR and isolate the true contribution of the structural bias [6-10]. The quantitative findings of the present research confirm this hypothesis: the structural age difference alone accounts for a calculated Structural Hazard Ratio (HR Structural) of 1.211. Importantly, we demonstrate that the non-comparability of the baseline cohort (NV) caused its observed incidence rate to drop nearly 70% below the expected rate. The overwhelming majority of the observed association is captured by the residual Hazard Ratio (HR Residual of 2.240) which stands as a clear measure of the uncorrected methodological failure. We further show that applying a reasonable recalculation of risk, which corrects the failure of the baseline incidence rate, reduces the observed risk signal from 2.714 to a clinically insignificant HR of approx. 1 (HR Corrected). Essentially, the quantitative findings of the present research has move along two concatenated directions. First we confirm this hypothesis that the structural age difference alone accounts for a calculated structural Hazard Ratio (HR Structural) of 1.211. This means that the observed demographic imbalance explains the 12.31% of the initially reported excess risk. Then, we show that the majority of the association captured by the residual Hazard Ratio (HR Residual) of 2.240 simply stands as a clear measure of the uncorrected methodological failure. This substantial residual value not only strongly indicates that the cohorts were not subject to a common support, leading to a profound violation of the assumption of comparability required by the Cox Proportional Hazards model but can be also explained if one takes into account the drop of nearly 70% in the incidence rate of Vitiligo in the NV groupPutting this information into the calculation easily leads to a corrected value of HR of almost 1 that practically annuls the risk difference between V and NV subgroups. Ultimately, the goal of this re-evaluation is to reassert the imperative for epidemiological validity in studies of vaccine safety derived from observational data. We demonstrate that sophisticated statistical adjustments cannot remedy fundamental flaws in cohort design where non-measured confounding factors, such as health-seeking 4 behavior, surveillance frequency and distribution of the disease incidence peaks are unevenly distributed [7]. By quantitatively isolating and measuring the non-causal structural bias, our analysis provides a critical framework for interpreting extreme risk estimates and ensuring that public health conclusions are based on associations that are epidemiologically sound, rather than artifactual. 2. Methods We here provide all the fundamental methods and data useful for the aim of pondering the structural bias on which we are inestigating. 2.1 Study Data and Baseline Characteristics We extracted the following key data from [1] to establish the basis of the structural bias as reported in the following Table 1. Table 1: Baseline Characteristics and Unadjusted Incidence Rates from [1]. Cohort Mean Age Standard Deviation (SD) Cumulative Incidence Rate (at 3 mo) (per 10,000 p-y) Non-Vaccinated (NV) 45.51 years 17.31 P(NV) = 0.67 Vaccinated (V) 56.32 years 16.55 P(V) = 2.22 2.2 Stratified Ponderation Analysis We performed a first preliminary quantitative analysis by combining the age distribution percentages P(i) of the V and NV groups of [1] with independent, established age-specific annual incidence rates IR(i) for Vitiligo in South Korea, based on 2019 data as reported in [5]. It is reported in Table 2 below. 5 Table 2: Input Data for Ponderation Analysis Age Group South Korean IR (per 10,000 p-y) % in NV Group P(i, NV) % in V Group P(i,V) < 20 y 3.4241 Not included in [1] Not included in [1] 20-29 y 1.5717 18.46% 9.92% 30-39 y 1.7813 25.49% 7.70% 40-49 y 1.9053 20.92% 10.82% 50-59 y 2.5874 14.16% 24.76% >= 60 y 3.3643 20.97% 45.79% Total — 100% 100% A key observation is that, it should be noted here that the composition of the NV group is strongly influenced by the presence of young individuals who are already past the first peak of Vitiligo onset, contrasting with the massive presence in the NV group of older individuals entirely included in the interval where the second incidence peak of Vitiligo manifests, which is universally recognized as having a bimodal distribution (under 20, over 40/50 years) [5]. 2.3 Calculation of HR Structural and HR Residual The Expected Annual Incidence Rate, IR(Expected,) for each cohort, based solely on its structural age composition, can be calculated using the following Formula 1: IR(Expected) = ∑(IR(i) × P(i)). (1) 6 Where IR(i) are the South Korean age-specific incidence rates from external data (Table 2) and P(i) are the proportional distributions of the respective cohort (V or NV) reported in the same Table 2. Applying this ponderation to the Non-Vaccinated (NV) cohort demographics, we obtain the baseline expected incidence, IR(NV, Expected) exactly as follows: IR(NV, Expected) = (1.5717 × 0.1846) + (1.7813 × 0.2549) + (1.9053 ×0.2092) + (2.5874 × 0.1416) + (3.3643 × 0.2097 ≈ 2.2133 / 10,000. Similarly, applying the ponderation to the Vaccinated (V) cohort demographics yields IR(V, Expected): IR(V, Expected) = (1.5717 × 0.0992) + (1.7813 × 0.0770) + (1.9053 × 0.1082) + (2.5874 × 0.2476) + (3.3643 × 0.4579) ≈ 2.6809 / 10,000. This allowed us to calculate the HR Structural as follows: HR Structural = 2.6809 / 2.2133 ≈ 1.2114. Finally, the HR Residual can be computed as the ratio between the HR provided in [1] (termed HR Observed) and our computed HR Structural: HR Residual = HR Observed / HR Structural = 2.714 / 1.2114 ≈ 2.2404. The consistency of this multiplicative decomposition is confirmed by the near-exact reconstruction of the observed value: HR Structural × HR Residual ≈ 1.2114 × 2.2404 ≈ 2.714. 3. Results We here provide the two main set of results stemming from our analysis whose methodology has been anticipated in the previous Section. 3.1 Decomposition of the Observed Hazard Ratio and Incidence Discrepancy Following the calculation of the Expected Incidence Rates IR(Expected) based solely on the structural age compositions of the two cohorts (Section 2.3), we proceeded to quantify the true extent of the methodological failure. This involved decomposing the high, observed HR Observed =2.714 from [1] into two distinct components: the risk attributable purely to the structural age imbalance (HR Structural) and the risk stemming from all other 7 uncorrected design flaws and selection biases (HR Residual). Since Hazard Ratios combine multiplicatively, that is HR Observed = HR Structural × HR Residual, the HR Residual thus acts as a precise metric for the degree of non-comparability that persists despite accounting for the known age difference. The breakdown of this risk is definitely presented in Table 3. Table 3: Decomposing the Observed Hazard Ratio (HR=2.714 [1]) Parameter Description Value Contribution to Excess Risk (HR−1) HR Observed Unadjusted Hazard Ratio from [1] 2.714 100% HR Structural HR due to Age Structural Bias Alone 1.2114 12.31% HR Residual HR Unexplained by Structural Age Bias 2.2404 72.34% The most salient finding of the ponderation analysis shown above is the extreme divergence between the expected and observed incidence rates in the reference cohort. The demographic composition of the Non-Vaccinated (NV) subgroup predicted an expected Incidence Rate of 2.1611 / 10,000 based on established national age-specific rates. However, the rate observed in [1] was only 0.67 / 10,000. This discrepancy means the observed incidence in the reference cohort was nearly 70% lower than methodologically expected, providing immediate, quantifiable evidence of the profound non-comparability of the baseline groups. Furthermore, it is crucial to emphasize how our robust, age-specific ponderation analysis has shown that the structural age difference explains only 12.31% of the excess risk signaled by the authors of [1]. The overwhelming majority of the association (72.34%, resulting in an HR Residual of ≈ 2.24) is entirely attributable to uncorrected 8 methodological flaws which should be attributed to a basic failure in the construction of the cohort and theur subgroups as hypothesized below. 3.2 Recalculation of Risk: The Collapse of the Observed Association The massive quantitative discrepancy identified in Section 3.1, where the IR(NV, Observed) of 0.67 / 10,000 falls short of the IR(NV, Expected) of 2.2133 / 10,000, by nearly 70%, due to non-comparability bias, allows us to perform a fundamental recalculation of risk. We obviously hypothesize that had the NV cohort been properly matched for unmeasured confounders, the true baseline risk would have approached the expected incidence rate of 2.2133 / 10,000. Furthermore, if we assume, for the sake of this calculation, that the Detection Bias acting on the Vaccinated (V) cohort is negligible (i.e., that IR(V, Observed) ≈ IR(V, True), we can calculate the resulting Hazard Ratio (HR Corrected) as follows: HR Corrected = IR(V, Observed) / IR(NV, Expected) ≈ 1. This simple recalculation demonstrates that by correcting the methodological failure of the denominator alone, the observed risk signal of 2.714 collapses entirely to ≈ 11. An HR of 1 would mean a clinically insignificant increased risk, effectively reducing the study's finding to a null hypothesis. 4. Discussion We will summarize here the key takeaways of this discussion into two primary issues to ensure they are properly highlighted, noting that at the heart of the matter lie problems of loss of comparability and resulting clinical significance. 4.1 The Collapse of Internal Validity: The Double Distortion Mechanism The persistence of the high residual HR (2.24) after robust adjustment for age structure (HR Structural = 1.21) provides a definitive quantitative proof that the cohorts have been constructed as non-comparable. The study's design of [1] suffers from a double distortion mechanism that fundamentally violates the core premise of observational epidemiology. First, the artificial baseline depression of the NV sub-group is attributed to the Selection Bias revealed in Section 3.1: the IR(NV) being 70% lower than expected indicates that the individuals composing the denominator are not 9 representative of the general population. This massive deficit is caused by the inclusion of a minority of older individuals (>= 60 years) who are part of an exceptionally healthy survivor cohort and/or have minimal medical interaction [8], thus ensuring profound under-detection of Vitiligo cases. This failure to capture the true baseline risk, depressing the denominator from the expected 2.2133 / 10,000 to the observed 0.67 / 10,000, is the primary quantitative driver of the resulting high HR Residual ≈ 2.24. Second, we confront an inflated incidence by detection and risk in the V subgroup. Conversely, the V group's composition of [1] (≈ 70% aged >= 50 years) guarantees maximal risk exposure because this demographic actively intercepts the second, higher incidence peak of Vitiligo (as seen in the IR(i) data before). This structural flaw, coupled with the heightened surveillance and utilization bias typical of vaccinated groups, ensures that both the intrinsic risk and the diagnostic detection rate are maximized. This extreme structural separation, especially in the high-risk and high-surveillance age categories, represents a violation of the common support assumption. The Cox model employed in the scrutinized study, therefore, did not compare like with like, but rather measured the risk differential between an artificially clean control group and a maximally surveilled risk group. 4.2 Clinical and External Validity Implications for Vaccine Safety Verification The extreme HR Residual ≈ 2.24 cannot be interpreted as a genuine biological effect. The recalculation of Risk performed in Section 3.2, which demonstrates the complete collapse of the risk signal to HR ≈ 1 upon correcting the methodological baseline failure, provides compelling evidence against a genuine biological association. If the HR Residual were indeed the genuine vaccine effect, it would imply that the extensive methodological failure observed in the denominator (≈ 70% under-reporting) must be entirely compensated by an unmeasured bias in the numerator. Epidemiologically, it is far more plausible that the 2.24 represents the cumulative effect of uncorrected Selection and Detection Bias, a direct consequence of the study's faulty design. The clinical implication is that the reported HR = 2.714 of [1] is gravely misleading for patients and clinicians. It does not reflect the incremental risk of vaccination but rather the difference in underlying health and healthcare seeking behavior between two demographically distinct groups in South Korea. This methodological failure is a serious breach of epidemiological reporting standards in the sense of the STROBE protocol and undermines the utility of national registry data for assessing vaccine safety signals when proper cohort matching is neglected.