Shortcomings in the Cochrane review on zinc and the common cold by Nault et al. (2024)
Abstract
Over a dozen placebo-controlled trials have examined the effects of zinc lozenges on common cold symptoms. Several meta-analyses of the trials with different statistical approaches have demonstrated strong evidence that properly composed zinc lozenges can shorten the duration of colds. A Cochrane review on zinc for the common cold was published in 2013; however, it was withdrawn because of data plagiarism. In 2024, a new Cochrane review was published on zinc for the common cold. The review concluded that “On the basis of this review, the current evidence is insufficient to provide firm conclusions or recommend zinc supplementation for the prevention or treatment of the common cold”. This conclusion is substantially different from many other meta-analyses which have indicated benefit. Therefore, I critically appraised the new Cochrane review (2024) and describe here several concerns which explain the different conclusions. There are errors in the inclusion and exclusion of randomized trials, and in data extraction and statistical analysis in the Cochrane review (2024). The Cochrane review authors consider that zinc lozenges are a form of dietary supplement; however, the effect of zinc lozenges is local and not explained by effects caused by ingestion. The analysis of adverse effects is misleading; based on the literature, it seems highly unlikely that the dosage used in controlled trials, 80–92 mg/day of zinc for 1-2 weeks, would cause severe adverse effects. In conclusion, the Cochrane review (2024) is not an adequate analysis of the available data on zinc lozenges for the common cold.
Full text
Shortcomings in the Cochrane review on zinc and the common cold by Nault et al. (2024) Harri Hemilä, MD, PhD Department of Public Health, University of Helsinki, Helsinki, Finland https://orcid.org/0000-0002-4710-307X https://loop.frontiersin.org/people/866706 https://www.scopus.com/authid/detail.uri?authorId=56223570700 https://pubmed.ncbi.nlm.nih.gov/?term=hemila+zinc https://www.mv.helsinki.fi/home/hemila/zinc.htm 2024-9-14 Available at: https://doi.org/10.5281/zenodo.13762570 Summary Over a dozen placebo-controlled trials have examined the effects of zinc lozenges on common cold symptoms. Several meta-analyses of the trials with different statistical approaches have demonstrated strong evidence that properly composed zinc lozenges can shorten the duration of colds. A Cochrane review on zinc for the common cold was published in 2013; however, it was withdrawn because of data plagiarism. In 2024, a new Cochrane review was published on zinc for the common cold. The review concluded that “On the basis of this review, the current evidence is insufficient to provide firm conclusions or recommend zinc supplementation for the prevention or treatment of the common cold”. This conclusion is substantially different from many other meta-analyses which have indicated benefit. Therefore, I critically appraised the new Cochrane review (2024) and describe here several concerns which explain the different conclusions. There are errors in the inclusion and exclusion of randomized trials, and in data extraction and statistical analysis in the Cochrane review (2024). The Cochrane review authors consider that zinc lozenges are a form of dietary supplement; however, the effect of zinc lozenges is local and not explained by effects caused by ingestion. The analysis of adverse effects is misleading; based on the literature, it seems highly unlikely that the dosage used in controlled trials, 80–92 mg/day of zinc for 1-2 weeks, would cause severe adverse effects. In conclusion, the Cochrane review (2024) is not an adequate analysis of the available data on zinc lozenges for the common cold. 1
Contents Page Strong evidence that properly composed zinc lozenges can shorten the duration of colds 3 Flaws in the preceding Cochrane review (2011/2013/2015) on zinc for the common cold 4 Conclusion of Cochrane review (2024) on therapeutic zinc for the common cold 5 Revision of Nault’s main meta-analysis on zinc treatment of common cold duration 6 Pharmacology of zinc lozenges is not considered in the Cochrane review (2024) 11 Zinc lozenges are not all equal: many lozenges are ineffective 14 Inclusion of flawed trials, and trials inconsistent with inclusion criteria 18 Ignoring a 48-fold variation in zinc dosage in the nasal administration trials 23 Unsound subgroup comparison by zinc dosage and by the type of zinc salt 26 Analysis of ongoing colds at the end of the follow-up is a statistically unsound approach 28 Experimental colds and natural colds should not be pooled in the same meta-analyses 30 Misunderstandings about RCT methods in the Risk of Bias (RoB) assessment 32 Analysis of adverse effects is not appropriate 37 PICO criteria were not considered in the Cochrane review (2024) 43 Potential explanations for the negative results in the Hemilä (2020) trial were ignored 46 Misleading Background section 48 Practical implications and guidance for further research 49 Flaws in data extraction 52 Errors in other Cochrane reviews 57 References 59 2
Strong evidence that properly composed zinc lozenges can shorten the duration of colds Over one dozen randomized trials (RCTs) have examined the effects of zinc lozenges on the duration of the common cold. The findings have been variable, however there is a plausible explanation for the heterogeneity. Eby demonstrated that a substantial proportion of the variation in the results can be explained by the amount of free zinc ions released from the zinc lozenges in that low levels of free zinc ions explained negative findings, and high levels of free zinc ions explained positive findings [1-5]. Bakar also showed that the level of free zinc ions closely correlated with the observed efficacy of zinc lozenges [6]. Ebys’s analyses, based on solution chemistry, are informative. However, the impact of zinc dosage level can also be seen in a simple analysis by the total zinc dosage. There has been more than a 7-fold variation in the daily dose of elemental zinc in the zinc lozenge trials. In 2011, I showed that 5 RCTs that used low doses of zinc (<75 mg/day elemental Zn) uniformly found no benefit from zinc lozenges. In contrast, 3 trials used zinc acetate in daily doses of ≥75 mg elemental Zn and the pooled result indicated a 42% (95% CI: 35-48%) reduction in the duration of colds [7]. Five trials used zinc gluconate in daily doses of ≥75 mg, and colds were shortened on average by 20% (95% CI: 12-28%) [7]. Although the point estimate of effect was quite different for acetate and gluconate salts in the 2011 analysis, a further metaanalysis directly comparing trials of zinc acetate lozenges with trials of zinc gluconate lozenges did not demonstrate a significant difference [8]. The pooled estimate from 7 trials of acetate and gluconate salts with zinc dose ≥75 mg/day indicated that common colds were shortened on average by 33% (95% CI: 21-45%; P = 0.000 000 05 = 10-7) [8]. Such a very narrow confidence interval – as also indicated by the P-value – is very strong evidence that properly formulated zinc lozenges can reduce the duration of colds. Because the evidence for zinc acetate lozenges was stronger, the findings of the zinc acetate lozenge trials were analyzed further. In an Individual Patient Data (IPD) meta-analysis, colds were shortened by 36-40% (2.7-day reduction for the average of 7-day colds in the placebo group), and the effect of zinc acetate lozenges was not modified by age, gender, ethnic group, allergy status, smoking, or baseline severity of the common cold [9]. In another IPD metaanalysis using Cox regression, zinc acetate lozenges increased the recovery rate from colds by rate ratio (RR) = 3.1 [10]. Based on data available for 3 zinc gluconate trials the increase in recovery rate was by RR about 2 to 3 [10]. Zinc lozenges are dissolved in the oropharyngeal region, thus it is possible there are differences in the size of the effect of zinc lozenges on the duration of pharyngeal symptoms compared with nasal symptoms. However, a meta-analysis of the 3 zinc acetate lozenge trials did not find substantial differences in the effects on various respiratory symptoms. Zinc acetate lozenges shortened the duration of nasal discharge on average by 34%, nasal congestion by 37%, sneezing by 22%, scratchy throat by 33%, sore throat by 18%, hoarseness by 43%, and cough by 46% [11]. Muscle ache was shortened by 54%. The quantile treatment effect (QTE) analysis of the 3 zinc acetate lozenge trials firmly demonstrated that the average treatment effect (ATE) of 2.7 days [9] was inconsistent with the effects on short and long colds. The ATE exaggerates the effect on short colds, and underestimates the effect on long colds [12-14]. The QTE analysis shows that the relative scale much better captures the effect of zinc acetate lozenges, hence the 36% reduction in common cold duration is a more useful estimate for the effect of zinc acetate lozenges than the 2.7-day reduction in duration [12,14]. 3
Flaws in the preceding Cochrane review (2011/2013/2015) on zinc for the common cold Cochrane reviews are often promoted as high-quality systematic reviews that can be trusted: “Cochrane: Trusted evidence. Informed decisions. Better health.” https://www.cochrane.org (Accessed 2024-9-13). In 2011, I found that there were severe errors in the Cochrane review “Zinc for the common cold” (2011) [15]. I provided formal feedback in which I pointed out several flaws and encouraged the authors to correct them [16]. However, when the next version of the Cochrane review was published in 2013 [17], almost all the errors remained. In the 2023 version, there also seemed to be plagiarism of text and data from another article as documented in a separate report [18]. Unfortunately, the editors of the Cochrane Acute Respiratory Infections group had not checked that the flaws demonstrated in 2011 were corrected in the 2013 revision. Eventually, this process ended with the withdrawal of that Cochrane review “Zinc for the common cold” in 2015: Hemilä identified multiple errors in this Cochrane Review and made allegations of plagiarism of text and data from a previously published systematic review… The Editor in Chief carried out further investigation into the alleged plagiarism of data, with the co‐ operation of the review authors, who provided supplementary information in support of their work. The allegations related to the derivation of means and standard deviations of data from some of the included studies. Although the authors acknowledge and cite the Hemilä 2011 review, the Editor in Chief considered that the authors’ explanation regarding some similarities in presented data between the two reviews was not conclusive. This version of the review will therefore remain withdrawn [19]. As a result of the withdrawal of the Cochrane review on zinc for the common cold (2013), the associated JAMA Clinical Evidence Synopsis was also retracted [20-22]. Interestingly, the first author of the Cochrane zinc for the common cold review [15,17,19], which was withdrawn because of plagiarism, still serves as an editor of the Cochrane Acute Respiratory Infections group despite violating publication ethics: “Contact Editors: Dr Meenu Singh, India” https://ari.cochrane.org/contact-us (2024-9-10). There are also flaws in certain other Cochrane reviews; see the last section of this document. 4
Conclusion of Cochrane review (2024) on therapeutic zinc for the common cold A new Cochrane review on zinc for the common cold by Nault et al. was published in 2024 [23]. The conclusions for treatment of the common cold were rather negative: On the basis of this review, the current evidence is insufficient to provide firm conclusions or recommend zinc supplementation for the prevention or treatment of the common cold. [23, p.28: Implications for practice]. This was based on Nault’s calculation in Analysis 9.1 [23, p.163], which led to the estimated effect: When zinc is used for cold treatment, there may be a reduction in the mean duration of the cold in days (MD ‐2.37, 95% CI ‐4.21 to ‐0.53; I² = 97%; 8 studies… [23, p.2: Abstract]. This estimate has a much wider confidence interval than the above-described estimate [8] (see above), which indicates that the efficacy of zinc may be doubtful. However, there are numerous shortcomings in the Cochrane (2024) meta-analysis on zinc and the common cold by Nault [23]. This document describes the shortcomings. 5
Revision of Nault’s main meta-analysis on zinc treatment of common cold duration There are 4 major shortcomings in Nault’s analysis of zinc for treating the common cold. First, in the general community there is a large variation in the duration of colds, which can last for 1 day to 3 weeks and over. Obviously, the 1-day colds cannot be shortened by the 2.37 days calculated by Nault. Thus, the “2.37-day” ATE [23] is not applicable over the wide variation in cold duration. Furthermore, given the ATE of 2.37 days, and the impossibility of 1-day colds to be shortened by more than 1 day, it is obvious that the effect of zinc lozenges must be greater on some colds that are longer than 2.37 days. This is demonstrated with the QTE analysis [12-14]. Previously, it has been shown that the relative scale much better captures the effect of treatments on many continuous outcomes such as the duration of colds [12-14,24-28]. Therefore, the relative scale (percentage) was used in our Cochrane review on vitamin C and the common cold in the 2004 [29], 2007 [30,31] and 2013 versions [32]. The relative scale estimate is applicable to the 1-day colds as well as to the 3-week colds. Metaanalysis on the percentage effect scale can be done easily in the Cochrane RevMan program [29,30,32]. Second, Nault writes that: We undertook meta-analyses only where meaningful, that is, if the treatments, participants, and the underlying clinical question(s) were similar enough for pooling to make sense. Because the likely mechanisms and potential adverse effects of oral and intranasal zinc differ, we analysed trials of oral zinc (including tablets, capsules, syrups, or lozenges) separately from trials of intranasal zinc (including sprays or gels). If statistical pooling was not appropriate, we produced a narrative summary [23, p.10]. We pooled data from studies that we judged to be clinically homogeneous [23, p.11]. However, in the calculation of the above-mentioned 2.37-day ATE, Nault pooled 3 trials that administered zinc nasally [33-35] with 5 trials that administered zinc as lozenges [36-40], see Figure 1. The two treatments are not similar enough for pooling to make sense because administering zinc to different anatomical regions might cause different effects. Pooling the zinc lozenge trials with nasal zinc trials in Analysis 9.1 is particularly strange since Nault writes in the Methods section that they did not pool, see the text above. Given that there are 10 authors in the Cochrane review [23], it is surprising that none of them picked up this inconsistency between the reporting of methods and the actual methods. Furthermore, there has been concern over decades that nasal zinc administration may cause long-lasting or permanent anosmia in some patients [41-44]. This concern is also mentioned by Nault [23, p.8], though not properly discussed. Therefore, nasal administration of zinc is not appropriate and estimating the effect of nasal zinc administration is irrelevant unless it emerges that the concerns are unfounded. No concerns about anosmia have been expressed about the administration of zinc lozenges. Finally, in Analysis 9.1, the lowest zinc dose is 0.044 mg/day [34] while the highest zinc dose is 190 mg/day [36]; see Figure 1. When the intervention dose differs by a factor of 4300 the trials are not “similar enough”. 6
Figure 1. Nault’s Analysis 9.1.: Mean duration of colds (measured in days from start to resolution of the cold, as defined by each study) - treatment, Outcome 1: Mean duration of colds/URTIs (treatment): primary analysis. Upper: 3 of the included trials were nasal zinc trials, and 5 were zinc lozenge trials. Lower: the lowest dose of zinc was 0.044 mg/day, and the highest dose was 190 mg/day. 7 Nasal zinc Others zinc lozenges 0.044 mg/day 190 mg/day
Third, Nault does not include the Mossad (1996) trial [45] in Analysis 9.1. Nault includes the Mossad (1996) trial in a dozen other Analyses (10.1, 14.1, 14.2, 14.3, 14.4, 14.6, 14.7, 14.8, 14.10, 14.11, 14.12, 14.13), which implies that Nault has no concerns with the trial methods. Mossad (1996) reported the duration of colds as survival curves. The IPD can be extracted from the curves and thereby the mean and SD can be calculated. That has been published previously, so the mean and the SD were available [7,8,12,27]. Nevertheless, calculating the means and SDs is straight forward. Nault does not give any explanation for the exclusion of the Mossad (1996) trial from the main analysis on treating the common cold (Analysis 9.1). This exclusion of the Mossad (1996) trial from Analysis 9.1 is inconsistent with the Methods section which states that missing data were imputed: If numerical outcome data were missing, such as standard deviations (SD) or correlation coefficients, and we were unable to obtain these from the trial authors, we calculated them from other available statistics… [23, p.11] Fourth, one of the zinc lozenge trials included in the calculation of the 2.37 day ATE was carried out with children: Macknin (1998) [38]. Nault does not discuss potential issues with trials in children. In some vitamin C and common cold trials there was strong evidence that children and adolescents switched their tablets [46,47]. Therefore, common cold trials with children are not as reliable as trials with adults. In addition, there can be differences in the size of the effect between adults and children. Consequently, adults and children were separated in the Cochrane reviews on vitamin C for the common cold [29,30,32]. I revised the meta-analysis corresponding to Nault’s Analysis 9.1 by including only the zinc lozenge trials, adding the Mossad (1996) trial, and using the relative scale. I calculated that zinc lozenges shortened colds in adults by 37% (95% CI: 27-46%; P = 0.000 000 0007 = 10-9); Figure 2. Thus, with the above-justified modifications, using the data that Nault collected, the calculated estimate provides very strong evidence that zinc lozenges can shorten common colds in adults. This is consistent with the previous statistical analyses [1-14]. Inclusion of the trial with children (median age 13) (Macknin 1998) [38] has a minimal effect on the estimate (Figure 2). However, there is a highly significant difference between the 5 adult trials and the Macknin (1998) trial with children (P = 0.0001). Therefore, it is most informative to keep the adult and child trials separate. So far there is no evidence that zinc lozenges have an effect on the duration of colds in children. Nault writes in the Methods that: We considered the following factors to be potential causes of heterogeneity in the effects of the intervention: … the age (children, adults, elderly) of participants However, Nault did not separate the Macknin (1998) trial from the adult trials with zinc lozenges. Investigation would have revealed that there is significant heterogeneity between adult and child trials (Figure 2). 8
Thus, the 5 adult trials with zinc lozenges lead to a conclusion that properly composed zinc acetate and zinc gluconate lozenges can shorten common cold duration in adults by 37% on average. Figure 2. Pooling the zinc lozenge trials included in Nault’s Cochrane review [23]. The child trial by Macknin (1998) is significantly inconsistent with the 5 trials with adults. Based on the ratio of means estimate [25], zinc lozenges shortened colds in adults by 37% (95% CI: 27-46%; P = 10-9). The data are from [23], except for the Mossad (1996) trial, which are from [27]. The included trials were pooled with the metagen function of the R package meta [48-50], using the inverse variance, random effects options. ROM, ratio of means [25]; RoM = 1.0 indicates that the mean duration of colds is identical in the intervention and control groups. 9
Attention to these omissions is critical if we wish to learn the effects of treating common colds with zinc lozenges, and if we wish to reconcile the negative report of Turner et al. [63] with the very positive reports of Petrus et al. [37] and Prasad et al. [39]. For example, Prasad et al. [39] showed that 50% of zinc acetate recipients were well in 3.8 days, compared with 7.7 days for 50% of placebo recipients. This duration data corresponds well with other generally accepted data. In the report by Turner et al. [63], the actual zinc compound exposed to the oral mucosa was not zinc acetate, but nonmiscible fat complexes of zinc. The zinc acetate lozenges were not described as producing a dry or astringent feeling in the mouth; in all cases where the ZIA value is sufficiently high to allow Zn2+ ions to shorten the duration and severity of common colds, there has been and there will be a dry or astringent feeling in the mouth. This dry feeling is identical to the “clean” mouth feeling produced by swishing water in the mouth for 30–60 s. The cumulative effect of the above omissions teaches readers that zinc acetate lozenges in general do not have efficacy against common cold; however, properly made zinc acetate lozenges work very well in reducing the duration of common colds. Turner did not respond to Eby’s criticism, which means that Eby’s criticism was not challenged. Furthermore, the 30 mg/day dose is particularly low. Finally, the actual usage of lozenges was not reported, and therefore it is not clear whether the actual use of the higher dosage was 69 mg/day or much less. Failing to cite Eby’s letter [64] or Eby’s reviews describing the problems of Turner’s (2000) zinc acetate lozenges [3,4], Nault includes the Turner (2000) trials with zinc acetate lozenges in Analyses 11.1, 11.2, 11.3, 11.4, 11.5. and in text page 22. Eby (2006) [65] Zinc orotate lozenges Eby’s commented [4, p.485]: Zinc orotate is tightly bound (0 mg iZn [free zinc ions]) and essentially insoluble, and non-soluble compounds do not release iZn… Lozenges were nearly insoluble and required more than 1 h to dissolve in the mouth. This study was the second component of our 1984 clinical trial, and its results were published in 2 mid-90s books, but were not published as a peer reviewed article until 2006. Nault failed to take this problem into account when including the zinc orotate trial by Eby (2006) in Analyses 10.1, 11.1, 11.4, 11.5. If the goal is to estimate the effect of zinc acetate and/or zinc gluconate lozenges (Figure 3), the Eby (2006) trial is irrelevant. This field of problems in the composition of lozenges was ignored by Nault. It is not appropriate to pool trials purely on the basis that authors used the term “zinc lozenge”. For complex interventions there needs to be justification to conclude that 2 versions of the intervention are “similar enough for pooling to make sense” [23, p.10]. In addition to the chemical composition of zinc lozenges, there has also been variation in the size of the lozenges and the speed at which they are dissolved in the mouth [4, 66 table 4]. In the two Prasad trials (2000, 2008), the lozenges dissolved in about 30 minutes. In the Al16
Nakib (1987) trial, the lozenges dissolved in about 20 minutes, and in the Petrus (1998) and Eby (1986) trials, in about 15 minutes. In the Hemilä (2020) trial, the lozenges dissolved in 8 minutes, which was considered one potential cause for the lack of benefit in that trial [66]. Although several randomized placebo-controlled double-blind trials have shown that properly composed zinc lozenges can shorten common cold duration (Figures 2 and 3) and [1-14], it is not easy for members of the public to find proper zinc lozenges. In his 2010 review [4], Eby wrote that he had looked at the contents of many dozens of zinc lozenges in the US market [4, 67]: Lack of regulatory review through use of homeopathic laws and dietary supplement regulations of the United States – even though throat lozenges are not allowed under the United States Dietary Supplement Health and Education Act of 1994 – has resulted in commercialization of zinc lozenges that are poorly effective to non-effective… A 2008 cold-season market survey by this author of zinc lozenges found in national chain stores in Austin, Texas, USA, showed that none met the criteria of high iZn content and long dissolution times, and nearly all released zero iZn… Most zinc lozenges also contained citric acid… Some zinc lozenges contained non-ionizable (at pH 7.4) zinc compounds including zinc oxide, aspartate, tartrate, picolinate, orotate and various amino acid chelates, which are believed inefficacious against colds. Choices of zinc compounds that do not release iZn at pH 7.4 are believed predicated on commercial desires to avoid the orally astringent and drying nature of iZn, thus a cure for the common cold is precluded by marketing forces, not science. A current assessment of more than 40 over-the-counter zinc lozenges is maintained on the Internet [not available any more, see copy at [67]]. Zinc lozenges marketed in the United States appear to compete based upon taste rather than efficacy [4, p.490]. Of the 40 different brands of over-the-counter zinc lozenges and many variations of them currently available in the US, very few – based upon this analysis and ingredients listed on their labels – appear to release useful amounts of iZn regardless of total zinc content, and none of them can be considered as a cure for common colds. With several exceptions, nearly all appear likely to have a null effect on colds [4, p.490]. 17
Inclusion of flawed trials, and trials inconsistent with inclusion criteria In addition to the above-described problems with the zinc lozenge compositions in certain trials, there are other problems in some trials included by Nault [23]. Nault included the Weismann (1990) [68], Kurugöl (2006) [69], Kurugöl (2007) [70],Vakili (2009) [71], Rerksuppaphol (2013) [72], Sánchez (2014) [73], and Somé (2015) [74] trials in the Cochrane review although these also have various issues. In Analyses 11.1, 11.2, 11.4, Nault included the Weismann (1990) trial [68]. In the Methods section of the Cochrane review, Nault writes: “Types of studies: We included randomised controlled trials (RCTs) and excluded studies using quasi-randomisation because they are susceptible to selection bias” [23, p.8]. However, the Weismann (1990) [68] trial did not use randomization, instead it used alternative allocation. For my first meta-analysis of zinc lozenges, I contacted Kaare Weismann and in the Supplement 2 [7, p.xiii], I documented: The 1990 study report did not describe the method of allocation. Dr. Kaare Weismann described that they had used consecutive allocation (personal communication by email 2 July 2010). When Nault’s inclusion criteria strictly require random allocation, a trial should not be included if the allocation method is unclear. Furthermore, when starting a new project on zinc for the common cold, Nault should have read the previous analyses on the same topic and would have found the above information about Weismann’s allocation. In Analyses 1.1, 3.1, 4.1, 7.1-7.9, 8.2, Nault included the Kurugöl (2006) trial [69]. Kurugöl reported that the duration of colds was 5.3 (SD 0.7) days in the placebo group and 4.7 (SD 0.8) days in the zinc group, table III in [69]. Nault copies these figures to Cochrane review Analysis 3.1. The weight of the Kurugöl (2006) trial is the greatest in Analysis 3.1, 44.7%, so it is not a minor issue if there are flaws in the Kurugöl data. The very narrow SD values are not believable. In our Cochrane review we imputed some missing SD values on the basis of estimating the ratio between SD/mean. We wrote [32, p.8]: Some trials presented the mean duration or severity of colds, but not the respective SD… we estimated SD as identical with the mean of the treatment group. This is based on our analysis that for trials reporting the SD, the ratio of SD to mean is on average 0.7 so that our ratio of 1.0 used in the SD imputation is somewhat conservative. The consequence of this is that we are putting slightly reduced weight in our estimates of effect on these trials with missing SD values, compared to the average. In the Kurugöl (2006) trial, the SD/mean ratios are 0.13 and 0.17 in the placebo and zinc groups, respectively. This ratio is very small compared with the usual distribution of common cold duration, so the “SD” values are doubtful. In comparison, Nault Analysis 3.1 also includes the Kartasurya (2012) trial. In that trial, the SD/mean ratios are 0.67 and 0.64 in the placebo and zinc groups, respectively, which are close 18
to the mean value 0.7 in the large set we analyzed for our Cochrane review on vitamin C and the common cold [32]. Furthermore, Kurugöl’s table III reports the SD for total cold symptoms, which is 0.7 days in the placebo group as mentioned above. The table also reports the SD for cough (SD 2.0 days), nasal drainage (SD 1.8 days), nasal congestion (SD 1.0 days). It is not reasonable to assume that the variation in the component symptoms would sum up to the total cold symptoms with less variance than the variance of the component symptoms. Furthermore, in figure 1, Kurugöl reports total symptom severity scores by day in both the zinc and placebo groups. He calculated that P = 0.000 for day 2, although the SE-bars of the two groups substantially overlap. Thus, Kurugöl’s statistical analysis is questionable. I was able to contact Dr. Kurugöl, who sent an email to me on 2015-1-16. However, when I described the problems in the SD values in Kurugöl’s 2006 paper and asked whether the data set was still available, there was no response. Therefore the reported SD estimates should not be trusted. Evidently, a trial should not be included in a meta-analysis, if it is not clear whether the dispersion is reported as SD or SE; or a conservative SD = mean imputation should be used for common cold duration. Nault included the Kurugöl (2007) trial [70] in the text section [23, p.20-25]. The Kurugöl (2007) report does not refer to the Kurugöl (2006) report; see above. This is very strange since the previous zinc trial by the same authors is obviously relevant for the new trial report. The Kurugöl 2007 report has similar problems to the 2006 report. For example, in figure 1, Kurugol reports total symptom severity scores by day in the zinc and placebo groups. He calculated that P = 0.000 for day 2, although the SE-bars of the two groups substantially overlap. Thus, the statistical analysis is also not trustworthy in the 2007 report. In Analyses 2.1, 4.1, 8.2 Nault included the Vakili (2009) trial [71]. Vakili reported that the average common cold occurrence was 3.1 (SD 0.55) in the placebo group and 1.7 (SD 0.86) in the zinc group, table 2 in [71]. Events such as the occurrence of colds are usually over-dispersed compared with a Poisson distribution. The Poisson distribution has the relation variance = mean, but for natural events the distribution is usually wider so that variance > mean. Therefore, given the mean = 3.1 in the placebo group, the SD should be ≥1.76. Vakili’s SD = 0.55 corresponds to Var = 0.30 which is not believable for a Poisson type outcome with mean 3.1. Furthermore, Vakili states in the Methods that “a total of 200 children were randomly assigned to supplementation with 10 mg elemental zinc as a tablet (n = 100, 50 males, and 50 females), or placebo (n = 100, 50 males, and 50 females).” However, it is extremely unlikely that random allocation of 200 participants leads to exactly the same number of both sexes within both treatment groups. 19
Finally, no drop-outs were reported even though 200 children were followed for 5 months. Evidently, in such a large group some drop-outs are to be expected. For all the above reasons the results of the Vakili 2009 trial should be questioned. In 2018, I tried to contact Dr. Vakili and some of the colleagues whom I was able to identify from PubMed papers on the basis of them being co-authors. I was not successful in contacting Dr. Vakili. When the published SD is statistically impossible, the trial should not be included in a metaanalysis; or a conservative SD should be imputed. In the Methods section Nault writes: “Types of studies: We included randomised controlled trials (RCTs) and excluded studies using quasi-randomisation because they are susceptible to selection bias” [23, p.8]. However, given the identical distribution of 50 patients in each cell of the 2×2 table by treatment and sex, it seems unlikely that the Vakili (2009) trial was randomized. In Analyses 1.1, 3.1, 4.1, 7.1-7.3, 8.1, Nault included the Rerksuppaphol (2013) trial [72]. Rerksuppaphol (2013) randomized 100 children to the zinc and placebo groups using blocks of two. According to their Table 1 [72], the average age was 10.0 yr (SD 0.5 yr) in the zinc group, but 11.4 yr (SD 0.8 yr) in the placebo group. The authors calculated P = 0.0001 for the difference in baseline age between the zinc and placebo groups. Consistently, the height was also highly significantly different in the zinc group (135.5 cm) compared with the placebo group (141 cm), with P = 0.0001. In their paper, Rerksuppaphol et al. do not propose an explanation for the highly significant imbalance in baseline age; they just write that: Despite strict randomization and a double blind design, there was an imbalance in the study between the demographic profiles of the two groups. This was most likely because of statistical probability… This is not a reasonable explanation for a P = 0.0001 difference in a baseline variable, particularly for a highly essential variable such as age. If 20 baseline variables are measured in an RCT, and one or two of them give a P-value somewhat less than 0.05 for the baseline difference in a t-test, that is consistent with successful randomization. However, successful randomization is refuted when P = 0.0001 for a baseline variable as essential in biology as age. As their most interesting finding, Rerksuppaphol reported that coughs and runny noses were shorter in the zinc group, with P = 0.01. The median duration for cough was 1.0 days in the zinc group, and 6.0 days in the placebo group. The total duration of the trial was 3 months and it seems obvious that many participants had more than 1 cough episode per 3 months. However, the Methods and Results sections do not describe whether the duration of cough means duration per episode or duration per person. Thus the outcomes are not well described. Differences in cumulative duration of cough can be 1) due to difference in incidence of cough episodes or 2) due to difference in the duration of cough episodes, or 3) due to a combination of both. 20
Given the reported highly significant baseline imbalance in age and height, and the lack of transparency in the outcomes, it is difficult to trust the reports on coughs and runny noses. In the RoB table, Nault assigns a “+” mark for the “random sequence generation” indicating that Nault is satisfied with the allocation even though there is published evidence by the authors that the groups were highly significantly biased at baseline with P = 0.0001. Nault describes the findings of the Sánchez (2014) trial [73] in the text section [23, p.18]: Sánchez 2014 found incidence rates of acute respiratory infections (ARI) for those receiving chelated zinc (1.42 per 1000 child-days), zinc sulfate (1.57 per 1000 childdays), and placebo (3.3 per 1000 child-days). The Sánchez (2014) trial was not an individual-level RCT. Sánchez writes in the abstract: “Randomized triple-blind community trial with 301 children between 2-5 years of age from six child daycare centers in Medellin, Colombia. Children were distributed in three groups receiving zinc amino acid chelate, zinc sulfate and placebo” [73]. Allocation is described in figure 1 of [73]. Google Translate gives the following version of the allocation process [original in Spanish]: Randomization and masking The six clusters were randomly assigned by means of a ballot so that two children’s centers were in each of the three study groups. One group received zinc sulfate, another group received aminochelated zinc, and the last group received placebo [73, p.81]. Although 6 units can be “randomized” to 3 groups as 2 + 2 + 2, it is not reasonable to consider that such “randomization” leads to meaningful baseline similarity. There is no justification to believe that a very small number of observation units are balanced on baseline variables. The prevalence of viruses varies between geographic regions and between social groups, etc. Any differences between a few child daycare centers can be caused by differences between exposures to viruses or other factors. If the number of units is substantial, then it is reasonable to assume that on average the distributions are similar over the numerous units so that two large groups of units would give similar means if there is no treatment. For example, the Somé (2015) [74] trial included 25 villages, see below. In contrast, the comparison of 2 vs 2 vs 2 childcare centers is not meaningful because of the natural variations in the prevalence of viruses, etc. Under the title “Mean number of colds/URTIs developed”, Nault describes the findings of the Somé (2015) [74] trial in the text [23, p.18]. Nault writes: Somé 2015 reported their longitudinal prevalence for URTI at 7.3 (6.2 to 8.4) and 8.1 (6.9 to 9.3) for the 5 mg and 10 mg zinc groups, respectively, compared with the group who received a placebo tablet’s longitudinal prevalence of 7.5 (6.3 to 8.7). However, “longitudinal prevalence” is not a form of “mean number of colds/URTIs developed”. 21
In their paper, Somé [74] defines: … longitudinal prevalence as the per cent of total days of observation (or ‘recalled’ days) on which the disease was present (ie, the numerator is the total number of days with a disease and the denominator is the total number of days of observation). Differences in “longitudinal prevalence” can be 1) due to difference in incidence of episodes or 2) due to difference in the duration of episodes, or 3) due to a combination of both. In any case, it is not a measure of “Mean number of colds/URTIs developed”. 22
Ignoring a 48-fold variation in zinc dosage in the nasal administration trials In Analysis 9.1, Nault pools 3 nasal zinc administration trials together with 5 zinc lozenge trials. In Analysis 9.2.1, Nault pools 3 nasal zinc trials as a separate subgroup. In the latter metaanalysis, there is extreme heterogeneity between the 3 nasal zinc trials with I2 = 99%. The maximum of the I2-heterogeneity scale is 100% and the minimum is 0%, thus the heterogeneity over the 3 nasal zinc trials is extreme. When there is substantial heterogeneity, the goal should be to search for explanations for the heterogeneity, instead of pooling trials which are very different. One evident possibility to explain the extreme heterogeneity in this case is the dose used in the trials, which is not considered by Nault. Belongia (2001) wrote that the “total maximum daily dose was 0.044 mg elemental zinc” [34, p.104]. Mossad (2003) describes a “total daily dose of elemental zinc of about 2.1 mg” [35, p.37]. Thus, the daily dose of zinc used by Mossad (2003) was 48 (= 2.1/0.044) times greater than the daily dose of zinc used by Belongia (2001). Mossad found a statistically significant reduction in common cold duration of 2.4 days, whereas Belongia found no significant difference between the zinc and placebo groups. An obvious explanation is that the dose used by Belongia was simply so small that no effect is expected. Figure 5 shows the comparison of the 2 trials as a forest plot. Heterogeneity between the 2 trials is very high (P = 0.0022). Figure 5. Comparison of the Belongia (2001) [34] and Mossad (2003) [35] trials in a forest plot on the ratio scale. There is strong evidence that the trials by Belongia (2001) and by Mossad (2003) do not estimate the same treatment effect because the P-value for the test of inconsistency is very small, P = 0.0022. The data are from [23]. ROM, ratio of means [25]. Assuming that Mossad’s 2.4-day (37%) reduction in cold duration is a reasonable estimate for the effect of the 2.1 mg/day nasally administered zinc, and assuming a linear dose-response relationship, we can interpolate that the effect of the 0.044 mg/day zinc in the Belongia study should be that colds are 0.05 days (or 0.8%) shorter in the zinc group (2.4 days/48 or 23
37%/48). Figure 6 shows the relation between dosage and the percentage effect in the two trials. The observed difference between the zinc and placebo groups in the Belongia (2001) trial is consistent with the expected effect. Figure 6. Dosage of zinc and the difference between the zinc and placebo groups in the nasal zinc administration trials by Belongia (2001) [34] and Mossad (2003) [35]. The red diagonal line indicates interpolation of dose-response assuming linearity with the constraint that the zinc dose of 0 mg/day equals placebo with null effect. The point estimate of the Belongia (2001) trial is very close to the expected null effect. The vertical lines indicate the 95% CI range and the point in the middle is the observed effect. The effect estimates are from Figure 5. The third nasal zinc trial in Nault’s Analysis 9.1 was Hirt (2000) [33], who reported the volume of zinc gel to both nostrils per day (0.96 mL), but not the concentration of zinc in the gel. In their introduction they state that they intended to repeat an earlier study that administered gel containing 33 mmol/L of elemental zinc. If that was the concentration of Hirt nasal zinc gel, that would lead to zinc dose of 2.06 mg/day. Hirt found significant benefit from zinc, consistent with the Mossad (2003) trial. The Hirt (2000) trial is not included in Figure 6 since the dose is not reported in the paper [33], though it can be assumed. 24
Naoult wrote: We undertook meta-analyses only where meaningful, that is, if the treatments, participants, and the underlying clinical question(s) were similar enough for pooling to make sense [23, p.10]. When there is a 48-fold difference in the dosage of a drug in 2 trials, it does not make sense to assume that the interventions are “similar enough”. Furthermore, Nault writes in Methods [23, p.11]: We considered the following factors to be potential causes of heterogeneity in the effects of the intervention: … dose of the zinc intervention … However, Nault did not publish any analysis comparable to Figure 6. Nault also writes [23, p.21]: When three studies utilising intranasal zinc ... were compared with five studies utilising zinc lozenges … there was no apparent difference between subgroups (I2 = 0%; Analysis 9.2). Here Nault ignores the lack of statistical power in the comparison of the zinc lozenge and nasal zinc trials. The effect estimate in the “intranasal zinc” group is not accurate enough for meaningful comparisons. The 95% CI for the estimate of effect for the nasal zinc group is from -7.36 days to +1.05 days. This means that if the estimate in the zinc lozenge subgroup is within this range, the difference between the point estimates for nasal zinc and zinc lozenges would not be statistically significant. When the estimate of effect within the nasal zinc group is very extreme with I² = 99% (mostly explained by the 48-fold variation in zinc dose), the mean effect for such a group is not a useful basis for the comparison with other kinds of zinc treatments. 25
Misunderstandings about RCT methods in the Risk of Bias (RoB) assessment This section considers the validity of conclusions in the RoB assessment [23, p.16-17: figure 3]. Allocation concealment Nault put a question mark “?” for “allocation concealment (selection bias)” for the following 7 trials: Caesar (2012), Kartasurya (2012), Mossad (2003), Petrus (1998), Smith (1989), Turner (2000) challenge, Turner (2000) natural. However, the same 7 trials have plus mark “+” (acceptable) on “Blinding of participants and personnel (performance bias): All outcomes” and on “Blinding of outcome assessment (detection bias): All outcomes”. This indicates that there is some confusion about what “allocation concealment” means. In the old literature “double-blind” was used to indicate that both patients and researchers did not know which treatment group the patient was in until the end of the trial. Over time this evolved to describe blinding in more detail. There are usually three aspects: the patient, the person(s) carrying out the intervention, and the person(s) assessing the outcome, and in the ideal case all three parties do not know what the treatment is until the trial ends. That is sometimes described as triple-blind, though mostly the term double-blind is used to indicate complete blinding. “Allocation concealment” is a much more recent concept arising from the fact that in many studies it is not possible to keep all parties blind until the end of the trial. For example, in surgery it is difficult to carry out a trial that is double-blind until the trial ends. In 1983, Thomas Chalmers showed that there were significant differences in findings depending on whether “the randomization process was blinded” [78]. This caused interest in keeping the allocation stage blinded even if the later stages of a trial cannot be blinded, e.g. in surgery. The name of this concept was later changed to “allocation concealment”. If Nault is satisfied that there is “Blinding of participants and personnel” and “Blinding of outcome assessment” until the trial ends, then – logically – there must be blinding at all preceding time points during the trial, including the process of allocation. Thus, all 7 trials should have a “+” for allocation concealment. Random sequence generation For numerous trials, Nault put a question mark “?” for the item “Random sequence generation (selection bias)” even though the reports state that the trials were randomized. These trials have “?” on “randomization” Al-Nakib (1987) “members of each group were randomly allocated to receive zinc gluconate lozenges or placebo” [62] Eby (1984) “was given to each subject, using a double-blind, random method” [53] Farr Trial 2 (1987) “In trial 2, 23 subjects were randomly chosen to receive zinc and 22 were randomly chosen to receive placebo” [54] 32
Petrus (1998) “The subjects of this randomized, double-masked, placebo-controlled study” [37] Prasad (2000) “A research consultant prepared the randomization code and the packages of medication” [39] Prasad (2008) “A research consultant prepared the randomization code and the packages of medication” [40] Smith (1989) “Upon enrollment, subjects were randomly assigned to receive either zinc gluconate or placebo” [61] Turner (2000) natural “were randomized to receive 1 of the 3 treatments” [63] Turner (2001) “Volunteers were randomly assigned to receive either the active preparation or placebo” [75] The trials above with the question mark “?” can be compared with trials that have a “+”. These trials have “+” on “randomization” Belongia (2001) “The active and placebo nasal spray bottles were randomly placed in blocks of four by the manufacturer” [34] Douglas (1987) “either zinc acetate or placebo randomly allocated to the sequence” [60] Farr Trial 1 (1987) “were assigned by prior computer randomization to receive either zinc gluconate or placebo lozenge therapy in trial 1” [54] Godfrey (1992) “Randomization by a third party was used to assign the 87 participants to treatment groups” [36] Kurugöl (2006) “A statistical consultant programmed a computer-generated randomization code and prepared the packages of medication” [69] Kurugöl (2007) “A statistical consultant programmed a computer-generated randomization code and prepared the packages of medication” [70] Malik (2014) “We randomized the treatment allocation by simple randomization using computer generated random numbers (Excel 2010)” [79] Rerksuppaphol (2013) “Using a computerized programme (GraphPad QuickCals), the enrolled children were randomized to the zinc or placebo group using blocks of two by a statistical consultant who was not involved in the implementation phase of the study” [72] Turner (2000) challenge “Subjects were randomized to receive study medication” [63]. There are several inconsistencies in Nault’s assessment of randomization. Nault gave the Farr (1987) Trial2 a “?” but the Trial1 a “+” although the texts in the Farr report are similar for both trials, compare above. It is unlikely that the randomization methods in parallel trials run by the same researchers would differ substantially. Nault gave Turner (2000) natural trial a “?” but the challenge trial a “+” although the texts in the Turner report are also very similar for both trials, compare above. Again, it is unlikely that 33
the randomization methods in parallel trials run by the same researchers would differ substantially. Similarly, why is the “research consultant prepared the randomization code” in the (Prasad 2000, 2008) trials unsatisfactory, whereas “Randomization by a third party” (Godfrey 1992) and “A statistical consultant programmed a computer-generated randomization code” (Kurugöl 2006) and “A statistical consultant programmed a computer-generated randomization code” (Kurugöl 2007) and “randomly allocated to the sequence” (Douglas 1987), etc. are satisfactory? Nault put a “?” for the Petrus (1998) trial. I contacted Petrus and asked about details of their methods and received the following response: The bottles of the zinc lozenges and placebo were sent by the manufacturer and each bottle was identical except a sequential number. At registration, after qualifying for the study each patient was given a bottle of 180 lozenges. At the conclusion of the study, when the diaries were assembled, the code for the bottles was sent by the manufacturer, and the patients were placed in the zinc or placebo category. Then the results were tabulated and the statistical analysis was undertaken (Edward Petrus 24 March 2016). [8,9,10,27] When starting a new analysis on zinc treatment for common cold, Nault should have read the previous meta-analyses on zinc lozenges and would have found that there is additional information about the Petrus (1998) trial. Another questionable classification by Nault was to put a “+” mark for the RoB table for the Rerksuppaphol (2013) trial which reported highly significant baseline difference for age and height (P = 0.0001); see above. The goal of randomization is to minimize selection bias. When such a great imbalance is observed and reported, the bias does not disappear with a “satisfactory” description of randomization methods. Selective reporting Nault put a question mark “?” on nearly all trials [23, figure 3]. The Cochrane Handbook states [80]: This domain addresses bias that arises because the reported result is selected (based on its direction, magnitude or statistical significance) from among multiple intervention effect estimates that were calculated by the trial authors. The item is relevant, for example, in trials on pain, in which case there can be a dozen different measures for pain experience. In fact, pain is used as one example in the Cochrane Handbook [80]: Selective reporting of a particular outcome measurement (based on the results) from among estimates for multiple measurements assessed within an outcome domain. 34
Examples include: … use of multiple measurement instruments (e.g. pain scales) and only reporting data for the instrument with the most favourable result ... There are no similar concerns about the duration of the common cold which cannot be measured in a dozen different ways. It is not clear why Nault has given a “?” to the majority of trials. Potential for bias depends on what was found If a trial finds a positive effect, then it is reasonable to consider whether systematic biases might explain the observed difference, or part of it. However, if there is no difference between trial groups, then the concern of bias is different. If the observed null effect is thought to be a result of bias, then the critic believes that the true unbiased effect is not null. Weismann (1990) found no difference between the zinc lozenge and placebo groups. Nault put the “-” mark on the “Random sequence generation” item of the trial in the risk of bias table, which means “High risk of selection bias”. Thus, this indicates that the null effect is biased in Nault’s view. However, Nault et al. do not discuss whether they believe that the unbiased estimate in Weismann’s trial indicates harm or benefit from zinc lozenges, if they consider that the observed null effect is biased. Smith (1989) found no difference between the zinc lozenge and placebo groups. Nault put a “-” mark on the “Incomplete outcome data” item of the trial in the risk of bias table, which means “High risk of attrition bias”. However, Nault et al. do not discuss whether they believe that the unbiased estimate in Smith’s trial indicates harm or benefit from zinc lozenges, if they consider that the observed null effect is biased. Turner (2001) found no difference between the nasal zinc and placebo groups. Nault put a “?” mark on the “Blinding of participants and personnel” and “Blinding of outcome assessment” items of the trial, which means “Unclear risk of performance and detection bias”. However, Nault et al. do not discuss whether they believe that the unbiased estimate in Turner’s (2001) trial indicates harm or benefit from zinc lozenges, if they consider that the observed null effect is biased. Thus, it is not clear that Nault considered the actual potential for bias in the included trials. Some notes on valid and invalid criticisms Comments on methodological evaluation of RCTs by Meinert (1986) are thoughtful [81]: Valid and invalid criticisms: There is no such thing as a perfect study, only varying degrees of imperfection. The professional critic can always cite one or more of the criticisms listed in Table 26-3 without fear of contradiction. For example, he can always argue that the results of the trial should be ignored because the investigators studied the “wrong” population. Or he can challenge the choice of treatments or the way in which they were administered. And it is always possible to chide investigators because they failed to collect “important” 35
data—at least as viewed from the perspective of the critic. The problem is not coming up with criticisms, but in deciding whether or not they are valid. The trouble with the criticisms listed in Table 26-3 is that they are so broad and sweeping as to be beyond debate. A criticism, to be valid, should: • Have some basis in fact • Be buttressed with supporting evidence • Make a difference in the interpretation of the results [italics added] All three tests should be met. Among the three, the third is the most difficult one to satisfy. For example, it is fairly easy to criticize a trial because of differences in the baseline composition of the treatment groups. However, it is quite another thing to show how those differences might have accounted for the results observed. The variability has to be sizable and must occur in connection with an important predictor of outcome to make any real differences in the results. Table 26-3 Universal criticisms [shortened to a few examples] • Treatment protocol not followed in all cases • Treatments not properly administered • Design of the study flawed (wrong design, inadequate stratification, wrong method of randomization) • Execution of the trial faulty • Results not definitive Nault does not show how the ”bias” may have accounted for the lack of observed benefit in the Weismann, Smith, and Turner trials. 36
Analysis of adverse effects is not appropriate The most common adverse effects of zinc lozenges are bad taste and irritation and dryness in the mouth, and gastrointestinal discomfort. Nault’s conclusion on adverse effects of zinc treatment in the Abstract [23, p.2]: There is probably an increase in the risk of non-serious adverse events when zinc is used for cold treatment (RR 1.34 …); no treatment study provided information on serious adverse events. However, there are flaws in Nault’s analysis of adverse effects. First, the variation in lozenge composition is not taken into account: uniform adverse effects should not be expected. Second, estimating the size of the adverse effect as a RR is not meaningful when the focus is on mild adverse effects. In his reviews, Eby pointed out that the taste problems of zinc lozenges largely depend on the composition of the lozenges: Unlike some unhelpful zinc gluconate lozenges, zinc acetate lozenges are flavour-stable and do not become bitter regardless of time or storage conditions. Properly prepared ZIA 100 zinc acetate lozenges have no objectionable taste or aftertaste in common cold treatment. They do not seem to produce side-effects previously associated with zinc gluconate lozenges, perhaps because much less zinc acetate is required to produce identical results than zinc gluconate or any other suitable zinc compound [2, p.491]. ZG [zinc gluconate] lozenges manufactured without dextrose-based carbohydrates are bland tasting and produce a tannic acid-like mouth feel. However, when carbohydrates (excluding fructose) are used in lozenges, a slow chemical reaction occurs, which over a few weeks to a few months time results in a change in flavor of ZG from bland to noisomely bitter. Consequently, some means of preventing this reaction was needed to produce pleasant tasting lozenges containing ZG. Two to ten moles of glycine relative to ZG prevents the adverse flavor reaction according to US patent 4,684,528 licensed to the Quigley Corporation (Doylestown, PA), the manufacturer of Cold-Eeze brand zinc gluconate–glycine (ZGG) lozenges [3, p.29]. Several trials used other food acids to flavor-mask the bitter ZG/dextrose reaction, resulting in loss of efficacy… [3, p.31]. Although pure ZG is bland and chalky in taste, it reacts with dextrose and related carbohydrates (excluding fructose) upon aging of lozenge compositions to produce noisome bitterness and compliance-related inefficacy. ZG releases large amounts of neutrally charged hydroxide species likely to cross cell membranes and causes oral irritation. Bitterness occurs in all ZG lozenges except those that either do not contain carbohydrates (excluding fructose), or that contain strong extramolar zinc binding agents, which results in something other than ZG. For these reasons, ZG is no longer believed suitable for use in zinc lozenges for treating colds. [3, p.34-35]. On the other hand, ZA [zinc acetate] allows the production of pleasant tasting, flavor stable lozenges releasing large amounts of iZn [free zinc ions] either in hard candies or 37
compressed tablets without flavor or stability issues. The mouth-feel produced is sufficiently like the mouth-feel of tea (slight astringency) to allow using tannic acid without added bitter agents as a placebo in clinical trials [3, p.35]. Fructose is the only carbohydrate sweetener that does not become bitter after aging for several weeks when combined with zinc gluconate [4, p.484]. Due to serious taste issues zinc gluconate was a poor choice for treating colds. Zinc gluconate forms extremely bitter complexes with all sweet carbohydrates except fructose… The overriding source of failure was requirement by pharmaceutical marketing companies for pleasant tasting, candy-like, non-metallic, non-astringent and non-drying zinc lozenges [4, p.488]. Given this impact of zinc lozenge composition on taste, it is evident that the composition should be considered when considering non-severe adverse effects. Furthermore, the pertinent question is the frequency and type of adverse effects from the lozenge compositions that worked (Figures 2 and 3), and not the frequency and type of adverse effects from the zinc lozenges that did not work, such as lozenges containing citric acid, tartaric acid, mannitol/sorbitol, orotate, etc. In my RCT, I also pointed out that the rapid dissolution of lozenges may have exacerbated taste and other adverse effects. If 80 mg/day of zinc is dissolved in mouth over 0.7 hours per day, the temporary zinc ion concentrations in the oropharyngeal region are several times higher compared with the same zinc dose being dissolved over some 3 hours per day [66]. It is also evident that the dosage influences, for example, gastrointestinal discomfort, so the occurrence of adverse effects should be considered as a function of dosage. Finally, regarding minor adverse effects such as taste, it is not reasonable to compare zinc lozenges with placebo lozenges, when the goal in an RCT is to formulate a placebo that tastes the same as the zinc lozenge. If the placebo is perfect, the placebo lozenge and the zinc lozenge cannot be distinguished by taste or other effects unrelated to the common cold. However, this means that both the placebo and the zinc lozenges can taste so terrible that no patient wants to continue the intervention, even though the adverse effects of zinc lozenges would not differ from the placebo lozenges when measured as a RR. Thus, the lack of difference between an ideal placebo and zinc lozenge does not indicate lack of adverse effects of the particular zinc lozenges. Because of these issues, Nault’s calculation of a single estimate for adverse effect such as RR = 1.34 “when zinc is used for cold treatment” [23, p.2] is scientifically unsound. When the adverse effects are minor and short-lasting, such that they disappear with the termination of zinc lozenge administration, the RR is not a reasonable measure for the adverse effects. Nault writes about the analysis of adverse effects of zinc treatments as follows: We pooled 16 treatment studies (Belongia 2001; Caesar 2012; Eby 1984; Eby 2006; Godfrey 1992; Hemilä 2020; Hirt 2000; Macknin 1998; Mossad 1996; Mossad 2003; Petrus 1998; Prasad 2000; Prasad 2008; Turner 2000 challenge; Turner 2000 natural; Weismann 1990). There is probably an increase in the risk of experiencing non-serious 38
adverse events for those taking zinc compared with those taking placebo (RR 1.34, 95% CI 1.15 to 1.55; I2 = 44%; 16 studies, 2084 participants; moderate-certainty evidence; Analysis 11.1). [23, p.22] Belongia (2001) administered 0.044 mg/day zinc. Eby (1984) administered 207 mg/day zinc, which is 4700 times Belongia’s dose. Godfrey (1992) administered 190 mg/day zinc, which is 4300 times Belongia’s dose. There is no basis to consider that trials with over a 4000-fold difference in zinc dosage are “similar enough for pooling to make sense”. Caesar (2012) administered 20 mg/day zinc as powder, while powder is different from nasal zinc and zinc lozenges, and there is no basis to assume the same adverse effects. Furthermore, there are calculation errors in the doses of Mossad (1996) and Prasad (2000) trials, see section “Flaws in data extraction”. In Analysis 11.4. (Adverse event by dose of zinc), Nault classifies these trials in the “high dose” >85 mg/day subgroup, whereas the correct doses are less than 85 mg/day and thus the trials should be in the “low dose” <85 mg/day subgroup. As noted above, there is also no basis to assume that the adverse effects of zinc lozenges (Eby 1984; Godfrey 1992; Hemilä 2020; Macknin 1998; Mossad 1996; Petrus 1998; Prasad 2000; Prasad 2008; Turner 2000; Weismann 1990) and nasal zinc (Belongia 2001; Hirt 2000; Mossad 2003) are similar enough to pool them together. In fact, Nault writes in the Methods section that they did not pool oral and intranasal zinc: Because the likely mechanisms and potential adverse effects of oral and intranasal zinc differ, we analysed trials of oral zinc (including tablets, capsules, syrups, or lozenges) separately from trials of intranasal zinc (including sprays or gels). However, in Nault’s Analysis 11.1 this description of methods was not followed. Given that there are 10 authors in the Cochrane review [23] and all are expected to read and accept the text, it is concerning that not one of them wondered whether the reported methods and the actual methods were consistent. The most relevant measure for assessing mild adverse effects of zinc lozenges seems to be the proportion of patients who do not experience discomforts which are too annoying, and the proportion who do not discontinue the treatment. With this reasoning, I assembled relevant data about the zinc lozenge groups of the more important trials (Table 2). 39
Table 2. Adverse effects of zinc lozenges. Proportion without the adverse effect Trial / Adverse effects Patients with Patients without Godfrey (1992) Zinc gluconate, N = 35 Any adverse effects 20 15 43% Gastrointestinal discomfort 13 22 63% Mouth irritation 12 23 66% Dizziness 3 32 91% Mossad (1996) Zinc gluconate, N = 49 Any adverse effects 44 5 10% Nausea 10 39 80% Constipation 1 48 98% Mouth irritation 12 37 76% Mouth dryness 6 43 88% Bad taste 39 10 20% Prasad (2000) Zinc acetate, N = 25 Nausea 0 25 100% Constipation 6 19 76% Mouth irritation 10 15 60% Mouth dryness 18 7 28% Bad taste 13 12 48% Prasad (2008) Zinc acetate, N = 25 Nausea 3 22 88% Constipation 2 23 92% Mouth irritation 1 24 96% Mouth dryness 13 12 48% Bad taste 15 10 40% Sweet taste 11 14 56% Bitter taste 8 17 68% Sour taste 7 18 72% Hemilä (2020) Zinc acetate, N = 46 Any adverse effects 29 17 37% Stomach ache 12 34 74% Taste problems 24 22 48% The table shows the reported adverse effects in the zinc groups of the trials. 40
Godfrey (1992) [36] reported that 43% of the zinc lozenge participants did not report any adverse effects. In the zinc lozenge group, 1 withdrew for sports injury, 1 for influenza, 1 for bacterial infection, and 1 because of doubting efficacy: these 4 are not withdrawals because of adverse effects. In addition, in the zinc lozenge group, 1 withdrew for nausea and 3 failed to appear at a follow-up. The latter 4 patients may be related to adverse effect of zinc lozenges, which gives 2% (1/43) as the minimum percentage (though nausea may also be caused by the virus), and 9% (4/43) as the maximum percentage who withdrew because of adverse effects. Mossad (1996) [45] reported that 10% of the 49 zinc lozenge patients had no adverse effects. Mossad wrote: “One patient in the zinc group withdrew from the study on the first day because she could not tolerate the lozenges.” However, Mossad continued: “All other patients, as directly observed by the study nurse, indicated that they had good tolerance of the first lozenge.” Furthermore, “two zinc recipients dropped out after 7 to 16 days” but there is no information on the reasons. Thus, 2% withdrew because of not tolerating the zinc lozenge, while 94% continued to the end (not making guesses for the 2 dropouts at the later stage). Prasad (2000) [39] and Prasad (2008) [40] did not publish how many participants reported no adverse effects. In both trials, issues with bad taste were reported by about half the participants. However, none of the participants in the zinc groups (50 = 25 + 25 in the zinc groups of the 2 trials) withdrew from the trial, indicating that the adverse effects were quite mild in the opinion of the patients. In the 2000 trial, “two participants in the placebo group dropped out on day 2” but that is not explained as an adverse effect of the zinc lozenges. Hemilä (2020) [66] found that 37% (95% CI: 23% to 53%) of zinc lozenge participants did not complain of any adverse effects. In summary, there is great variation in whether, and how, different patients experience discomfort from the tested zinc lozenges. While a substantial proportion of patients have not experienced adverse effects, a few have discontinued trials because of them. All the reported adverse effects in the zinc lozenge trials were minor inconveniences. Thus, if a patient starts testing whether zinc lozenges are useful for his or her current common cold, the patient suffering from acute adverse effects such as bad taste can simply stop taking the lozenges. Such an approach would not be different from the ordinary use of over-the-counter medicines. For example, if a patient takes a painkiller for headache, but gets stomach ache as an adverse effect, the patient can discontinue the medication any time. A stomach ache for an individual patient does not prevent other people from using painkillers for their headaches. The proportion of patients who don’t get adverse effects from zinc lozenges (Table 1) appears a much more useful approach to analyze the occurrence of minor adverse effects from zinc lozenges than the “the risk of non-serious adverse events when zinc is used for cold treatment (RR 1.34 …)” reported by Nault [23, p.2]. In the Plain Language Summary, Nault writes [23, p.3]: Studies administering intranasal zinc did not report any cases of anosmia (loss of sense of smell) However, analysis of adverse effects of zinc should not be limited to the small RCTs included in the meta-analysis estimating the treatment efficacy. There are relevant data outside of the trials included by Nault. 41
Misleading Background section In the Background section, Nault writes: Whilst many people believe that zinc may be helpful in preventing or treating colds, there is currently no established intervention to prevent colds or to shorten their duration. This statement ignores our Cochrane review on vitamin C and the common cold [29-32]. We showed that regular vitamin C supplementation of ≥0.2 g/day shortened the duration of colds separately in adults by 7.7% (P = 0.0002) and in children by 14% (P = 0.000 05) [32]. We also calculated that in 5 trials with participants under heavy short-term physical stress, vitamin C reduced the incidence of colds by RR = 0.52 (P = 0.000 001). In our 2023 meta-analysis on the effects of vitamin C on the severity of common cold symptoms, we restricted to 15 comparisons in which ≥1 g/day vitamin C was administered to participants initially in good health, and when a common cold occurred, the severity was decreased on average by 15% (P = 0.000 002) in the vitamin C groups [47]. Thus, there is very strong evidence that in certain contexts vitamin C can prevent colds and reduce their duration and severity. Bias against using vitamin C to prevent and treat the common cold has been demonstrated in several papers [24,108-128]. Nault also ignores 4 placebo-controlled RCTs on nasal iota-carrageenan [129-132] and 2 meta-analyses based on those RCTs [133,134]. In an Individual Patient Data (IPD) metaanalysis of 2 trials in which iota‐carrageenan was administered nasally 3 times per day for 7 days for patients with the common cold [129,130], we calculated that nasal carrageenan increased the recovery rate from all colds by 54% (95% CI: 15-105%; P = 0.003) [133]. The increase in the recovery rate was 139% for coronavirus infections, 119% for influenza A infections, and 70% for rhinovirus infections. In a Quantile Treatment Effect analysis of the nasal carrageenan trials, there was no indication of a meaningful benefit for patients who had colds shorter than one week; however, for patients who had colds lasting two weeks or more, carrageenan shortened colds by up to 5–7 days [14,133]. Another IPD meta-analysis also concluded that there was evidence that carrageenan was effective [134]. Furthermore, in addition to the 2 carrageenan trials for which IPD was available, 2 further RCTs reported that nasal carrageenan was beneficial for common cold patients [131,132]. Thus, independent of the zinc lozenge trials (Figures 2 and 3), there is strong evidence from RCTs that vitamin C and nasal iota-carrageenan can be beneficial for colds. 48
Practical implications and guidance for further research Implications for practice This Cochrane review found varied evidence to support the effectiveness of zinc in the prevention or treatment of the common cold. Zinc supplementation may provide some limited benefits for people with the common cold. On the basis of this review, the current evidence is insufficient to provide firm conclusions or recommend zinc supplementation for the prevention or treatment of the common cold [23, p.28]. “Zinc supplementation may provide some limited benefits for people with the common cold… … zinc supplementation… ” As described above, zinc lozenges and nasal zinc are not forms of dietary supplementation of zinc. As to the RCT results in the trials on zinc lozenges, the statement “may provide some limited benefit” is an understatement, see Figures 2 and 3. The statement “some limited benefits” is obscure. If a reduction in the duration of colds by the Cochrane estimate of 2.37-days [23] is too limited to be of practical relevance, how large an effect would Nault require before viewing the effect as relevant for ordinary common cold patients.? In comparison, painkillers are often used to help common cold symptoms, but there is no evidence that they shorten colds at all. In any case, with a more statistically-robust analysis (Figures 2 and 3), Nault could have concluded that “on the basis of published trials on zinc acetate and zinc gluconate lozenges, they can shorten common colds in adults on average by 37%”. In doing so, the authors would leave it to the reader to decide whether the effect is large enough to justify the reader to test zinc lozenges himself or herself. Although the zinc lozenge trials have implications for practice, the readers should be informed of the problems of the lozenges on the market [4,67], and should be guided to search for formulations that are expected to be effective. See also the section “PICO criteria were not considered” 49
If a researcher has carefully reviewed a topic, usually it is constructive to suggest directions for future research. Nault concludes: Implications for research … Future researchers should note the need for standardised methods when providing zinc supplementation for the prevention or treatment of the common cold. More research is needed to determine the exact zinc type, dose, and duration of supplementation appropriate for the prevention or treatment of the common cold. Future directions for research might be to consider exploring the subjective experience or quality of life of participants while taking zinc, increasing the diversity of characteristics in the participants recruited (varied race/ethnicity, age, gender, health status), and further examining factors that may interfere with the bioavailability of zinc [23, p.29] This instruction is not particularly helpful for further research. “Standardized methods” is obscure without further definition. “… to determine the exact zinc type, dose, and duration of supplementation appropriate for the prevention or treatment of the common cold” What type of zinc salt should have priority? What is a reasonable dose? Ordinary tablets or zinc lozenges or nasal zinc? Given that Nault has read the reports on zinc trials, questions such as these should be commented on. To my knowledge there is no evidence indicating that ordinary zinc tablets that are intended to be swallowed whole have effects on colds in the Western adult population. Nault also does not provide any evidence for benefits of dietary zinc supplementation. Nasal zinc administration should not be a high priority until the concerns about anosmia have been resolved; see above. There is strong evidence that properly composed zinc lozenges can reduce common cold duration as shown in Figures 2 and 3 and in references [1-14] and therefore further research on zinc lozenges should have a high priority. My 2011 and 2017 meta-analyses indicated that there was much stronger evidence for the effects of zinc acetate lozenges when compared with zinc gluconate lozenges [7,8], which is also shown in Figure 3. Therefore, the highest priority should be to further examine zinc acetate lozenges. There does not seem to be any particular benefit from using zinc gluconate lozenges. My 2017 dose-response analysis did not find any difference in the effects of 5 trials using elemental zinc doses from 80 to 92 mg/day compared with 2 trials that used substantially higher doses of zinc, 192 and 207 mg/day [8]. However, the Turner (2000) trial found no benefit from 80 mg/day zinc (instructed dose, but actual dose was not reported) as zinc gluconate lozenges [63], and my own RCT with 65 mg/day zinc (actual used dose) as zinc acetate lozenges did not find any benefit, though my trial demonstrated the rebound effect when the 5-day zinc dosage was terminated [66]. On this basis, it seems obvious that 200 mg/day elemental zinc is not needed, but 80 mg/day may be too low to lead to consistent benefits. Therefore, a dosage of around 120 mg/day might be reasonable in further research when trying to repeat the previous positive findings. Thereafter the lowest effective doses and optimal lozenge compositions could be searched for. 50
A further aspect of the zinc lozenges is the time taken to dissolve in the mouth, which has been considered by Eby [4] and Hemilä [66]. In 2008, Eby wrote about implications for research on zinc lozenges [135]: Future studies should focus on plain zinc acetate lozenges that are chemically identical to the lozenges reported to be effective by Petrus et al. [37] in 1998, by Prasad et al. [39] in 2000, and in the upcoming report by Prasad [40]. Such lozenges have been previously described [5] and have been highly successful in treating colds both in clinical trials and in the general population. Although billions of dollars have been spent on zinc lozenges for treatment of colds, money spent on zinc lozenges containing amino, citric, or ascorbic acids may have been wasted. 51
Flaws in data extraction I did not systematically check data extraction for the Nault meta-analysis, but happened to note some substantial errors. Nault writes about the Turner (2001) [75] trial: “Zinc gluconate 1200 mg/day nasal gel” [23, p.142] However, Turner (2001) wrote: “Study medication… The active preparation (Zicam) consisted of 33 mM zinc gluconate… Study medications were administered as a single nasal spray of 120 μL per nostril… 5 times each day.” 120 μL × 2 nostrils × 5 times each day = 1200 μL. Thus, it seems that Nault confused between mg/day and μL. The actual dose is: 0.033 mol/L × 0.00012 L × 2 nostrils × 5 times each day × 455.7 g/mol = 0.0180 g = 18 mg zinc gluconate per day (2.59 mg/day elemental zinc). Nault’s Cochrane review [23] has 10 authors and it is concerning that not one of them read the manuscript carefully enough to wonder whether it is possible to administer 1.2 grams of zinc gluconate nasally per day. Nault writes about the Hirt (2000) [33] trial: “Zinc gluconate 960 mg/d nasal gel” [23, p.70] but on the same page: “Dose per day (elemental zinc in mg): unclear” [23, p.70] 120 μL × 2 nostrils × 4 times each day = 960 μL. Thus, Nault confused also here between mg/day and μL. Reporting of the zinc dose in Hirt (2000) [33] was not clear, giving justification to the second statement “unclear”. However, in table 2, Characteristics of interventions, Nault gives a third zinc dose in the Hirt (2000) trial: 33 mmol concentration/spray 1 × spray per nostril 4 × per day (8 total doses per day) [leading to] “Approximate zinc dose per day” of 2.1 mg/d [23, p.191]. Thus, Nault gives 3 different zinc doses for the Hirt (2000) trial: 1) 960 mg/d, 2) unclear, and 3) 2.1 mg/d. In the calculation of the 2.1 mg/d, Nault assumed “33 mmol concentration/spray” but Hirt did not report such a zinc concentration of their gel [33]. Nevertheless, the 2.1 mg/day does have indirect justification from the Hirt (2000) report [33, p.778]: Recently, a new approach to zinc therapy – an over-the-counter nasal gel formulation (Zicam) – was the subject of a preliminary study (C.B. Hensley, PhD, and R. Davidson, PhD, unpublished data, 1999). That study demonstrated that the direct application of the nasal gel containing 33 mmol/L of ionic zinc within 24 hours of the onset of common 52
cold symptoms significantly shortened the duration of those symptoms. The researchers hypothesized that the delivery of ionic zinc directly to the site of infection should be more effective than oral delivery. The goal of our study was to attempt to reproduce those results and independently assess the effect of zinc nasal gel on the duration of common cold symptoms. Thus Hirt did not state in their methods that they used 33 mmol/L zinc gel, but the introduction suggests so. The methods section of Hirt describes 120 μL four times per day for both nostrils, which would lead to 2.07 mg of elemental zinc, if we assume that the concentration was 33 mmol/L: 0.033 mol/L × 0.00012 L × 2 nostrils × 4 times each day × 65.4 g/mol = 2.07 mg/day elemental zinc I did not check through all trials included by Nault, but these below are selected further examples of data extraction and calculation errors, in addition to the two trials described above. Hemilä (2020) Nault writes [23, p.67]: “Frequency of dose: 6/d… Dose per day (elemental zinc in mg): 78 mg/d” However, Hemilä (2020) reported: “The zinc lozenge was a commercially available zinc acetate lozenge with 13 mg elemental zinc per lozenge” [66, p.3] “Over the period from the second to the fifth days, the zinc participants used on average 5.05 lozenges per day” [66, p. 4] This gives the correct dose of daily elemental zinc as: 13 mg * 5.05 = 65.6 mg/d Mossad (1996) Nault writes [23, p.98]: “Frequency of dose: 1 lozenge every 2 hours while awake. Dose per day (elemental zinc in mg): 13.3 mg* (max) 12 oz/day = 159.6 mg/d” However, Mossad (1996) reported: “… lozenges that weighed 4.4 g and contained 13.3 mg of zinc.” [45, p.82] “the zinc group took an average of 6 ± 2 lozenges per day (median, 5 lozenges per day)” [45, p. 84] This gives the correct dose of daily elemental zinc as: 13.3 mg * 6 = 80 mg/d Thus, the correct dose is half of that calculated by Nault. This error has further consequences in Nault’s analysis. 53
In Analysis 11.4. (Adverse events), Nault classifies the Mossad (1996) trial in the “high dose” >85 mg/day subgroup, whereas the correct dose is <85 mg/day and thus the trial should be in the “low dose” <85 mg/day subgroup. Prasad (2000) Nault writes [23, p.113]: “Dose per day (elemental zinc in mg): 12.8 mg x 8 = 102.4 mg (estimated number of lozenges per day)” However, Prasad (2000) reported: “Each lozenge contained 12.8 mg of zinc.” [39, p.246] “The average number of lozenges taken daily was 6.2 in the zinc group” [39, p.249] This gives the correct dose of daily elemental zinc as: 12.8 mg * 6.2 = 79 mg/d This error has further consequences in Nault’s analysis. In Analysis 9.3. (Treatment effects), Nault classifies the Prasad (2000) trial in the “high dose” >85 mg/day subgroup, whereas the correct dose is <85 mg/day and thus the trial should be in the “low dose” <85 mg/day subgroup. In Analysis 11.4. (Adverse events), Nault classifies the Prasad (2000) trial in the “high dose” >85 mg/day subgroup, whereas the correct dose is <85 mg/day and thus the trial should be in the “low dose” <85 mg/day subgroup. Prasad (2008) Nault writes [23, p.119]: “Dose per day (elemental zinc in mg): 13.3 mg*8 = 106.4 mg/d” However, Prasad (2008) reported: “The active lozenges contained 13.3 mg of zinc” [40, p.796] “The average number of lozenges taken daily was 6.9 in the zinc group” [40, p.799] This gives the correct dose of daily elemental zinc as: 13.3 mg * 6.9 = 92 mg/d 54
Macknin (1998) trial with children [38] In Analysis 9.1, Nault states that the mean duration of colds was 8.7 days (SD 8.5 days) in the zinc lozenge group, and 8.7 days (SD 2.8 days) in the placebo group; see Figure 1 of this document. However, in the Characteristics of Included Studies, Nault [23, p.85] writes: Notes: duration was only reported as median time + CI, so while CI could be converted to SD, the median could not. ‘The median time to resolution of all cold symptoms was 9.0 days (95% CI, 8-9 days) in the placebo group and 9.0 days (95% CI, 7-10 days) in the zinc group (P=.71; figure 2).’ They also reported the number whose symptoms resolved over the course of the 21 days vs those who did not. That is reported here. There is no description where the SD 8.5 and 2.8 days of Analysis 9.1 come from. The distribution of recovery in the zinc lozenge and placebo groups is closely overlapping and thus there cannot be a 3-fold difference in the SD values between the two trial groups, see Figure 9. Measuring the curves to reach the IPD, imputing 18 days to the censored observations at the right-hand side, gives the mean duration 9.2 days (SD 4.6 days) in the zinc lozenge group, and 9.5 days (SD 4.6 days) in the placebo group. These SD values are quite different from the SD values published by Nault in Analysis 9.1. Figure 9. The recovery curves in the Macknin (1998) trial with children. In Analysis 9.1, Nault used SD = 2.8 days in the placebo group and SD = 8.5 days in the zinc group [23, p.163]; see Figure 1 of this document. However, the distributions are closely similar and thus there cannot be 3-fold difference in the SD in the two groups. Nault does not describe what is the source for the SD values in the Analysis 9.1. 55
Nault also reports the age in the Macknin (1998) trial as follows [23, p.83]: “Age (mean (SD)): 11.67 (7.5)” with the mean and SD identical in zinc and placebo groups. However, Macknin (1998) trial reports as follows [38, p.1964]: “Age, median (IQR), y 13 (6-16) [in Placebo] 13 (6-16) [in Zinc]” Macknin (1998) did not report the mean values and Nault does not describe the source for the medians. Nault’ s table 1: Study characteristics. Descriptions of “setting” are misleading. Nault states that the “setting” of Al-Nakib (1987), Farr (1987), and Turner (2001) was “community”. However, all these trials were about experimental colds, i.e., colds caused in a laboratory with rhinovirus inoculation (see Table 1 of this document). They were not about colds occurring naturally in the community. Because experimental colds and natural colds differ substantially, it is essential to analyze them separately; see above. A reasonable column to describe the type of infection is the “settings”, but it is misleading to describe that experimental colds are community colds. For several trials, Nault writes “Clinic/medical centre” (Belongia 2001; Eby 2006; Godfrey 1992; Kartasurya 2012; Kurugöl 2007; Mossad 2003; Prasad 2008), though the “Clinic/medical centre” was purely a contact place for outpatients with the communityoriginated common colds. In most other trials the setting was “Community” which more accurately describes that the patients were outpatients from the community. In this respect the descriptions in Nault’s table 1 are inconsistent. 56
Errors in other Cochrane reviews Cochrane reviews are often promoted as high-quality systematic reviews that can be trusted: “Cochrane: Trusted evidence. Informed decisions. Better health.” https://www.cochrane.org (Accessed 2024-9-13). The Cochrane reviews by Nault et al. (2024) [23] and by Singh and Das (2011/2013/2015) [15,17,19] are not the only Cochrane reviews that have been shown to be flawed and that have misled readers. Given the promises of Cochrane, it is informative to look at certain other Cochrane reviews as they indicate that there may be wider problems in the procedures to review and update the Cochrane reviews. In 2021, I found flaws in the Cochrane review on Vitamin C and pneumonia (2020) [136], which falls within the responsibility of the Cochrane Acute Respiratory Infections group along with the reviews on zinc and the common cold. For example, the authors of the vitamin C and pneumonia review stated: We included studies involving: 1. healthy adults and children receiving vitamin C supplementation for the prevention of pneumonia. However, two trials (Coulehan et al. and Bancalari et al.) were included in the review despite the fact that they were common cold trials and neither mentioned pneumonia even as a secondary outcome [137]. Furthermore, according to the Cochrane review, the number of pneumonia “events” in the Bancalari trial was 21 in each group. However, that was not the number of infections, but the number of children who had ≥1 infections. There were 46 infection events in the placebo group (N=30) and 38 in the vitamin C group (N=32). However, none of these 84 respiratory infections was pneumonia; they were the common cold [137,138]. In 2009, I found that there were several errors in the Cochrane review Vitamin C supplementation for asthma (2009) [139]. For example, there were errors in data extraction; one trial had 20 participants, but only 11 participants were included in the Cochrane analysis. There were errors in the calculations - the unpaired t-test was used when the paired test should have been used. My feedback was published in the (2010) version of the “2009” review [140,141]. The managing editor of the Cochrane Airways group, Emma Welsh, wrote a response to my criticism in the (2012) version of the “2009” Cochrane review [142]. Simultaneously, the three analysis figures, which were the main focus of my critique, were removed from the review. In their absence, however, my feedback though originally valid became irrelevant. Furthermore, the remaining figures were renumbered so that the figure numbers in my feedback indicated real but different figures; therefore, readers would consider that I was confused. Finally, many responses by Emma Welsh did not seem valid [143]. I contacted David Tovey, Editor-in-Chief of Cochrane, but our correspondence did not lead to any constructive efforts to correct the problem. After a year, I contacted COPE which stated that the original (2009) version [139] must be made available to readers; see a brief summary of the long process [144]. 57
75. Turner RB. Ineffectiveness of intranasal zinc gluconate for prevention of experimental rhinovirus colds. Clin Infect Dis. 2001;33(11):1865-70. https://doi.org/10.1086/324347 76. Rao SS, Hendley JO, Hayden FG, Gwaltney JM. Symptom expression in natural and experimental rhinovirus colds. Am J Rhinol. 1995;9(1):49-52. https://doi.org/10.2500/105065895781874114 77. Turner RB, Witek TJ, Riker DK. Comparison of symptom severity in natural and experimentally induced colds. Am J Rhinol. 1996;10(3):167-72. https://doi.org/10.2500/105065896781794888 78. Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309(22):1358-61. https://doi.org/10.1056/nejm198312013092204 https://www.researchgate.net/publication/16561928 79. Malik A, Taneja DK, Devasenapathy N, Rajeshwari K. Zinc supplementation for prevention of acute respiratory infections in infants: a randomized controlled trial. Indian Pediatr. 2014;51(10):780-4. https://doi.org/10.1007/s13312-014-0503-z 80. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated Aug 2023). Cochrane, 2023. https://training.cochrane.org/handbook/current/chapter-08#section-8-7 https://training.cochrane.org/handbook/current/chapter-02 (Accessed 2024-9-13) 81. Meinert CL. Clinical Trials: Design, Conduct, and Analysis. Oxford, 1986, pp.276-277. https://doi.org/10.1093/acprof:oso/9780195035681.003.0026 82. Mathis S, Le Masson G, Soulages A, Duval F, Carla L, Vallat JM, Solé G. Olfaction and anosmia: From ancient times to COVID-19. J Neurol Sci. 2021;425:117433. https://doi.org/10.1016/j.jns.2021.117433 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9755649 83. Suzuki M, Saito K, Min WP, Vladau C, Toida K, Itoh H, Murakami S. Identification of viruses in patients with postviral olfactory dysfunction. Laryngoscope. 2007;117(2):272-7. https://doi.org/10.1097/01.mlg.0000249922.37381.1e https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7165544 84. Tian J, Pinto JM, Li L, Zhang S, Sun Z, Wei Y. Identification of viruses in patients with postviral olfactory dysfunction by multiplex reverse-transcription polymerase chain reaction. Laryngoscope. 2021;131(1):158-64. https://doi.org/10.1002/lary.28997 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7436707 85. Akerlund A, Bende M, Murphy C. Olfactory threshold and nasal mucosal changes in experimentally induced common cold. Acta Otolaryngol. 1995;115(1):88-92. https://doi.org/10.3109/00016489509133353 86. Sudre CH, Keshet A, Graham MS, et al. Anosmia, ageusia, and other COVID-19-like symptoms in association with a positive SARS-CoV-2 test, across six national digital surveillance platforms: an observational study. Lancet Digit Health. 2021;3(9):e577-e586. https://doi.org/10.1016/s2589-7500(21)00115-1 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8297994 87. Doty RL. The mechanisms of smell loss after SARS-CoV-2 infection. Lancet Neurol. 2021;20(9):693-5. https://doi.org/10.1016/s1474-4422(21)00202-7 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8324112 88. Rothman KJ, Greenland S. Causation and causal inference in epidemiology. Am J Public Health. 2005;95 Suppl 1:S144-50. https://doi.org/10.2105/ajph.2004.059204 89. Rothman KJ. The wrong message from the wrong talk. Observational Studies. 2020;6(2),30-2. https://doi.org/10.1353/obs.2020.0006 https://muse.jhu.edu/pub/56/article/793347/summary 90. Pories WJ, Henzel JH, Rob CG, Strain WH. Acceleration of healing with zinc sulfate. Ann Surg. 1967;165:432-6. https://doi.org/10.1097/00000658-196703000-00015 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc1617499 91. Barcia PJ. Lack of acceleration of healing with zinc sulfate. Ann Surg. 1970;172:1048-50. https://doi.org/10.1097/00000658-197012000-00019 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc1397149 64
92. Serjeant GR, Galloway R, Gueri M. Oral zinc sulphate in sickle-cell ulcers. Lancet. 1970;2:891-2. https://doi.org/10.1016/s0140-6736(70)92067-2 93. Greaves MW, Skillen AW. Effects of long-continued ingestion of zinc sulphate in patients with venous leg ulceration. Lancet. 1970;2:889-91. https://doi.org/10.1016/s0140-6736(70)92066-0 94. Hallböök T, Laner E. Serum-zinc and healing of venous leg ulcers. Lancet. 1972;2:780-2. https://doi.org/10.1016/s0140-6736(72)92143-5 95. Czerwinski AW, Clark ML, Serafetinides EA, Perrier C, Huber W. Safety and efficacy of zinc sulfate in geriatric patients. Clin Pharmacol Ther. 1974;15(4):436-41. https://doi.org/10.1002/cpt1974154436 96. Simkin PA. Oral zinc sulphate in rheumatoid arthritis. Lancet. 1976;2:539–42. https://doi.org/10.1016/s0140-6736(76)91793-1 97. Lyckholm L, Heddinger SP, Parker G, et al. A randomized, placebo controlled trial of oral zinc for chemotherapy-related taste and smell disorders. J Pain Palliat Care Pharmacother. 2012;26:111–4. https://doi.org/10.3109/15360288.2012.676618 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc4042409 98. Bamford JT, Gessert CE, Haller IV, Kruger K, Johnson BP. Randomized, double-blind trial of 220 mg zinc sulfate twice daily in the treatment of rosacea. Int J Dermatol. 2012;51:459–62. https://doi.org/10.1111/j.1365-4632.2011.05353.x 99. Hadwan MH, Almashhedy LA, Alsalman AR. Oral zinc supplementation restores high molecular weight seminal zinc binding protein to normal value in Iraqi infertile men. BMC Urol. 2012;12:32. https://doi.org/10.1186/1471-2490-12-32 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc3503568 100. Ala A, Walker AP, Ashkan K, Dooley JS, Schilsky ML. Wilson's disease. Lancet. 2007;369(9559):397-408. https://doi.org/10.1016/s0140-6736(07)60196-2 101. Członkowska A, Litwin T, Dusek P, Ferenci P, Lutsenko S, Medici V, Rybakowski JK, Weiss KH, Schilsky ML. Wilson disease. Nat Rev Dis Primers. 2018;4(1):21. https://doi.org/10.1038/s41572-018-0018-3 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc6416051 102. Brewer GJ, Askari FK. Wilson’s disease: clinical management and therapy. J Hepatol. 2005;42 (Suppl 1):S13-S21. https://doi.org/10.1016/j.jhep.2004.11.013 103. Hoogenraad TU. Zinc treatment of Wilson’s disease. J Lab Clin Med. 1998;132:240-1. https://doi.org/10.1016/s0022-2143(98)90034-8 104. Marcellini M, Di Ciommo V, Callea F, Devito R, Comparcola D, Sartorelli MR, Carelli G, Nobili V. Treatment of Wilson's disease with zinc from the time of diagnosis in pediatric patients: a single-hospital, 10-year follow-up study. J Lab Clin Med. 2005;145(3):139-43. https://doi.org/10.1016/j.lab.2005.01.007 105. Cochrane Library. What is PICO. https://www.cochranelibrary.com/about-pico (Accessed 2024-9-13) 106. Hemilä H, Chalker E. Abrupt termination of vitamin C from ICU patients may increase mortality: secondary analysis of the LOVIT trial. Eur J Clin Nutr. 2023;77(4):490-4. https://doi.org/10.1038/s41430-022-01254-8 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc10115628 107. Hemilä H, Chalker E. Rebound effect explains the divergence in survival after 5 days in a controlled trial on vitamin C for COVID-19 patients. Front. Med. 2024;11:1391346. https://doi.org/10.3389/fmed.2024.1391346 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc11151746 108. Hemilä H. Vitamin C supplementation and common cold symptoms: problems with inaccurate reviews. Nutrition 1996;12:804–9. https://doi.org/10.1016/s0899-9007(96)00223-7 https://hdl.handle.net/10138/225877 109. Hemilä H. Vitamin C, the placebo effect, and the common cold: a case study of how preconceptions influence the analysis of results. J Clin Epidemiol. 1996;49(10):1079-84. https://doi.org/10.1016/0895-4356(96)00189-8 https://hdl.handle.net/10138/225872 110. Hemilä H. To the dissent by Thomas Chalmers. J Clin Epidemiol. 1996;49(10):1087. https://doi.org/10.1016/0895-4356(96)00191-6 https://hdl.handle.net/10138/225873 65
111. Hemilä H. Do Vitamins C and E Affect Respiratory Infections? Ph.D. Thesis. University of Helsinki, Helsinki, Finland, 2006. https://hdl.handle.net/10138/20335 https://zenodo.org/doi/10.5281/zenodo.6395595 112. Hemilä H. The role of vitamin C in the treatment of the common cold. Am Fam Physician. 2007;76(8):1111, 1115. https://www.aafp.org/pubs/afp/issues/2007/1015/p1111a.html https://pubmed.ncbi.nlm.nih.gov/17992770 113. Hemilä H. Vitamin C for the common cold should not be rejected on the basis of old and erroneous articles. J Allergy Clin Immunol. 2009;124(4):859. https://doi.org/10.1016/j.jaci.2009.06.015 114. Hemilä H. Randomised trials on vitamin C. Br J Nutr. 2011;105(3):485-7. https://doi.org/10.1017/s000711451000351x 115. Hemilä H. The effect of vitamin C on the common cold. J Pharm Pract. 2011;24(2):241-2. https://doi.org/10.1177/0897190010392376 116. Louhiala P, Hemilä H. Can CAM treatments be evidence-based? Focus Altern. Complement Ther. 2014;19:84-9. https://doi.org/10.1111/fct.12110 https://hdl.handle.net/10138/228056 117. Hemilä H. Thomas Chalmers, vitamin C and the common cold. J R Soc Med. 2016;109(2):46. https://doi.org/10.1177/0141076815606279 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4793763 118. Hemilä H. Vitamin C in clinical therapeutics. Clin Ther. 2017;39(10):2110-2. https://doi.org/10.1016/j.clinthera.2017.08.005 https://hdl.handle.net/10138/228957 119. Hemilä H, Carr A, Chalker E. Vitamin C may increase the recovery rate of outpatient cases of SARS-CoV-2 infection by 70%: reanalysis of the COVID A to Z randomized clinical trial. Front Immunol. 2021;12:674681. https://doi.org/10.3389/fimmu.2021.674681 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8141621 120. Hemilä H, Chalker E. Vitamin C and zinc lozenges for COVID-19? J Am Pharm Assoc. 2021;61(5):e39. https://doi.org/10.1016/j.japh.2021.05.018 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8163692 121. Hemilä H, Chalker E. Commentary: The long history of vitamin C: from prevention of the common cold to potential aid in the treatment of COVID-19. Front Immunol. 2021;12:659001. https://doi.org/10.3389/fimmu.2021.659001 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8047412 122. Hemilä H, Chalker E. Bias against vitamin C in mainstream medicine: examples from trials of vitamin C for infections. Life (Basel) 2022;12(1):62. https://doi.org/10.3390/life12010062 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc8779885 123. Hemilä H, Chalker E. Assessment of vitamin C effects on pneumonia and COVID-19 using Mendelian randomization: analysis may be misleading. Eur J Clin Nutr 2022;76(9):1347-8. https://doi.org/10.1038/s41430-022-01091-9 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8853427 124. Pauling L. Ascorbic acid and the common cold. Am J Clin Nutr. 1971;24(11):1294-9. https://doi.org/10.1093/ajcn/24.11.1294 125. Pauling L. Ascorbic acid and the common cold: Evaluation of its efficacy and toxicity. Part I. Medical Tribune. 1976;17:18-9. https://zenodo.org/doi/10.5281/zenodo.11546757 126. Pauling L. Ascorbic acid and the common cold. Part II. Medical Tribune. 1976;17:37-8. https://zenodo.org/doi/10.5281/zenodo.11547088 127. Pauling L. Medical establishment and vitamin C. In: Vitamin C, the Common Cold, and the Flu. Freeman; San Francisco, CA, USA, 1976, pp.121–138. 128. Pauling L. Organized medicine and the vitamins. In: How to Live Longer and Feel Better. Freeman; San Francisco, CA, USA, 1986, pp.225–236. 66
129. Fazekas T, Eickhoff P, Pruckner N, Vollnhofer G, Fischmeister G, Diakos C, Rauch M, Verdianz M, Zoubek A, Gadner H, Lion T. Lessons learned from a double-blind randomised placebo-controlled study with a iotacarrageenan nasal spray as medical device in children with acute symptoms of common cold. BMC Complement Altern Med. 2012;12:147. https://doi.org/10.1186/1472-6882-12-147 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc3575307 130. Ludwig M, Enzenhofer E, Schneider S, Rauch M, Bodenteich A, Neumann K, Prieschl-Grassauer E, Grassauer A, Lion T, Mueller CA. Efficacy of a carrageenan nasal spray in patients with common cold: a randomized controlled trial. Respir Res. 2013;14(1):124. https://doi.org/10.1186/1465-9921-14-124 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc3840586 131. Eccles R, Meier C, Jawad M, Weinmüllner R, Grassauer A, Prieschl-Grassauer E. Efficacy and safety of an antiviral iota-carrageenan nasal spray: a randomized, double-blind, placebo-controlled exploratory study in volunteers with early symptoms of the common cold. Respir Res. 2010;11(1):108. https://doi.org/10.1186/1465-9921-11-108 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2923116 132. Eccles R, Winther B, Johnston SL, Robinson P, Trampisch M, Koelsch S. Efficacy and safety of iotacarrageenan nasal spray versus placebo in early treatment of the common cold in adults: the ICICC trial. Respir Res. 2015;16:121. https://doi.org/10.1186/s12931-015-0281-8 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4595062 133. Hemilä H, Chalker E. Carrageenan nasal spray may double the rate of recovery from coronavirus and influenza virus infections: Re-analysis of randomized trial data. Pharmacol Res Perspect. 2021;9(4):e00810. https://doi.org/10.1002/prp2.810 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8204093 134. Koenighofer M, Lion T, Bodenteich A, Prieschl-Grassauer E, Grassauer A, Unger H, Mueller CA, Fazekas T. Carrageenan nasal spray in virus confirmed common cold: individual patient data analysis of two randomized controlled trials. Multidiscip Respir Med. 2014;9(1):57. https://doi.org/10.1186/2049-6958-9-57 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4236476 135. Eby GA. Therapeutic effectiveness of ionic zinc for common colds. Clin Infect Dis. 2008;46(3):483-4. https://doi.org/10.1086/527479 136. Padhani ZA, Moazzam Z, Ashraf A, Bilal H, Salam RA, Das JK, Bhutta ZA. Vitamin C supplementation for prevention and treatment of pneumonia. Cochrane Database Syst Rev. 2020;4(4):CD013134. https://doi.org/10.1002/14651858.cd013134.pub2 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7192369 137. Hemilä H, Chalker E. Commentary: Vitamin C supplementation for prevention and treatment of pneumonia. Front Med (Lausanne). 2021;7:595988. https://doi.org/10.3389/fmed.2020.595988 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc7854566 Supplement with the details of the problems: https://www.frontiersin.org/articles/10.3389/fmed.2020.595988/full#supplementary-material 138. Hemilä H, Chalker E. Vitamin C supplementation for prevention and treatment of pneumonia (2020) [Shortcomings in a Cochrane review]. Zenodo https://zenodo.org/doi/10.5281/zenodo.11621606 139. Kaur B, Rowe BH, Arnold E. Vitamin C supplementation for asthma. [2009 version; all figures present, no feedback by Hemilä]. Cochrane Database Syst Rev. 2009;CD000993. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6176494 140. Kaur B, Rowe BH, Arnold E. Vitamin C supplementation for asthma. [2010 version; both the feedback by Hemilä and all the original figures; the feedback is on p.27]. Cochrane Database Syst Rev. 2009;CD000993. https://hdl.handle.net/10138/296440 https://zenodo.org/doi/10.5281/zenodo.10853677 141. Hemilä H. Vitamin C supplementation for asthma [Feedback separately] version 2010. University of Helsinki Open Repository, 2010. https://hdl.handle.net/10138/296440 https://zenodo.org/doi/10.5281/zenodo.6405775 67
142. Kaur B, Rowe BH, Stovold E. Vitamin C supplementation for asthma. [2012 version; feedback by Hemilä present, 3 figures removed]. Cochrane Database Syst Rev. 2009;CD000993. https://doi.org/10.1002/14651858.CD000993.pub3 https://zenodo.org/doi/10.5281/zenodo.10959542 143. Hemilä H. Vitamin C and exercise-induced bronchoconstriction: further problems in the Cochrane review “vitamin C for asthma”. University of Helsinki Open Repository, 2013. https://hdl.handle.net/10138/40816 https://zenodo.org/doi/10.5281/zenodo.6405803 144. Hemilä H. Cochrane has not consistently followed the COPE guidelines. Eur J Clin Invest. 2020;50(4):e13216. https://doi.org/10.1111/eci.13216 https://hdl.handle.net/10138/326823 145. Kaur B, Rowe BH, Stovold E. WITHDRAWN: Vitamin C supplementation for asthma. Cochrane Database Syst Rev. 2013 Oct 23;2013(10):CD000993. https://pubmed.ncbi.nlm.nih.gov/24155047 https://doi.org/10.1002/14651858.CD000993.pub4 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6176495 146. Bonini M, Di Mambro C, Calderon MA, Compalati E, Schünemann H, Durham S, Canonica GW. Beta₂-agonists for exercise-induced asthma. Cochrane Database Syst Rev. 2013;(10):CD003564. https://doi.org/10.1002/14651858.cd003564.pub3 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11348701 147. Hemilä H, Friedrich JO. Many continuous variables should be analyzed using the relative scale: a case study of β2-agonists for preventing exercise-induced bronchoconstriction. Syst Rev. 2019;8(1):282. https://doi.org/10.1186/s13643-019-1183-5 Additional file 1 describes the errors in the Bonini (2013) Cochrane review, pp. 12-15: https://static-content.springer.com/esm/art%3A10.1186%2Fs13643-019-1183-5/MediaObjects/ 13643_2019_1183_MOESM1_ESM.pdf 148. Hemilä H. Beta2-agonists for exercise-induced bronchoconstriction: data extraction inconsistencies and errors in the Cochrane review by Bonini et al. (2013). Zenodo 2024. https://zenodo.org/doi/10.5281/zenodo.10959750 149. Ioannidis JPA. Cochrane crisis: secrecy, intolerance and evidence-based values. Eur J Clin Invest. 2019;49(3):e13058. https://doi.org/10.1111/eci.13058 https://pubmed.ncbi.nlm.nih.gov/30520025 150. Newman M. Has Cochrane lost its way? BMJ. 2019;364:k5302. https://doi.org/10.1136/bmj.k5302 Clarification: https://doi.org/10.1136/bmj.l670 151. Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evid Based Med. 2018;23(5):165‐8. https://doi.org/10.1136/bmjebm-2018-111012 152. Gøtzsche PC. What is the moral collapse in the Cochrane Collaboration about? Indian J Med Ethics. 2019;4 (NS)(4):303-309. https://doi.org/10.20529/ijme.2019.064 https://pubmed.ncbi.nlm.nih.gov/31791931 153. Vesper I. Mass resignation guts board of prestigious Cochrane Collaboration. Nature 2018-9-17. https://www.nature.com/articles/d41586-018-06727-0 (Accessed 2024-9-13) 154. Nass M, Noble JH Jr. Whither Cochrane? Indian J Med Ethics. 2018;4(1):1-5. https://doi.org/10.20529/ijme.2018.092 https://pubmed.ncbi.nlm.nih.gov/30683645 68