This was an observational study of 2,541 patients... Carefully read the strengths and weakness part of the paper...
The strengths of this retrospective cohort study were the inclusion of consecutive patients with laboratory-confirmed COVID-19 from a large healthcare system that included a representative population of socially vulnerable, ethnically diverse individuals. Statistical methods included efforts to adjust for possible confounders through multivariable Cox regression and via propensity score matching. However, the limitations of are important to consider. First, the precision of the results is impacted by immortal time bias, since several time-dependent covariates were not modelled in this manner. Fortunately, since the average time to receipt of treatment was only 1 day, this bias may be small; nonetheless, it favors treatment and should be taken into consideration. Second, there is an important potential for residual confounding because there are a number of prognostic factors (e.g. frailty, residence in long term care, or “do not resuscitate” orders), potentially important markers of disease severity (e.g. ferritin, C-reactive protein (Zeng et al., 2020), troponins (Vrsalovic and Vrsalovic Presecki, 2020), and D-dimer (Zhang et al., 2020), and co-administration of potentially beneficial therapies (e.g. anticoagulants (Paranjpe et al., 2020) that were not included in the analysis. Third, confounding by severity or indication (Kyriacou and Lewis, 2016) is likely. While there was a hospital treatment protocol in place, unmeasured clinical factors likely influenced the decision not to treat 16.1% of patients, in a center where 78% received treatment. These factors are often difficult to capture in an observational study. Were the decision to withhold treatment related to poor prognosis (e.g. palliative intent), it stands to reason that patients receiving neither hydroxychloroquine nor azithromycin would have the highest mortality. Indeed, the non-treated group had an overall mortality that was higher than the rate of admission to the ICU (26.4% vs. 15.2%), suggesting that many patients were not considered appropriate for critical care. Such being the case, their care may have differed in other substantive ways that was also associated with death (e.g. terminal illness or advanced directives limiting invasive care). In the hydroxychloroquine treatment groups, the inverse was true with mortality lower than the rate of admission to the ICU (16.1% vs. 26.9%). While a propensity score analysis might further account for some differences between treatment groups, this approach is still limited to the information available in the dataset. Fourth, the chronological time point during the course of the pandemic whereby patients were managed was not included in the study. As the Henry Ford Health System became more experienced in treating patients with COVID-19, survival may have improved, regardless of the use of specific therapies. Hospital-specific guidelines regarding COVID-19 screening eligibility, as well as the availability of COVID-19 testing may have also changed over time, introducing additional chronological bias. Finally, concomitant steroid use in patients receiving hydroxychloroquine was more than double the non-treated group. This is relevant considering the recent RECOVERY trial that showed a mortality benefit with dexamethasone (Horby et al., 2020) among individuals requiring supplemental oxygen or mechanical ventilation, potentially biasing this study’s results in favor of hydroxychloroquine.
