It is a mistake to abandon evidence-based medicine during COVID-19

Anthony Hatswell is a statistician and health economist. He specialises in the analysis of interventions without randomised clinical trials.

In the current COVID-19 pandemic, many (including world ‘leaders’) have been tempted to abandon the ideas of evidence based medicine of needing clinical trials to justify the use of treatments. This is a mistake.

The example of parachutes has been evoked to justify this position — parachutes are the classic example of an intervention which has no randomised evidence but instead an ‘obvious’ (mechanical) explanation for how they clearly work.¹ Is this realistic in the context of COVID-19? No, is the simple answer. The rate of survival from COVID-19 varies with age, sex, weight, and many more factors.² Given case mix differences between studies, controlled trials are required due to the difficulty in interpreting uncontrolled data. Nothing shows this better than the trials we have of promising treatments which have shown lukewarm findings in the presence of a control arm.³

COVID-19 also offers a perfect example of where findings can be ‘obvious’; there have been no RCTs of ventilators, and no demand for such a trial. These have clear biological plausibility, and an undisputed immediacy of effect. To date, no proposed pharmaceutical offers such a step change. Even worse, if we use only uncontrolled studies, we need see a dramatic change in outcomes for any effect to be apparent –— resulting in therapies with modest effects being discarded.⁴ This is what we have observed with remdesivir — had uncontrolled studies been used the improvement seen of a fall in mortality from 11.9% to 7.1% would have been missed.⁵ This level of efficacy may not be overwhelming but it now represents the only proven treatment for COVID-19.*

Given these positive findings however, there is now a shortage of remdesivir and inadequate supply for all patients. This again is not a new problem — following World War II, Britain was still reeling from the impact of supply shortages and rationing. Streptomycin showed real promise in the treatment of tuberculosis. Austin Bradford-Hill suggested that given insufficient medicine to treat all patients, randomisation would be the only ethical approach — also in allowing us to know if the treatment worked. The answer, in 1948, was the first RCT, run by the Medical Research Council. After six months the death rate was 15/52 (29%) for bed rest (representing standard care) versus 4/55 (7%) for streptomycin giving a resounding answer.⁶ The situation we face as a society with COVID-19 is no different — constrained resources, and a need to establish how best to treat an infectious disease. The answer remains the same.

COVID-19 is a challenge facing society with a large susceptible population worldwide — even a study with sufficient power to comprehensively estimate the effect size of a treatment will comprise only a small percentage of the potential patient population. This concept was highlighted by Anscombe in 1963,⁷ who argued that the optimum should be for society to have the fewest patients possible treated with an inferior treatment; randomising as necessary to make sure we get the right answer to know which treatment that is. From this school of thought we have trials even in the rarest of diseases. The arguments are even more compelling for COVID-19 where findings can be leveraged worldwide to minimise harm.

The idea of randomising patients to standard of care rather than a positive (if unproven) action is difficult however. Such resistance was seen in HIV in trials of delayed vs immediate treatment. This strays in to the territory of clinical equipoise, defined as the lack of consensus on the appropriate course of action which may occur in spite of the strong views of some practioners.^8,9 The best way to resolve equipoise is to run well designed studies, of interventions versus the agreed upon standard of care.¹⁰ This can eliminate any debate about the most appropriate course of action — evidence, not beliefs has the power to change practice in the presence of uncertainty. Due to the results of the HIV trials, early treatment is now the standard and has prevented many lives from being lost prematurely due to unjustified variation in practice.

COVID-19 may be the largest challenge for medicine since HIV — and certainly the largest facing this generation of practitioners. However to forget the hard-won lessons of evidence based medicine would do a disservice to patients: past, present, and future. We can and should act, but in many cases that action should be to use tools and processes to gather data. This should mostly be in the form of controlled clinical trials; the knowledge from which could be transformative.

 

*Editor’s note: The Oxford dexamethasone study was announced in the media after this article was submitted.

References

1. Smith GCS, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ 2003;327(7429):1459–61.
2. ICNARC. ICNARC report on COVID-19 in critical care. 2020; Available from: https://www.icnarc.org/DataServices/Attachments/Download/f48efee2-d38b-ea11-9125-00505601089b
3. Cao B, Wang Y, Wen D, et al. A Trial of Lopinavir–Ritonavir in Adults Hospitalized with Severe Covid-19. New England Journal of Medicine 2020;0(0)
4. Byar DP, Schoenfeld DA, Green SB, et al. Design Considerations for AIDS Trials. New England Journal of Medicine 1990;323(19):1343–8.
5. Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the Treatment of Covid-19 — Preliminary Report. New England Journal of Medicine 2020;0(0)
6. Crofton J. The MRC randomized trial of streptomycin and its legacy: a view from the clinical front line. Journal of the Royal Society of Medicine 2006;99(10):531–534.
7. Anscombe FJ. Sequential Medical Trials. Journal of the American Statistical Association 1963;58(302):365–83.
8. Miller FG, Joffe S. Equipoise and the Dilemma of Randomized Clinical Trials. New England Journal of Medicine 2011;364(5):476–80.
9. Anderson DR. Umbrellas and lions. J Clin Epidemiol 1991;44(3):335–7.
10. Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987;317(3):141–5.

 

Featured photo by Volodymyr Hryshchenko on Unsplash