What we Choose to Believe
/Did lockdowns work against COVID? A week ago that question would never have crossed my mind. Of course “lockdowns,” such as they were, helped slow the spread of the virus. Of course anything we did to slow the spread “worked,” right? I mean, I’m here and I’m safe and I never had COVID, nor did my kids or their spouses. I stayed safe at home, wore a mask, did all the right things… why would I even question whether that was smart?
But then I heard someone casually toss out this comment: “They’ve found that lockdowns didn’t work, you know.” I bit my tongue, but I wondered:
· Who are “they”?
· And what “lockdowns” did they evaluate, using what analytics to draw their conclusions?
· And where did this guy find his information?
So I did what I always do in such situations: I did some research and, while I can’t offer one definitive “official” study, I can tell you that I uncovered some information you might find interesting if for no other reason than the opportunity to look at things from a new perspective.
The first challenge is in defining a “lockdown.” Were you locked in? I wasn’t. No one ever told me I may not leave my home. In fact, I’m pretty sure there were no actual lockdowns in the U.S. However, in light of CDC guidelines, some of us followed the “safer at home” or “shelter in place” advice. But there were no locks involved, except that some businesses did have to lock their doors. So, the first problem is that the U.S. really didn’t have any true “lockdowns.” The second problem is that, based on my research, “lockdown” doesn’t take into consideration the voluntary stay-at-home non-socialization many of us chose to observe and the impact it might have had on the virus. And the third problem is that each state - and, often, each city - made its own rules and created its own version of “lockdown.”
In the absence of actual, enforced, legal mandates to stay in one’s home, how might a study define “lockdown”? Would you believe that some studies call “anything other than an NPI” a lockdown? Know what NPI means? (I didn’t.) It’s a “non-pharmaceutical intervention.” So, that means most studies call anything other than a vaccination “a lockdown”: mask mandates, closing of bars and restaurants, social distancing, work-from-home policies, reduced-attendance events… Any initiative to keep people safe OTHER THAN vaccinations are considered a “lockdown” in most studies.
So, that’s pretty broad and fuzzy, but now consider this issue: There were no control groups. A careful study of the efficacy of one approach is to compare it to another group that was identical except for one crucial difference: They were not locked down. In the case of COVID in the U.S., there were no control groups. Of course! We were ALL scrambling for safety. Nobody was encouraged to go out and mingle mightily and exchange all germs possible.
One source explained: “Properly evaluating the effectiveness of any health-oriented treatment, be it a new medicine, vaccine or lockdown measure, involves comparing its introduction with a counterfactual situation where everything is the same except for what’s being tested. This is why medical trials have a control group, whose members are given a placebo and whose characteristics match the testing group as closely as possible. Unfortunately, this isn’t possible for lockdowns. There are no directly equivalent parts of a country or the world that can act as true test and control groups, and so no possibility of a controlled trial. Instead, researchers must resort to other methods to try to measure lockdowns’ effects.”
Mathematical models can be used - and are often effective - but “Their outputs depend on making assumptions and estimates, including about human behaviour, which is notoriously difficult to capture accurately.” Yet this study goes on to say: “While accepting that no method of assessment is perfect, most published studies* have found evidence to support lockdown measures being effective. They also point out that different actions achieved different things in different countries.” [*Did you click on that link? It lists 78 research articles, 63 of which find NPIs generally effective and 6 of which find them generally ineffective.]
One researcher pointed out that the timing of countries’ responses did make a difference in deaths: New Zealand reacted instantaneously with non-pharmaceutical interventions (“lockdowns”) and achieved much more favorable results than countries like the UK, which imposed no mandates until after some COVID deaths had been logged. The conclusion? “Applying protective measures as early and hard as possible consistently led to better outcomes.”
I don’t know about you, but to me that’s all pretty clear. Now, I read summaries of a half-dozen studies that all came to pretty much the same conclusion, with this caveat: Several mentioned “lockdown fatigue,” pointing out that distancing measures are effective only as long as people will comply - and that can be quite a limited period of time. Overall, the research seems to show that lockdowns, such as they existed, did help to slow the COVID spread in direct proportion to the speed at which they were imposed and the duration the “lockees” were willing to endure. Seems pretty clear to me: Lockdowns pretty much worked.
So why is this guy saying they didn’t? (And we weren’t even discussing COVID lockdowns. He just threw that out there, as if he were scoring a point.) Well, there is one article published in Studies in Applied Economics that provides “an analysis from a trio of economists [that] says pandemic lockdowns during the first wave of the COVID-19 pandemic did little to reduce the number of COVID-related deaths.” Aha! I was pretty sure I’d found this man’s source.
The study, which I read about on WebMD, claims that lockdowns “reduced COVID-related deaths by only 0.2%, and shelter-in-place orders reduced deaths by 2.9%.” Now why would this study contradict all the rest? Well, first of all, it was conducted by economists, not by medical professionals or virologists or such. And, secondly, it has not been peer-reviewed; it’s considered a “working paper.” And, thirdly, although it was published under the “Johns Hopkins” name, only one of the economists is affiliated with Johns Hopkins - and not with the School of Medicine. The study’s conclusion was that “lockdowns should be rejected out of hand as a pandemic policy instrument.”
BUT, here’s the crux of the matter concerning the conversation in which I was mired that day: According to WebMD, “many conservatives have seized on the paper to argue that lockdowns never worked and to criticize governments that implemented them.” I found a tweet by Sean Hannity proclaiming the important findings of this “study by Johns Hopkins” with a note that Dr. Oz and Herschel Walker would weigh in on it tonight on Hannity.
Seth Flaxman, PhD, a statistician at Imperial College London, said the three economists “systematically excluded from consideration any study based on the science of disease transmission, meaning that the only studies looked at in the analysis are studies using the methods of economics.” But Sarah Palin posted a video to Facebook, highlighting this working paper and asking whether lockdowns were about “power, not safety.” Sarah Palin, indeed.
And there you have it: confirmation bias at its best! How do I know? Because earlier in that same conversation this man had said, “I’m a conservative,” so I shouldn’t be surprised that he latched onto the one anti-Biden-administration report, however weak, right along with Sean Hannity and Sarah Palin.
Confirmation bias is defined as the tendency to process information by looking for, or interpreting, information that is consistent with one's existing beliefs. Do only conservatives fall prey to that tendency? Not at all. We choose our cable TV channels and networks from a biased perspective, seek out reporters we know we’re going to agree with, read books we suspect will support our position, and give kudos to social media posts that reinforce what we already believe. We are all at risk of this behavior all the time, before, during and after a pandemic. It’s good, once in a while, to have an illustration like this to remind us, as frustrating as it is to be on the receiving end of it.
Should you wish to do some research of your own on pandemic lockdowns, here are some sources you might consider. (The first one is the pre-report to which this gentleman was likely referring.)
Herby, Jonas et al. “A Literature Review and Meta-Analysis of the Effects of Lockdowns on COVID-19 Mortality.” Studies in Applied Economics, Institute for Applied Economics, Global Health, and the Study of Business Enterprise, Johns Hopkins University. posted Jan 2022.
World Health Organization. “Coronavirus disease (COVID-19): Herd immunity, lockdowns and COVID-19.” 31 Dec 2020.
Flaxman, Seth et al. “Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.” Nature. 584 (2020).
Lai, Shengjie et al. “Effect of non-pharmaceutical interventions to contain COVID-19 in China.” Nature. 585 (2020).
Sharfstein, Joshua, vice dean of the Johns Hopkins Bloomberg School of Public Health. Statement emailed to FactCheck.org. 8 Feb 2022.
Best, Paul. “Lockdowns only reduced COVID-19 death rate by .2%, study finds: ‘Lockdowns should be rejected out of hand.'” Fox News. 1 Feb 2022.
Meyerowitz-Katz, Gideon. @GidMK. “This paper has been doing the rounds, claiming that lockdown was useless (the source of the 0.2% effect of lockdown claim). Dozens of people have asked my opinion of it, so here we go: In my opinion, it is a very weak review that doesn’t really show much, if anything 1/n.” Twitter.com. 4 Feb 2022.
Hanke, Steve H., founder and co-director of the Johns Hopkins Institute for Applied Economics, Global Health, and the Study of Business Enterprise. Email interview with FactCheck.org. 18 Feb 2022.
Ferguson, Neil, director of the MRC Centre for Global Infectious Disease Analysis, Jameel Institute, Imperial College London. Statement posted by Science Media Centre. 3 Feb 2022.
Oxford COVID-19 Government Response Tracker. Blavatnik School of Government, University of Oxford. https://covidtracker.bsg.ox.ac.uk/. website accessed 20 Feb 2022.
Chisadza, Carolyn, senior lecturer in economics at the University of Pretoria. Email interview with FactCheck.org. 15 Feb 2022.
Clance, Matthew, associate professor in the Department of Economics at the University of Pretoria. Email interview with FactCheck.org. 16 Feb 2022.
Our World in Data. Cumulative confirmed COVID-19 deaths. website accessed 22 Feb 2022.
Bjornskov, Christian. “Did Lockdown Work? An Economist’s Cross-Country Comparison.” CESifo Economic Studies. 67.3 (2021).
Fuller, James A. et al. “Mitigation Policies and COVID-19–Associated Mortality — 37 European Countries, January 23–June 30, 2020.” Morbidity and Mortality Weekly Report. 70.2 (2021).
Goldstein, P. et al. “Lockdown Fatigue: The Diminishing Effects of Quarantines on the Spread of COVID-19.” Harvard University Center for International Development. 2021.
Stockenhuber, Reinhold. “Did We Respond Quickly Enough? How Policy-Implementation Speed in Response to COVID-19 Affects the Number of Fatal Cases in Europe.” World Medical & Health Policy. 12.4 (2020).
Riley, Lee, chair of the Division of Infectious Disease and Vaccinology at the University of California, Berkeley School of Public Health. Email interview with FactCheck.org. 14 Feb 2022.
Spiegel, Matthew and Heather Tookes. “Business Restrictions and COVID-19 Fatalities.” The Review of Financial Studies. 34.11 (2021).
Chernozhukov, Victor et al. “Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.” Journal of Econometrics. 220. 1 (2021).
Hale, Thomas et al. “Global Assessment of the Relationship between Government Response Measures and COVID-19 Deaths.” medrxiv.org. 6 Jul 2020.
Hale, Thomas et al. “Government responses and COVID-19 deaths: Global evidence across multiple pandemic waves.” Plos One. 9 Jul 2021.
Brauner, Jan M. et al. “Inferring the effectiveness of government interventions against COVID-19.” Science. 371.6531 (2020).
Haug, Mils et al. “Ranking the effectiveness of worldwide COVID-19 government interventions.” Nature Human Behaviour. 4 (2020).
Sharma, Mrinank et al. “Understanding the effectiveness of government interventions against the resurgence of COVID-19 in Europe.” Nature Communications. 12 (2021).
Yang, Jiannan et al. “The impact of non-pharmaceutical interventions on the prevention and control of COVID-19 in New York City.” Chaos: An Interdisciplinary Journal of Nonlinear Science. 31.2 (2021).
Achenbach, Joel and Laura Meckler. “Shutdowns prevented 60 million coronavirus infections in the U.S., study finds.” Washington Post. 8 Jun 2020.
Hsiang, Solomon et al. “The effect of large-scale anti-contagion policies on the COVID-19 pandemic.” Nature. 584 (2020).
Chernozhukov, Victor et al. “Comments on the ‘John Hopkins’ Meta Study (Herby et al., 2022) and Chisadza et al. (2021).” Provided to FactCheck.org. 4 Mar 2022.
Chernozhukov, Victor, professor, Massachusetts Institute of Technology Department of Economics and the Statistics and Data Science Center. Phone interview with FactCheck.org. 8 Mar 2022.