I would answer in the following way. If you are looking for a small effect then an RCT is absolutely essential. If there is a black and white effect, then an RCT is not required. The impact of masking at best is tiny, and at worst it may even have a negative effect.
Now in terms of COVID, even if you don't look at the RCTs, it is evident by just comparing similar countries and similar US states that masking had no impact. e.g. Sweden versus UK, for example or any of the other continental european countries.
Lastly, why do you care about flattening the curve. All that does is prolong the agony because the area under the curve remains unchanged. The only reason to attempt to flatten the curve is if the healthcare system becomes completely overwhelmed. For COVID this was never the case even at the beginning in NYC. Recall NYC never made use of the hospital ship or the Jaffitz center. Yes, things were busy but then hey always are in the winter.
As for the article you linked to, just read it again. they make a lot of assertions for which they have absolutely no evidence. For example, they assume masks work and that this is beyond doubt and "settled science". But it's far from settled, and the RCT data clearly show that masks don't work in the real world. Sure, they may do something under carefully controlled lab conditions, such as speaking into a small hole (e.g. the initial NIH study by Bax & Anfinrud published initially as a letter in the NEJM with a follow-up paper in PNAS), but that's not relevant to real life.
Thanks for this. I found this a very salient point and often missed:
"This approach assumes (a) RCTs are the “best” evidence and (b) combining results from multiple RCTs will give you an average “effect size”.
But RCTs are only the undisputed gold standard for certain kinds of questions. For other questions, a mix of study designs is better. And RCTs should be combined in a meta-analysis only if they are all addressing the same research question in the same way."
Glad it made sense David. You may also enjoy another recent Australian contribution to making medicine better.
<snip>
Two world-leading doctors reveal the true state of modern medicine and how doctors are letting their patients down.
In Hippocrasy, rheumatologist and epidemiologist Rachelle Buchbinder [Cochrane Library editor (Cochrane Musculoskeletal) and author of over 100 Cochrane reviews] and orthopaedic surgeon Ian Harris argue that the benefits of medical treatments are often wildly overstated and the harms understated. That overtreatment and overdiagnosis are rife. And the medical system is not fit for purpose: designed to deliver health care not health.
This powerful exposé reveals the tests, drugs and treatments that provide little or no benefit for patients and the inherent problem of a medical system based on treating rather than preventing illness. The book also provides tips to empower patients – do I really need this treatment? What are the risks? Are there simpler, safer options? What happens if I do nothing? Plus solutions to help restructure how medicine is delivered to help doctors live up to their Hippocratic Oath.
How do you explain that countries who adopted masks early (SE Asia) fared much better with virus control compared to those in the West who didnt? Some would argue that this was like a natural randomized trial of sorts.
My guess is that having an individual as the unit in an RCT on masking during a pandemic is not going to work; you mask well and reach home only to get it from your family; a confounder which cannot be taken care of as it it a pandemic situation. An RCT with the family as the unit, or an RCT on universal masking (as naturally happened in the East vs the West approach), gives the actual answer.
Interesting idea to change the unit of analysis beyond individuals. Since masks are an intervention an individual must wear, how would you see that working practically?
My point is that masking in a pandemic setting may not have worked because the infection may have come at home from the rest of the family, unless they were also masking.
We have seen a very clear difference in the incidence of covid in countries which diligently followed universal masking like Vietnam or Thailand, compared to the west. How else does one explain the difference? It did work practically then. It wouldn't work practically now because it is no longer a pandemic setting, but an endemic setting.
Essentially, if another similar pandemic comes later, universal masking is an answer, unlike the conclusion of the Cochrane review.
Ah, makes sense, thanks! I think the point you make about the different efficacy of masks in a pandemic setting (where nearly everyone is immunologically naive) vs endemic setting is a really important one and not one fully addressed in the Cochrane review or by those arguing that the whole masks issue is settled science.
"no compelling evidence that masks did much to halt the spread of respiratory viruses". Ian Miller in multiple places has charts that show masks did little to slow or stop spread and even wrote a book full of those population level charts. Mask do help a little but does that help overcome the disadvantages? Clarity of speech, facial expressions and the minutia of human interaction are lost behind the coverings. Worse is the sense of confidence in protection when the effect is so small. OTOH, a mask does remind us that we must be cautious except when masking is universal, negating even that benefit.
It seems to me that as far as community transmission and general hospital transmission is concerned, the Cochrane Review shows beyond doubt that masking is completely ineffective at a population level. After all, the data show that surgical masks have no impact whether the respiratory tract infection is a virus-confirmed one or not. Likewise for surgical masks vs N95 masks in the general hospital community. One could do as many large cluster controlled RCTs as one likes, and one is going to find the same result. And that's because, the effect, if it exists, is tiny and insignificant in terms of preventing infection or reducing transmission. (And by insignificant I'm not talking about statistical significance but rather the size of the effect is so small it makes no difference whether it is statistically significant or not).
The remaining question is whether masking and what type of masking (surgical vs N95) has any effect in the context of seeing patients with viral respiratory tract infections whether in the ER or an a COVID ward. In other words, the only remaining question is whether there are circumstances where it would be advisable to wear a mask in terms of protection for the individual and whether having the patient wear a mask under such circumstances reduces transmission. And for that narrowly constructed RCTs in hospitals would be required and in addition one would have to distinguish transmission within the hospital from transmission in the community (which can be hard to do unless one is very careful at monitoring, given that nobody words 24/7 365 days a year in the hospital or more specifically in a COVID ward). Anecdotally I suspect the masks may help for short duration hospital visits/examinations (e.g. 5-10 min tops) but after that the masks are useless. And that's from seeing so many people that I know in households who have infected one another despite wearing masks religiously when one member gets sick. But that's just observational.
Flu shots have become de riguer in many parts of the world and questioning their use and utility labels you as an anti-vaxxer in ways similar to the Covid-19 situation.
Can you'll address this issue? If the Cochrane group says that there is no good evidence that flu shots work, then what exactly is going on worldwide? Thanks.
Good salesmanship along with governments desiring production facilities have a standby purpose. I gave up several years ago. Why take the risk of a defective product?
Very well written and clear - thank you! What I love about this one is the description of how a medical study is like a test, it doesn't provide certainty but it updates our priors. What is useful about this is that it allows us to see where the biases are, and, in this case, they're in the 'common sense' section, which pulls current anecdata about mask compliance to state the priors.
And what is clear is Mandrola's priors that "masks don't work" is coming from a place of selective evidence. But this framing is helpful to show that we don't disagree about this study or its conclusions. The bias is in our priors and what we choose to cite for 'common sense.' My "common sense" is the fact that I worked as a hospitalist during the first wave in NYC, wearing only surgical masks for the first week, spending hours in rooms with patients, then n95s until that summer. Mask adherence was high among co-workers. The only healthcare workers I heard of that got sick were those who ate in communal settings in breakrooms. It seems Cochrane's methodology of excluding observational studies is a big mistake here because it seems like those studies should at least inform our priors, even if we take that evidence with a grain of salt.
The challenge with Cochrane reviews, as pointed out by Trish Greenhalgh, is they're insufficient for real world questions and challenges where policymakers may need to make decisions under high uncertainty and may lean toward precaution. Despite my various levels of immunity, in the absence of concrete evidence it seems reasonable to err on the side of caution and wear masks in certain settings. I still do because the risk of harm (to myself or others) is low while the possibility of benefit (to myself and others) seems to outweigh that.
What is worse is that this debate is playing out publicly. I appreciate Mandrola's nuanced take here, and the nuance of the study authors. But the problems arise in the press. Bloomberg, for example, dedicated their health newsletter to the issue this morning, and their takeaway was "masks don't work," framing a looming controversy with how long can Hong Kong's mask policy hold out since masks aren't shown to work?
The nuance, uncertainty and priors are lost in the translation from dissecting the study on its merits to the reporting on it in the press. And, what follows is the propagation of misinformation - overstating the conclusions of a study when even the study authors were quite circumspect and nuanced in how they reported it.
I struggle with this because this is exactly what we should be doing as scientists - debating the science, being transparent about our priors and how much any new study should update those. But channels are getting crossed as we debate this publicly and nuance is lost in translation, making the issue even more political. Which raises the questions, is this the proper place for this debate? Or is it better suited for a medical journal?
See my post. In the context of the study it seems to me that the case against masking in the community and in the general hospital environment is cut and dried. Masks simply don't work and this is something that has been known ever since the Spanish Flu in 1918 and reconfirmed N times. What you really want to know is whether masks are at all helpful and for how long are masks helpful if one is examining a patient with COVID or another respiratory virus illness.
Saw your post. I appreciate the additional questions you bring to the issue, which I think highlights the many uncertainties here. I'd add that the situation with masks now is different than March 2020 facing a novel virus for which any population-wide immunity was lacking and in which setting masks may prove more (or potentially less) effective at preventing viral transmission.
Currently, I find it hard to agree with such unequivocal statements like "masks simply don't work." We know RCTs can only answer yes/no questions, to reject or accept null hypotheses. But, as you're alluding to in your questions, it seems nuance is important here, and the question we should be asking is "under what conditions and and in what contexts do masks work (and what kind of masks)?" But we can't answer that question directly with the tools we have. We have to answer it over time with serial experiments. Until we determine those conditions, policymakers or hospital/clinic administrators do while the important question remains unsettled?
Not so sure. Thomas Jefferson made an interesting point recently in regard to this whole business. For example, he was talking about the impact of sending people home from the workplace who had an ILI (which includes COVID of course). Well of course this reduces spread in the workplace, but as a consequence increases spread in the home. So there's no way to escape.
In the context of masks in the hospital environment, it is clear to me (and I believe to VP as well) that, in general, masks have no effect. Indeed, the recent RCT at the Cleveland Clinic comparing surgical masks with fitted N95s in healthcare workers in their network (of some 30,000 or more people) showed no difference whatsoever. Since the fit of an N95 is clearly superior to that of a surgical mask and the filtration properties of an N95 are vastly superior to those of a regular surgical mask, this result goes to prove that mask wearing in the hospital environment is in general useless because nobody is wearing them 100% of the time, few are even wearing the N95s properly, people are taking them off to eat and drink, and of course nobody in their right mind is wearing a mask at home.
So the only circumstance where a mask may be useful is when a physician is examining a covid patient for a short period of time. Whether this is even useful under these circumstances needs to be properly tested, but I think most can agree that wearing a mask for 5 min (including a fitted N(5 which is really uncomfortable) is not a big deal. But wearing a mask for 8 hrs a day in the hospital is a very big ask and a big deal, and really shouldn't be done if it has no benefit. And the Cochrane report shows this general use of masking simply doesn't work.
I still struggle with the idea the issue is that clear when the issue seems pretty complicated. I agree that we need to be following the evidence, but for interventions like masking, I'm not convinced an RCT (and a meta analysis aggregating apples and oranges) really provides more clarity or just elicits confirmation bias. What are your thoughts on this piece that Dennis Perry posted on this thread:
The article in the conversation you linked to says absolutely nothing. They present absolutely no evidence for any of their assertions. Basically they say “ we believe masks work and therefore they work” . Except in the real world they don’t work.
Here’s the deal. If masks were truly effective in the real world they would have impacted the course of the pandemic and prevented new waves. But they didn’t.
Nothing is 100%, right? Seems like masks may have an efficacy but it's not like an on-off switch where if people wore masks it would have stopped the pandemic in its tracks. I thought the question we're trying to answer was whether masks can flatten the curve. Maybe I'm missing something but that seems like a matter of percentages and doesn't seem so black and white to me.
Same with that article - I'm surprised by the statement that the article presents no evidence at all. It's true the studies it presented were observational or real-world trials which make it difficult to generalize, but I wouldn't go so far as to call it 'absolutely no evidence'. If I'm hearing you right, it sounds like you would define evidence as RCTs only, is that right?
I'm still fascinated by the psychology and group dynamics that allowed the entire world to do a complete 180 on masks, to completely abandon the scientific method, seek out the weakest of evidence to support our superstitions we embraced out of nowhere.... it really was like a new Dark Age swept over us in 2020. I'm still in awe by it. There's a certain magic of being able to witness this first hand, as I ignorantly assumed we were "better than that". The superstition phase of medicine, whether it was bloodletting, electroshock, lobotomies, mercury elixirs - all of that quackery could never arise again now that we had "science" and EBM to shield us from such irrationality.
It really is remarkable. I truly hope my grandchildren will think I am bullshitting them when I tell them about 2020 someday.
Could you say more about what psychology you're referring to? I'm trying to think about how back in 2020 there was so much that was still unknown, yet a novel virus was causing hospitals to fill at unsustainable rates in China, then Italy and Iran. What would have been your recommendations at the time? I'm trying to understand what you mean that you assumed we would have been "better than that."
While it was a novel virus, we could reasonably assume that it would behave like the other known coronaviruses and comparable respiratory viruses - none of which we believed could be thwarted by masks - at least not since we tried them 100 years ago and found they didn't work. [1] That they were useless combating respiratory viruses was still with the mainstream media as late as April 2020 [2]
In the 100 years since the Spanish Flu, masks were never a part of public health messaging for any pandemic since, including but not limited to the flus of 1957, 1958, 2009, SARS 03, and MERS15. Outside of the operating room, no one in the west ever wore a mask, and even in the OR it was known that the mask didn't even appear to reduce bacterial infection [3][4][5]
In 2014 they were attempted as a means to coerce Nurses who didn't wish to get their annual flu vaccine in Canada, forcing them to wear masks during flu season if they didn't get the shot, but the nurses union successfully sued hospitals attempting this noting "that forcing healthy registered nurses to wear masks for up to six months during the influenza season did little or nothing to prevent transmission of the virus in hospitals" [6]
By February 2020, when Anthony Fauci was still privately telling friends that viruses pass through masks and they aren't helpful, mask enthusiasm was slowly starting to build in the US and people began making the ill-informed argument that masks are part of the culture in Japan and South Korea.
I say "Ill informed" because this ignores 4 key facts:
1) Japan just had a bad flu season in 2019 which their masks didn't seem to help
2) As recent as 2015 it was considered rude in South Korea to wear a mask [7]
3) Mask use in Japan had little to do with avoiding viruses and more about allergies, wanting to not wear makeup, avoid the smell of subways, make their faces look smaller, and avoid people talking to them [8]
4) Mask use in Japan was seen as useless [9] and addictive [10] by western medicine
The argument adopted a rationale common to pseudoscience - the appeal to exotic/Asian/native medicine. You see this with Turmeric, accupuncture, grinding up pangolin bones - this claim that allopathic medicine has not discovered an ancient cure used in the East. I love Japanese culture, so I know they do a lot of weird things there [11], but it seemed unlikely they discovered the cure to the flu 20 years ago and we just decided not to test it by RCT while ~20 million people died worldwide from the flu in 21st century.
Botton line, as of March 2020 the consensus and all available evidence was that masks - and certainly anything below a form-fitted, single use n95, lacked evidence of efficacy [12,13].
And then, out nowhere, everything flipped. Suddenly masks were the single greatest way to stop viruses. Fauci flipped with no new evidence. Robert Redfield declared we could "bring the pandemic to it's knees in 2 weeks if just 80% of us masked". The media either removed or added addendums to any previous story noting that masks didn't work.
Suddenly, people who thought they practiced "science" by demanding rigorous evidence were "maskholes". If you asked for RCTs and replication, you were labeled a Republican. Surreal.
What followed next was unprecedented. Out of nowhere Public Health managed to conjure up hundreds of terrible papers [14], all of which managed to find incredible efficacy in cloth masks. To me, the Missouri Salon MMWR is the capstone of this insanity - as a "study" that would be rejected by a 5th grade science fair was heralded by the CDC and uncritically passed on as "evidence" by the press [15].
Just think about it - we had been doing disinterested studies for nearly 50 years on masks and struggled to find any benefit - yet now, all of the sudden it was easy to find a massive impact. How could that be? Either the researchers between 1970 and 2020 were wrong, or the researchers between 2020 and 2022 were wrong. My money would be that the people writing studies when it became politically favorable were the ones who got it wrong, not the people who had no incentives on which way the signal pointed.
This is where I say the psychology fascinates me, because we entered the modern equivalent of "Tulip Mania".
"It has never been shown that wearing surgical face masks decreases postoperative wound infections. On the contrary, a 50% decrease has been reported after omitting face masks. The present study was designed to reveal any 30% or greater difference in general surgery wound infection rates by using face masks or not"
"Masks may be used to protect the operating team from drops of infected blood and from airborne infections, but have not been proven to protect the patient operated by a healthy operating team"
[4] Unmasking the surgeons: the evidence base behind the use of facemasks in surgery
"In the modern era, there has also been a scarcity of experimental evidence to support the effectiveness of facemasks in the prevention of surgical site infections. The earliest retrospective studies7 failed to demonstrate any statistically significant improvement in surgical site infection rates following the use of masks. "
[5] Disposable surgical face masks for preventing surgical wound infection in clean surgery
"Three trials were included, involving a total of 2113 participants. There was no statistically significant difference in infection rates between the masked and unmasked group in any of the trials."
Thanks for these details and citations. I'm still having trouble following. You said "Botton line, as of March 2020 the consensus and all available evidence was that masks - and certainly anything below a form-fitted, single use n95, lacked evidence of efficacy [12,13]."
But when I look at those citations I see:
- ""The evidence is really controversial. Some studies have shown there may be some protection by wearing face masks and others have shown there really isn't at all," says Carol McLay, an infectious disease consultant at the University of Kentucky." and
- "Temte says it's really too early to know whether masks are an effective way to protect against SARS. Researchers simply don't know enough about the virus and how it spreads. "On the flip side, there is probably not a whole lot of harm in it," Temte tells WebMD. "Any sort of barrier will reduce likelihood of droplet transmission.""
If I understand what you're saying, you perceive that there was a consensus at the time, and there was a sudden change where people suddenly started believing that masks worked. Is that right?
If so, I'm confused because the citations you point to (#12 & 13) note that it's controversial and there's no consensus. What am I missing?
Here's why I argue the consensus prior to April 2020 was that masks don't work:
It's pretty simple.
We didn't wear them anywhere outside the operation room in medical settings.
When you had the flu and saw your doctor, no one was masked. When your surgeon visited you pre and post op, no mask. When there's a surge of Hospital Acquired Infections, no one wore masks. In the rare case you had a patient with drug resistant TB, they might wear a surgical mask, the provider might wear a form-fitted n95.
I think this speaks to a consensus that masks weren't beneficial prior to April 2020 and why we weren't surprised to find out Dr. Fauci had privately said masks don't stop viruses to his friends [1]
This consensus likely rested on the following reasons:
1) We already tried them already and found they didn't work 100 years earlier
2) We ran many experiments and had yet to find benefit (even finding that they didn't even help achieve asepsis in the OR) - I think this can't be understated - that the research we did on efficacy when it was disinterested in outcome - found such low benefit that we didn't bother extending mask wearing even in doctors offices when you were sick.
3) While it may seem intuitive that covering your mouth could stop a respiratory virus, a basic understand of fluid dynamics would reveal why they couldn't work without a perfect seal (which is why I was disappointed to see BIll Nye make the ludicrous argument that because he couldn't blow out candles wearing a cloth mask, they must work)
4) The intuitive benefit of masks is further debunked when you consider just how small viruses are. The micropore in your typical cloth mask could fit over 300 million virions. Even the micropore of an n95 could fit 3,000 virions. Remember, we only discovered viruses because some smaller pathogen was found getting through ceramic Chamberland Filters which were designed to filter out bacteria. We named these unseen pathogens "Filter Passing Viruses" which became known as "Filterable Viruses" by 1903.
This brings us to April 2020 when there was a sudden and immediate flip on masks. They went from "doesn't really provide much protection" to "the science is settled - masks work" practically overnight. [2]
And this is where the psychology aspect becomes fascinating to me.
We know that governments have to recommend something - anything - during a crisis, even if the recommendation doesn't make much sense (George W Bush telling people to go shopping and visit Disneyworld immediately after 9/11 comes to mind). I get that. It wasn't surprising that government agencies like the CDC would propose something like facemasks.
What was surprising was the near hysterical obsession the media, public, and science community had with masking.
To your earlier question - "What would have been your recommendations at the time" - I could perhaps see a measured recommendation, something along the lines "out of caution we are recommending facemasks to be worn in hospitals while we gather more data on their potential benefit". That would have been a step following the science, while being pragmatic about the lack of evidence.
Instead we saw the media go on witch hunts, first erasing their own wrong-think [3], and then attacking anyone who referenced the "old" science, such as when Michael Osterholm had to prostrate before the crowd after "MASKS4ALL" Jeremy Howard caught him saying "Cloth masks, I think… have little impact, if any" in a radio interview [4] or when Lisa Brosseau, an expert on respirators regularly cited by the CDC [5] similarly had to defend her expertise [6].
The sloganeering followed [7], again, more evidence to me this was a social hysteria, as we bragged about putting children in cloth masks outdoors [8]
I get it though. Carl Sagan had predicted [9] we may wind up someday "clutching our crystals...unable to distinguish between what feels good and what’s true, we slide, almost without noticing, back into superstition and darkness."
It's understandable that in face of a crisis people would gravitate to anything that makes them feel good. Even if it isn't scientifically supported. Hell, even the 2nd best hospital in the world out by me offers a Reiki program (seriously) [10]
But I naively thought science had built in error correcting, that science had a rigid system to distinguish fact from fiction. That just because we wanted something to work, we couldn't fraudulently make it so. I naively thought something like Wakefield's quackery making it into The Lancet was a one-off, a mistake in the system that had been corrected in the decade since the retraction.
Instead, what happened was that instantly, despite decades of not being able to find a strong benefit of masking, every, single, study found huge benefit. That alone should raise alarm of pseudoscience making it past the error checking.
And it became self-referential. Consider one of the earliest papers to find massive benefit from masking. "Identifying airborne transmission as the dominant route for the spread of COVID-19". [11]
Released in June 2020, it instantly swept through the media landing in over 400 news stories, 30,000 tweets, and would go on to be cited 800 times. Nearly every mention of masks in the media during the summer of 2020 would cite this paper.
Yet with a simple read though, it was immediately obvious the authors simply data drudged their results (See Adam Pearce's analysis for example [12]. Almost immediately there was a retraction request [13] signed by nearly 50 scientists - all saying they think masks might work - but that this paper can't be used to support the hypothesis. The retraction request and others were [14] met with silence.
(I later learned apparently you can get fast tracked peer review if one of the authors listed is a Nobel Prize winner - even if the fields are different, here Molina, one of the contributing authors won a nobel prize in Chemistry IRRC and this was one of the reasons this not-very-good paper pass through uncritically).
The PNAS paper set the tone for what was to come. An avalanche of terrible papers drowned the medical and scientific community, which didn't even realize they were getting "Gish Galloped". Two Hairdressers in Missouri makes MMWR. Kansas Mask study is falsified immediately but get praised for showing mask mandates can lower transmission 100%.
The Bangladesh RCT finds a 1% absolute benefit *only* in surgical and *only* in people over 50, ignores all confounders, yet the authors get to tell the Washington Post “..this should basically end any scientific debate about whether masks can be effective in combating covid at the population level,” [15]
I could go on forever.... it's embarrassing and depressing that this could happen to us.
As Vinay correctly pointed out yesterday [16 ], we simply decided to only be skeptics and demand rigorous evidence for some things and not all things (and it turns out the things we give a pass are politically favorable). If someone shares a paper on Ivermectin efficacy, suddenly scientists are concerned with sample sizes, confidence intervals, pre-registration, p values and study design.
Yet a study on School Masking [17] where the authors aren't even aware some of the experimental groups were part of the control group [18], lobbied for the results they were seeking [19], failed to explain why the experimental group was already having a higher incidence rate before the intervention [20], and dismissed criticism they missed confounders by an appeal to authority [21] gets published in the NEJM.
[9] – "I have a foreboding of an America in my children’s or grandchildren’s time – when the United States is service and information economy; when nearly all the key manufacturing industries have slipped away to other countries; when awesome technological powers are in the hand of a very few, and no one representing the public interest can even grasp the issues; when the people have lost the ability to set their own agendas or knowledgeably question those in authority; when, clutching our crystals and nervously consulting our horoscope’s, our critical faculties in decline, unable to distinguish between what feels good and what’s true, we slide, almost without noticing, back into superstition and darkness."
[20] Just look at the rates prior to mandate lifting, schools which would remove masks already had a higher rate of infection which shows there was a confounder before study period. If someone presented thist to me I would have pointed out they data drudged the result but moving the start date as close as possible to avoid confronting the confounder.
Thanks for the references and the helpful explanation. I think what you're saying is consensus doesn't seem like a consensus to me as I still see a lot of controversy even prior to the pandemic. It raises the question: How do you define consensus? Can you give another example besides masks? I like the example of hydroxychloroquine - initially there was uncertainty and lots of physicians (myself included) prescribed it. But as more evidence came out, there began to be clarity - and consensus - around the fact that any benefit seemed minimal, and there was reasonable potential for harm.
Great post but the data must be translated to action
Question: You are getting on a plane holding a an n95 mask in your hand wracked with indecision and trying to decide what to do. In deed you have moderate to high uncertainty about the evidence but you must make a behavioral choice. You find your seat and note a big bloke already sitting in the adjoining seat who is slobbering and sneezing into his tee shirt.
{POLLING QUESTION
do you put your n95 on or not ?
YES
NO
PS ( would love to know what Dr Prasad would do? ---[but doubt he is packing n95s)
No. The flight already sounds like it going to be miserable. Why make it worse wearing a mask? If you were going to get sick, you were going to get sick.
Best to just ask the flight attendant for a double gin on the rocks.
This post had sizzle. Terse but yet had punch. Lucidly written for those of us without Yale diplomas on the wall or 67-foot yachts down at our local marina. I return now to my bowl of oat cereal with skim milk, some decaf coffee, and the ingestion of daily drugs: angiotensin receptor blockade agent and homeopathic doses of a statin and beta-blocker.
Under the standard homeopathic dilution process (30x 100:1), the likelihood of even a single molecule of the target agent being present is zero. Surely that’s not what you meant.
I would answer in the following way. If you are looking for a small effect then an RCT is absolutely essential. If there is a black and white effect, then an RCT is not required. The impact of masking at best is tiny, and at worst it may even have a negative effect.
Now in terms of COVID, even if you don't look at the RCTs, it is evident by just comparing similar countries and similar US states that masking had no impact. e.g. Sweden versus UK, for example or any of the other continental european countries.
Lastly, why do you care about flattening the curve. All that does is prolong the agony because the area under the curve remains unchanged. The only reason to attempt to flatten the curve is if the healthcare system becomes completely overwhelmed. For COVID this was never the case even at the beginning in NYC. Recall NYC never made use of the hospital ship or the Jaffitz center. Yes, things were busy but then hey always are in the winter.
As for the article you linked to, just read it again. they make a lot of assertions for which they have absolutely no evidence. For example, they assume masks work and that this is beyond doubt and "settled science". But it's far from settled, and the RCT data clearly show that masks don't work in the real world. Sure, they may do something under carefully controlled lab conditions, such as speaking into a small hole (e.g. the initial NIH study by Bax & Anfinrud published initially as a letter in the NEJM with a follow-up paper in PNAS), but that's not relevant to real life.
I've followed John's blog and writings for many years. And he is sensible. A small contribution from down under:
https://theconversation.com/yes-masks-reduce-the-risk-of-spreading-covid-despite-a-review-saying-they-dont-198992
Thanks for this. I found this a very salient point and often missed:
"This approach assumes (a) RCTs are the “best” evidence and (b) combining results from multiple RCTs will give you an average “effect size”.
But RCTs are only the undisputed gold standard for certain kinds of questions. For other questions, a mix of study designs is better. And RCTs should be combined in a meta-analysis only if they are all addressing the same research question in the same way."
Glad it made sense David. You may also enjoy another recent Australian contribution to making medicine better.
<snip>
Two world-leading doctors reveal the true state of modern medicine and how doctors are letting their patients down.
In Hippocrasy, rheumatologist and epidemiologist Rachelle Buchbinder [Cochrane Library editor (Cochrane Musculoskeletal) and author of over 100 Cochrane reviews] and orthopaedic surgeon Ian Harris argue that the benefits of medical treatments are often wildly overstated and the harms understated. That overtreatment and overdiagnosis are rife. And the medical system is not fit for purpose: designed to deliver health care not health.
This powerful exposé reveals the tests, drugs and treatments that provide little or no benefit for patients and the inherent problem of a medical system based on treating rather than preventing illness. The book also provides tips to empower patients – do I really need this treatment? What are the risks? Are there simpler, safer options? What happens if I do nothing? Plus solutions to help restructure how medicine is delivered to help doctors live up to their Hippocratic Oath.
...
</snip>
https://www.amazon.com.au/Hippocrasy-doctors-betraying-their-oath-ebook/dp/B09GLTYNFN
Sounds up my alley. I very much enjoyed this book:
https://www.penguinrandomhouse.com/books/205067/overdiagnosed-by-dr-h-gilbert-welch-dr-lisa-m-schwartz-and-dr-steven-woloshin/
How do you explain that countries who adopted masks early (SE Asia) fared much better with virus control compared to those in the West who didnt? Some would argue that this was like a natural randomized trial of sorts.
My guess is that having an individual as the unit in an RCT on masking during a pandemic is not going to work; you mask well and reach home only to get it from your family; a confounder which cannot be taken care of as it it a pandemic situation. An RCT with the family as the unit, or an RCT on universal masking (as naturally happened in the East vs the West approach), gives the actual answer.
Interesting idea to change the unit of analysis beyond individuals. Since masks are an intervention an individual must wear, how would you see that working practically?
My point is that masking in a pandemic setting may not have worked because the infection may have come at home from the rest of the family, unless they were also masking.
We have seen a very clear difference in the incidence of covid in countries which diligently followed universal masking like Vietnam or Thailand, compared to the west. How else does one explain the difference? It did work practically then. It wouldn't work practically now because it is no longer a pandemic setting, but an endemic setting.
Essentially, if another similar pandemic comes later, universal masking is an answer, unlike the conclusion of the Cochrane review.
Ah, makes sense, thanks! I think the point you make about the different efficacy of masks in a pandemic setting (where nearly everyone is immunologically naive) vs endemic setting is a really important one and not one fully addressed in the Cochrane review or by those arguing that the whole masks issue is settled science.
"no compelling evidence that masks did much to halt the spread of respiratory viruses". Ian Miller in multiple places has charts that show masks did little to slow or stop spread and even wrote a book full of those population level charts. Mask do help a little but does that help overcome the disadvantages? Clarity of speech, facial expressions and the minutia of human interaction are lost behind the coverings. Worse is the sense of confidence in protection when the effect is so small. OTOH, a mask does remind us that we must be cautious except when masking is universal, negating even that benefit.
It seems to me that as far as community transmission and general hospital transmission is concerned, the Cochrane Review shows beyond doubt that masking is completely ineffective at a population level. After all, the data show that surgical masks have no impact whether the respiratory tract infection is a virus-confirmed one or not. Likewise for surgical masks vs N95 masks in the general hospital community. One could do as many large cluster controlled RCTs as one likes, and one is going to find the same result. And that's because, the effect, if it exists, is tiny and insignificant in terms of preventing infection or reducing transmission. (And by insignificant I'm not talking about statistical significance but rather the size of the effect is so small it makes no difference whether it is statistically significant or not).
The remaining question is whether masking and what type of masking (surgical vs N95) has any effect in the context of seeing patients with viral respiratory tract infections whether in the ER or an a COVID ward. In other words, the only remaining question is whether there are circumstances where it would be advisable to wear a mask in terms of protection for the individual and whether having the patient wear a mask under such circumstances reduces transmission. And for that narrowly constructed RCTs in hospitals would be required and in addition one would have to distinguish transmission within the hospital from transmission in the community (which can be hard to do unless one is very careful at monitoring, given that nobody words 24/7 365 days a year in the hospital or more specifically in a COVID ward). Anecdotally I suspect the masks may help for short duration hospital visits/examinations (e.g. 5-10 min tops) but after that the masks are useless. And that's from seeing so many people that I know in households who have infected one another despite wearing masks religiously when one member gets sick. But that's just observational.
A little tangential, but if we accept Cochrane reviews as a kind of base truth to help with decision making, then what about the Cochrane reviews that question the efficacy of flu shots, both in healthy and elderly adults? https://community.cochrane.org/news/why-have-three-long-running-cochrane-reviews-influenza-vaccines-been-stabilised
Flu shots have become de riguer in many parts of the world and questioning their use and utility labels you as an anti-vaxxer in ways similar to the Covid-19 situation.
Can you'll address this issue? If the Cochrane group says that there is no good evidence that flu shots work, then what exactly is going on worldwide? Thanks.
Good salesmanship along with governments desiring production facilities have a standby purpose. I gave up several years ago. Why take the risk of a defective product?
Very well written and clear - thank you! What I love about this one is the description of how a medical study is like a test, it doesn't provide certainty but it updates our priors. What is useful about this is that it allows us to see where the biases are, and, in this case, they're in the 'common sense' section, which pulls current anecdata about mask compliance to state the priors.
And what is clear is Mandrola's priors that "masks don't work" is coming from a place of selective evidence. But this framing is helpful to show that we don't disagree about this study or its conclusions. The bias is in our priors and what we choose to cite for 'common sense.' My "common sense" is the fact that I worked as a hospitalist during the first wave in NYC, wearing only surgical masks for the first week, spending hours in rooms with patients, then n95s until that summer. Mask adherence was high among co-workers. The only healthcare workers I heard of that got sick were those who ate in communal settings in breakrooms. It seems Cochrane's methodology of excluding observational studies is a big mistake here because it seems like those studies should at least inform our priors, even if we take that evidence with a grain of salt.
The challenge with Cochrane reviews, as pointed out by Trish Greenhalgh, is they're insufficient for real world questions and challenges where policymakers may need to make decisions under high uncertainty and may lean toward precaution. Despite my various levels of immunity, in the absence of concrete evidence it seems reasonable to err on the side of caution and wear masks in certain settings. I still do because the risk of harm (to myself or others) is low while the possibility of benefit (to myself and others) seems to outweigh that.
What is worse is that this debate is playing out publicly. I appreciate Mandrola's nuanced take here, and the nuance of the study authors. But the problems arise in the press. Bloomberg, for example, dedicated their health newsletter to the issue this morning, and their takeaway was "masks don't work," framing a looming controversy with how long can Hong Kong's mask policy hold out since masks aren't shown to work?
The nuance, uncertainty and priors are lost in the translation from dissecting the study on its merits to the reporting on it in the press. And, what follows is the propagation of misinformation - overstating the conclusions of a study when even the study authors were quite circumspect and nuanced in how they reported it.
I struggle with this because this is exactly what we should be doing as scientists - debating the science, being transparent about our priors and how much any new study should update those. But channels are getting crossed as we debate this publicly and nuance is lost in translation, making the issue even more political. Which raises the questions, is this the proper place for this debate? Or is it better suited for a medical journal?
See my post. In the context of the study it seems to me that the case against masking in the community and in the general hospital environment is cut and dried. Masks simply don't work and this is something that has been known ever since the Spanish Flu in 1918 and reconfirmed N times. What you really want to know is whether masks are at all helpful and for how long are masks helpful if one is examining a patient with COVID or another respiratory virus illness.
Saw your post. I appreciate the additional questions you bring to the issue, which I think highlights the many uncertainties here. I'd add that the situation with masks now is different than March 2020 facing a novel virus for which any population-wide immunity was lacking and in which setting masks may prove more (or potentially less) effective at preventing viral transmission.
Currently, I find it hard to agree with such unequivocal statements like "masks simply don't work." We know RCTs can only answer yes/no questions, to reject or accept null hypotheses. But, as you're alluding to in your questions, it seems nuance is important here, and the question we should be asking is "under what conditions and and in what contexts do masks work (and what kind of masks)?" But we can't answer that question directly with the tools we have. We have to answer it over time with serial experiments. Until we determine those conditions, policymakers or hospital/clinic administrators do while the important question remains unsettled?
Not so sure. Thomas Jefferson made an interesting point recently in regard to this whole business. For example, he was talking about the impact of sending people home from the workplace who had an ILI (which includes COVID of course). Well of course this reduces spread in the workplace, but as a consequence increases spread in the home. So there's no way to escape.
In the context of masks in the hospital environment, it is clear to me (and I believe to VP as well) that, in general, masks have no effect. Indeed, the recent RCT at the Cleveland Clinic comparing surgical masks with fitted N95s in healthcare workers in their network (of some 30,000 or more people) showed no difference whatsoever. Since the fit of an N95 is clearly superior to that of a surgical mask and the filtration properties of an N95 are vastly superior to those of a regular surgical mask, this result goes to prove that mask wearing in the hospital environment is in general useless because nobody is wearing them 100% of the time, few are even wearing the N95s properly, people are taking them off to eat and drink, and of course nobody in their right mind is wearing a mask at home.
So the only circumstance where a mask may be useful is when a physician is examining a covid patient for a short period of time. Whether this is even useful under these circumstances needs to be properly tested, but I think most can agree that wearing a mask for 5 min (including a fitted N(5 which is really uncomfortable) is not a big deal. But wearing a mask for 8 hrs a day in the hospital is a very big ask and a big deal, and really shouldn't be done if it has no benefit. And the Cochrane report shows this general use of masking simply doesn't work.
I still struggle with the idea the issue is that clear when the issue seems pretty complicated. I agree that we need to be following the evidence, but for interventions like masking, I'm not convinced an RCT (and a meta analysis aggregating apples and oranges) really provides more clarity or just elicits confirmation bias. What are your thoughts on this piece that Dennis Perry posted on this thread:
https://theconversation.com/yes-masks-reduce-the-risk-of-spreading-covid-despite-a-review-saying-they-dont-198992
The article in the conversation you linked to says absolutely nothing. They present absolutely no evidence for any of their assertions. Basically they say “ we believe masks work and therefore they work” . Except in the real world they don’t work.
Here’s the deal. If masks were truly effective in the real world they would have impacted the course of the pandemic and prevented new waves. But they didn’t.
Nothing is 100%, right? Seems like masks may have an efficacy but it's not like an on-off switch where if people wore masks it would have stopped the pandemic in its tracks. I thought the question we're trying to answer was whether masks can flatten the curve. Maybe I'm missing something but that seems like a matter of percentages and doesn't seem so black and white to me.
Same with that article - I'm surprised by the statement that the article presents no evidence at all. It's true the studies it presented were observational or real-world trials which make it difficult to generalize, but I wouldn't go so far as to call it 'absolutely no evidence'. If I'm hearing you right, it sounds like you would define evidence as RCTs only, is that right?
I'm still fascinated by the psychology and group dynamics that allowed the entire world to do a complete 180 on masks, to completely abandon the scientific method, seek out the weakest of evidence to support our superstitions we embraced out of nowhere.... it really was like a new Dark Age swept over us in 2020. I'm still in awe by it. There's a certain magic of being able to witness this first hand, as I ignorantly assumed we were "better than that". The superstition phase of medicine, whether it was bloodletting, electroshock, lobotomies, mercury elixirs - all of that quackery could never arise again now that we had "science" and EBM to shield us from such irrationality.
It really is remarkable. I truly hope my grandchildren will think I am bullshitting them when I tell them about 2020 someday.
Could you say more about what psychology you're referring to? I'm trying to think about how back in 2020 there was so much that was still unknown, yet a novel virus was causing hospitals to fill at unsustainable rates in China, then Italy and Iran. What would have been your recommendations at the time? I'm trying to understand what you mean that you assumed we would have been "better than that."
No problem David,
While it was a novel virus, we could reasonably assume that it would behave like the other known coronaviruses and comparable respiratory viruses - none of which we believed could be thwarted by masks - at least not since we tried them 100 years ago and found they didn't work. [1] That they were useless combating respiratory viruses was still with the mainstream media as late as April 2020 [2]
In the 100 years since the Spanish Flu, masks were never a part of public health messaging for any pandemic since, including but not limited to the flus of 1957, 1958, 2009, SARS 03, and MERS15. Outside of the operating room, no one in the west ever wore a mask, and even in the OR it was known that the mask didn't even appear to reduce bacterial infection [3][4][5]
In 2014 they were attempted as a means to coerce Nurses who didn't wish to get their annual flu vaccine in Canada, forcing them to wear masks during flu season if they didn't get the shot, but the nurses union successfully sued hospitals attempting this noting "that forcing healthy registered nurses to wear masks for up to six months during the influenza season did little or nothing to prevent transmission of the virus in hospitals" [6]
By February 2020, when Anthony Fauci was still privately telling friends that viruses pass through masks and they aren't helpful, mask enthusiasm was slowly starting to build in the US and people began making the ill-informed argument that masks are part of the culture in Japan and South Korea.
I say "Ill informed" because this ignores 4 key facts:
1) Japan just had a bad flu season in 2019 which their masks didn't seem to help
2) As recent as 2015 it was considered rude in South Korea to wear a mask [7]
3) Mask use in Japan had little to do with avoiding viruses and more about allergies, wanting to not wear makeup, avoid the smell of subways, make their faces look smaller, and avoid people talking to them [8]
4) Mask use in Japan was seen as useless [9] and addictive [10] by western medicine
The argument adopted a rationale common to pseudoscience - the appeal to exotic/Asian/native medicine. You see this with Turmeric, accupuncture, grinding up pangolin bones - this claim that allopathic medicine has not discovered an ancient cure used in the East. I love Japanese culture, so I know they do a lot of weird things there [11], but it seemed unlikely they discovered the cure to the flu 20 years ago and we just decided not to test it by RCT while ~20 million people died worldwide from the flu in 21st century.
Botton line, as of March 2020 the consensus and all available evidence was that masks - and certainly anything below a form-fitted, single use n95, lacked evidence of efficacy [12,13].
And then, out nowhere, everything flipped. Suddenly masks were the single greatest way to stop viruses. Fauci flipped with no new evidence. Robert Redfield declared we could "bring the pandemic to it's knees in 2 weeks if just 80% of us masked". The media either removed or added addendums to any previous story noting that masks didn't work.
Suddenly, people who thought they practiced "science" by demanding rigorous evidence were "maskholes". If you asked for RCTs and replication, you were labeled a Republican. Surreal.
What followed next was unprecedented. Out of nowhere Public Health managed to conjure up hundreds of terrible papers [14], all of which managed to find incredible efficacy in cloth masks. To me, the Missouri Salon MMWR is the capstone of this insanity - as a "study" that would be rejected by a 5th grade science fair was heralded by the CDC and uncritically passed on as "evidence" by the press [15].
Just think about it - we had been doing disinterested studies for nearly 50 years on masks and struggled to find any benefit - yet now, all of the sudden it was easy to find a massive impact. How could that be? Either the researchers between 1970 and 2020 were wrong, or the researchers between 2020 and 2022 were wrong. My money would be that the people writing studies when it became politically favorable were the ones who got it wrong, not the people who had no incentives on which way the signal pointed.
This is where I say the psychology fascinates me, because we entered the modern equivalent of "Tulip Mania".
______________________________
[1] https://www.newspapers.com/clip/85322935/the-flu-mask-has-been-discarded-as-a/
[2] https://www.washingtonpost.com/history/2020/04/02/everyone-wore-masks-during-1918-flu-pandemic-they-were-useless/
[3] Postoperative wound infections and surgical face masks: A controlled study
https://link.springer.com/article/10.1007/BF01658736
"It has never been shown that wearing surgical face masks decreases postoperative wound infections. On the contrary, a 50% decrease has been reported after omitting face masks. The present study was designed to reveal any 30% or greater difference in general surgery wound infection rates by using face masks or not"
"Masks may be used to protect the operating team from drops of infected blood and from airborne infections, but have not been proven to protect the patient operated by a healthy operating team"
[4] Unmasking the surgeons: the evidence base behind the use of facemasks in surgery
https://www.ncbi.nlm.nih.gov/labs/pmc/articles/PMC4480558/
"In the modern era, there has also been a scarcity of experimental evidence to support the effectiveness of facemasks in the prevention of surgical site infections. The earliest retrospective studies7 failed to demonstrate any statistically significant improvement in surgical site infection rates following the use of masks. "
[5] Disposable surgical face masks for preventing surgical wound infection in clean surgery
https://pubmed.ncbi.nlm.nih.gov/24532167/
"Three trials were included, involving a total of 2113 participants. There was no statistically significant difference in infection rates between the masked and unmasked group in any of the trials."
[6] https://www.newswire.ca/news-releases/ona-wins-landmark-influenza-vaccine-or-mask-grievance-526265811.html
[7] News story covering a professor in South Korea kicking out a student from Hong Kong from wearing mask because it is considered rude
http://news.tvb.com/local/5575af7d6db28c594d000003/?lang=cht
[8] This interview gives a great glimpse into the psyche of people wearing masks in Japan pre 2020 https://www.youtube.com/watch?v=PFCapGQWfAw
[9] https://time.com/3814975/ladies-and-germs/
[10] https://www.straitstimes.com/asia/east-asia/mask-appeal
[11] https://www.telegraph.co.uk/news/worldnews/asia/japan/1451864/Tokyo-calls-for-ban-on-sale-of-used-schoolgirls-underwear.html
[12]
https://www.npr.org/sections/goatsandsoda/2015/06/02/411224670/south-koreans-mask-up-in-the-face-of-mers-scare
[13] https://www.webmd.com/lung/news/20030429/can-mask-protect-you-from-sars
[14] I've tracked them all here, building off the list Your Local Epidemiologist shared in November of 2020, without realizing she got Gish Gallopped
https://docs.google.com/spreadsheets/d/1ahaJui6Af0kGYMwHgAtnKCE6-bHbCLxnrQxuMC0kygA/edit?usp=sharing
[15] https://cdc.altmetric.com/details/85785045/news/page:5
Thanks for these details and citations. I'm still having trouble following. You said "Botton line, as of March 2020 the consensus and all available evidence was that masks - and certainly anything below a form-fitted, single use n95, lacked evidence of efficacy [12,13]."
But when I look at those citations I see:
- ""The evidence is really controversial. Some studies have shown there may be some protection by wearing face masks and others have shown there really isn't at all," says Carol McLay, an infectious disease consultant at the University of Kentucky." and
- "Temte says it's really too early to know whether masks are an effective way to protect against SARS. Researchers simply don't know enough about the virus and how it spreads. "On the flip side, there is probably not a whole lot of harm in it," Temte tells WebMD. "Any sort of barrier will reduce likelihood of droplet transmission.""
If I understand what you're saying, you perceive that there was a consensus at the time, and there was a sudden change where people suddenly started believing that masks worked. Is that right?
If so, I'm confused because the citations you point to (#12 & 13) note that it's controversial and there's no consensus. What am I missing?
Here's why I argue the consensus prior to April 2020 was that masks don't work:
It's pretty simple.
We didn't wear them anywhere outside the operation room in medical settings.
When you had the flu and saw your doctor, no one was masked. When your surgeon visited you pre and post op, no mask. When there's a surge of Hospital Acquired Infections, no one wore masks. In the rare case you had a patient with drug resistant TB, they might wear a surgical mask, the provider might wear a form-fitted n95.
I think this speaks to a consensus that masks weren't beneficial prior to April 2020 and why we weren't surprised to find out Dr. Fauci had privately said masks don't stop viruses to his friends [1]
This consensus likely rested on the following reasons:
1) We already tried them already and found they didn't work 100 years earlier
2) We ran many experiments and had yet to find benefit (even finding that they didn't even help achieve asepsis in the OR) - I think this can't be understated - that the research we did on efficacy when it was disinterested in outcome - found such low benefit that we didn't bother extending mask wearing even in doctors offices when you were sick.
3) While it may seem intuitive that covering your mouth could stop a respiratory virus, a basic understand of fluid dynamics would reveal why they couldn't work without a perfect seal (which is why I was disappointed to see BIll Nye make the ludicrous argument that because he couldn't blow out candles wearing a cloth mask, they must work)
4) The intuitive benefit of masks is further debunked when you consider just how small viruses are. The micropore in your typical cloth mask could fit over 300 million virions. Even the micropore of an n95 could fit 3,000 virions. Remember, we only discovered viruses because some smaller pathogen was found getting through ceramic Chamberland Filters which were designed to filter out bacteria. We named these unseen pathogens "Filter Passing Viruses" which became known as "Filterable Viruses" by 1903.
This brings us to April 2020 when there was a sudden and immediate flip on masks. They went from "doesn't really provide much protection" to "the science is settled - masks work" practically overnight. [2]
And this is where the psychology aspect becomes fascinating to me.
We know that governments have to recommend something - anything - during a crisis, even if the recommendation doesn't make much sense (George W Bush telling people to go shopping and visit Disneyworld immediately after 9/11 comes to mind). I get that. It wasn't surprising that government agencies like the CDC would propose something like facemasks.
What was surprising was the near hysterical obsession the media, public, and science community had with masking.
To your earlier question - "What would have been your recommendations at the time" - I could perhaps see a measured recommendation, something along the lines "out of caution we are recommending facemasks to be worn in hospitals while we gather more data on their potential benefit". That would have been a step following the science, while being pragmatic about the lack of evidence.
Instead we saw the media go on witch hunts, first erasing their own wrong-think [3], and then attacking anyone who referenced the "old" science, such as when Michael Osterholm had to prostrate before the crowd after "MASKS4ALL" Jeremy Howard caught him saying "Cloth masks, I think… have little impact, if any" in a radio interview [4] or when Lisa Brosseau, an expert on respirators regularly cited by the CDC [5] similarly had to defend her expertise [6].
The sloganeering followed [7], again, more evidence to me this was a social hysteria, as we bragged about putting children in cloth masks outdoors [8]
I get it though. Carl Sagan had predicted [9] we may wind up someday "clutching our crystals...unable to distinguish between what feels good and what’s true, we slide, almost without noticing, back into superstition and darkness."
It's understandable that in face of a crisis people would gravitate to anything that makes them feel good. Even if it isn't scientifically supported. Hell, even the 2nd best hospital in the world out by me offers a Reiki program (seriously) [10]
But I naively thought science had built in error correcting, that science had a rigid system to distinguish fact from fiction. That just because we wanted something to work, we couldn't fraudulently make it so. I naively thought something like Wakefield's quackery making it into The Lancet was a one-off, a mistake in the system that had been corrected in the decade since the retraction.
Instead, what happened was that instantly, despite decades of not being able to find a strong benefit of masking, every, single, study found huge benefit. That alone should raise alarm of pseudoscience making it past the error checking.
And it became self-referential. Consider one of the earliest papers to find massive benefit from masking. "Identifying airborne transmission as the dominant route for the spread of COVID-19". [11]
Released in June 2020, it instantly swept through the media landing in over 400 news stories, 30,000 tweets, and would go on to be cited 800 times. Nearly every mention of masks in the media during the summer of 2020 would cite this paper.
Yet with a simple read though, it was immediately obvious the authors simply data drudged their results (See Adam Pearce's analysis for example [12]. Almost immediately there was a retraction request [13] signed by nearly 50 scientists - all saying they think masks might work - but that this paper can't be used to support the hypothesis. The retraction request and others were [14] met with silence.
(I later learned apparently you can get fast tracked peer review if one of the authors listed is a Nobel Prize winner - even if the fields are different, here Molina, one of the contributing authors won a nobel prize in Chemistry IRRC and this was one of the reasons this not-very-good paper pass through uncritically).
The PNAS paper set the tone for what was to come. An avalanche of terrible papers drowned the medical and scientific community, which didn't even realize they were getting "Gish Galloped". Two Hairdressers in Missouri makes MMWR. Kansas Mask study is falsified immediately but get praised for showing mask mandates can lower transmission 100%.
The Bangladesh RCT finds a 1% absolute benefit *only* in surgical and *only* in people over 50, ignores all confounders, yet the authors get to tell the Washington Post “..this should basically end any scientific debate about whether masks can be effective in combating covid at the population level,” [15]
I could go on forever.... it's embarrassing and depressing that this could happen to us.
As Vinay correctly pointed out yesterday [16 ], we simply decided to only be skeptics and demand rigorous evidence for some things and not all things (and it turns out the things we give a pass are politically favorable). If someone shares a paper on Ivermectin efficacy, suddenly scientists are concerned with sample sizes, confidence intervals, pre-registration, p values and study design.
Yet a study on School Masking [17] where the authors aren't even aware some of the experimental groups were part of the control group [18], lobbied for the results they were seeking [19], failed to explain why the experimental group was already having a higher incidence rate before the intervention [20], and dismissed criticism they missed confounders by an appeal to authority [21] gets published in the NEJM.
It's surreal.
Citations:
____________________
[1] https://www.newsweek.com/fauci-said-masks-not-really-effective-keeping-out-virus-email-reveals-1596703
[2] https://www.behindheadlines.org/post/initial-coverage-on-mask-effectiveness
[3] Here, vox wrote this article "Six things to do to get ready", but stealthy reduce the list to 5 things (forgetting the headline still said 6) https://www.vox.com/future-perfect/2020/2/28/21156128/coronavirus-prepare-outbreak-covid19-health but the internet archived it https://web.archive.org/web/20200331101148/https://www.vox.com/future-perfect/2020/2/28/21156128/coronavirus-prepare-outbreak-covid19-health
[4] https://www.cidrap.umn.edu/news-perspective/2020/07/commentary-my-views-cloth-face-coverings-public-preventing-covid-19
[5] https://blogs.cdc.gov/niosh-science-blog/2009/10/14/n95/
[6] https://www.cidrap.umn.edu/covid-19/commentary-masks-all-covid-19-not-based-sound-data
[7] https://dph.illinois.gov/content/dam/soi/en/web/idph/files/covid19/covid-19-maskprotection2.pdf
https://www.albertahealthservices.ca/assets/info/ppih/if-ppih-covid-19-my-mask-your-mask.pdf
[8] https://www.facebook.com/watch/?v=449594609641575
[9] – "I have a foreboding of an America in my children’s or grandchildren’s time – when the United States is service and information economy; when nearly all the key manufacturing industries have slipped away to other countries; when awesome technological powers are in the hand of a very few, and no one representing the public interest can even grasp the issues; when the people have lost the ability to set their own agendas or knowledgeably question those in authority; when, clutching our crystals and nervously consulting our horoscope’s, our critical faculties in decline, unable to distinguish between what feels good and what’s true, we slide, almost without noticing, back into superstition and darkness."
-The Demon-Haunted World, 1996
[10] https://my.clevelandclinic.org/departments/wellness/integrative/treatments-services/reiki
[11] https://www.pnas.org/content/117/26/14857
[12] https://roadtolarissa.com/regression-discontinuity/
[13] https://web.archive.org/web/20200702090523/https://metrics.stanford.edu/sites/g/files/sbiybj13936/f/files/pnas_loe_061820_v3.pdf
[14] https://ncrc.jhsph.edu/research/identifying-airborne-transmission-as-the-dominant-route-for-the-spread-of-covid-19/
[15] https://www.washingtonpost.com/world/2021/09/01/masks-study-covid-bangladesh/
[16]
[17] https://www.nejm.org/doi/full/10.1056/NEJMoa2211029
[18] https://www.cbsnews.com/boston/news/massachusetts-schools-mask-mandate-lifted-list-dese/ (compare to their list, 12 of the schools they thought were masked actually had waivers because of vaccination rates)
[19] https://twitter.com/EpiEllie/status/1429102872470433795
and https://www.bostonglobe.com/2022/02/11/opinion/its-too-soon-lift-school-mask-mandate/
[20] Just look at the rates prior to mandate lifting, schools which would remove masks already had a higher rate of infection which shows there was a confounder before study period. If someone presented thist to me I would have pointed out they data drudged the result but moving the start date as close as possible to avoid confronting the confounder.
[21] https://twitter.com/EpiEllie/status/1557497452781096960?s=20&t=20X-EaQtKJAw3a0mwTzSTg
Thanks for the references and the helpful explanation. I think what you're saying is consensus doesn't seem like a consensus to me as I still see a lot of controversy even prior to the pandemic. It raises the question: How do you define consensus? Can you give another example besides masks? I like the example of hydroxychloroquine - initially there was uncertainty and lots of physicians (myself included) prescribed it. But as more evidence came out, there began to be clarity - and consensus - around the fact that any benefit seemed minimal, and there was reasonable potential for harm.
David, Thanks for actually reading my lengthy reply :)
Will respond to you question tomorrow - thanks
Excellent, informative, and succinct post. Thank you.
Great post but the data must be translated to action
Question: You are getting on a plane holding a an n95 mask in your hand wracked with indecision and trying to decide what to do. In deed you have moderate to high uncertainty about the evidence but you must make a behavioral choice. You find your seat and note a big bloke already sitting in the adjoining seat who is slobbering and sneezing into his tee shirt.
{POLLING QUESTION
do you put your n95 on or not ?
YES
NO
PS ( would love to know what Dr Prasad would do? ---[but doubt he is packing n95s)
No. The flight already sounds like it going to be miserable. Why make it worse wearing a mask? If you were going to get sick, you were going to get sick.
Best to just ask the flight attendant for a double gin on the rocks.
This post had sizzle. Terse but yet had punch. Lucidly written for those of us without Yale diplomas on the wall or 67-foot yachts down at our local marina. I return now to my bowl of oat cereal with skim milk, some decaf coffee, and the ingestion of daily drugs: angiotensin receptor blockade agent and homeopathic doses of a statin and beta-blocker.
“homeopathic doses of a statin and beta blocker”
Under the standard homeopathic dilution process (30x 100:1), the likelihood of even a single molecule of the target agent being present is zero. Surely that’s not what you meant.