The Cochrane Mask Fiasco
A Cochrane Report Tells the Truth, But Many Are not Ready to Hear It
Recently, Cochrane & the internet has blown up over the Physical Interventions to Interrupt or Reduce the Spread of Respiratory Viruses report — specifically the masks portion— and it has already become another riving chapter in the EBM history books. Here, I walk you through the play by play, and leave you with a 5 take home lessons.
The latest update of the Cochrane Report came out on Jan 30, 2023. The first author is Tom Jefferson, who has worked on this protocol since 2006 (remember… 2006). Cochrane, once again, examined all the randomized trials of masking, and came to 2 main conclusions. Let me quote them.
“Compared with wearing no mask in the community studies only, wearing a mask may make little to no difference in how many people caught a flu-like illness/COVID-like illness (9 studies; 276,917 people); and probably makes little or no difference in how many people have flu/COVID confirmed by a laboratory test“
And
“More high‐quality RCTs are needed to evaluate the most effective strategies to implement successful physical interventions in practice, both on a small scale and at a population level. It is very unfortunate that more rigorous planning, effort and funding was not provided during the current COVID‐19 pandemic towards high‐quality RCTs of the basic public health measures.”
(emphasis mine)
In my words:
the best data fails to show a benefit, and
we needed more trials. Too bad you all (CDC, WHO, and NIAID) f*&D that up.
If that sounds familiar, it is because I said that: here and here and here and here and here, pretty much all damn pandemic, I was a broken record and said just that, over and over.
How did I come to the same conclusion as Cochrane?
Because anyone worth their salt in EBM would reach that same conclusion. It is basic critical appraisal and reading. I even published my own review, which is one of those links above.
After the report came out, the criticism was immediate. Mask advocates claimed that the report did not prove masks don’t work, but just failed to find benefit. They said a large confidence interval meant that masks could work. The absence of evidence is not evidence of absence.
The problem with this argument was that this is a new standard— just for masks. I pointed out that medicine often abandons practices if RCTs fail to find benefit, even as the lower bound of the 95% CI is compatible with values that would constitute benefit. I show that here and here. Using 2 disparate examples.
Tom Jefferson himself gave an interview. In that interview he dropped this bombshell:
“In early 2020, when the pandemic was ramping up, we had just updated our Cochrane review ready to publish…but Cochrane held it up for 7 months before it was finally published in November 2020.” He added:
In other words, it is clear, Cochrane leadership was meddling with the Cochrane review. The unspoken message was clear: They did not want people saying masks lack evidence.
Enter Zeynep Tufecki. Tufecki is a sociologist who wrote an op-ed in NYTimes in first quarter 2020 and is credited with getting the CDC to push masks on the American public— without randomized evidence— including in children. (See timeline here).
She was working on a more recent NyTimes op ed with a ridiculous title, “Here’s Why the Science Is Clear That Masks Work,” and contacted Cochrane for a statement.
About Jefferson, she quoted Cochrane’s Editor in Chief, ““Soares-Weiser also said, though, that one of the lead authors of the review even more seriously misinterpreted its finding on masks by saying in an interview that it proved “there is just no evidence that they make any difference.” In fact, Soares-Weiser said, “that statement is not an accurate representation of what the review found.””
But, Soares-Weiser is wrong. There is ‘just no evidence they make a difference’ is the truth and correct statement of Tom Jefferson. (PS this man also did the TamiFlu analysis— he is an EBM legend).
Tufecki went on to cite a bunch of low credibility studies to support masks, but this is embarrassing. It is like citing mouse model research that a drug should work after the pooled analysis of randomized, phase 3 trials is negative.
Now, Tom Jefferson is back. In his substack he writes:
Jefferson gave another interview and said,
“he was given “little workable notice” before Cochrane published its statement and the authors had no clue what the statement would entail, prior to its publication.
“It was upsetting,” said Jefferson. “Cochrane has thrown its own researchers under the bus again. The apology issued by Cochrane is from Soares-Weiser, not from the authors of the review.””
So what are my take away conclusions?
Cochrane’s EIC should be fired. If a NYTimes writer calls you about a Cochrane review, and what the lead author may have said in the media, you stay silent. You don’t throw them under the bus, especially when the author was factually correct. It is bad leadership, and is the latest example of ‘admins’ who lack the courage to stand up against the mob or what they perceive as a mob. Stand by your people, or resign. I am embarrassed by cowardly ‘leaders’
Tom Jefferson signed up to study physical interventions to study respiratory viruses in 2006! Every sentence of the Cochrane review is correct, and I have never witnessed him ever make an error in data interpretation. That’s not true of others. Jefferson is old school EBM. Jefferson is just saying what Fauci said initial on 60 mins (aka the truth). Jefferson is real hero for his life’s work, including Tamiflu. The problem is people don’t want to hear the truth on this issue.
Yes, we should have done more RCTs of masking during the pandemic. Lord knows, I shouted that every chance I got. But folks like Tufecki were the problem. Pro mask advocates authored grandiose statements and basically ran a propaganda campaign stressing the certainty masks worked. They needed certainty to force the deplorables to mask. Yet, this false certainty prevented the conduct of RCTs. Even to this date the CDC director Walensky thinks she lacks equipoise for such a study. This very quarrel is the proof of the equipoise she denies. PS Tracy Hoeg and I have a new paper on this topic, read it here.
You can’t have a double standard for evidence. Masks and ivermectin both have no evidence to justify them, and both “don’t work” until that evidence is marshaled. You can’t say ivermectin doesn’t work, but masks might. Prove how it works, and until then: STFU.
People who want to disparage Tom Jefferson and argue he has some agenda besides a pursue of truth have to note he launched this in 2006. Instead, everyone else is Johnny Come Lately. My interest in this topic is only that I can read, and apply principles of EBM, and I have never witnessed so many smart people fall victim to cognitive traps until masks. That psychological weakness is a huge vulnerability for all EBM issues.
In conclusion, Cochrane’s EIC has disgraced herself, mask advocated sabotaged the conduct of RCTs now claim they wanted more, and Tom Jefferson might be one of the last principled scientists around. Now read the actual review and learn some EBM.
.
I read the review. I doubt my hospital's infection control department has. They are still auditing (spying) us for mask compliance. Everyone 2 and up must mask at our hospitals and clinics. It's a disgrace, especially the little kids.
From a policy perspective, both the effect size and the level of certainty that would be required for it to be appropriate to even consider mandatory masking are very high. The question isn't whether you can cherry pick an outcome and get p=.049; it's whether you can demonstrate a number needed to prevent for serious clinical outcomes that substantiates government action.
Which is to say, there's really no amount of massaging that can make this review reach that bar, so the continued efforts to do so are out of bad faith or incompetence.