When a friend makes a mistake, it is your responsibility to let him know. In the words of the Ad Council and the US Department of Transportation, “friends don’t let friends drive drunk.” They also “don’t let friends misinterpret the medical literature because they have been blinded by their priors.” Friends do this even if the results of the paper in question should change nobody’s behavior and if calling them out will invite the wrath of Sensible Medicine readers.
The paper in question, Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial, was published in the BMJ.1 Vinay authored a criticism of the paper on Sensible Medicine. Before we get to our disagreement, a quick review of the article.
The Study
Design
The objective of the study was to evaluate whether the use of masks, worn in public spaces for 2 weeks during late winter and early spring in Norway, would reduce self-reported respiratory symptoms. The authors randomized 4647 adults to use of a surgical face mask when they were in shopping centers, streets, public transports, and the like, or to not wear a mask.2
Results
The mean age of the participants was 51 and 60.9% were women. The two groups were well matched. The intervention group were probably more exposed to children (28.5% had children in the house vs. 25.5% in the control group). As you can see in the table below, the participants generally believed that masks were effective but did not wear them. (The intervention group is on the left, control on the right).
The primary analysis was intention to treat. In the intervention group, 8.9% of people developed symptoms consistent with respiratory infection. In the control group, 12.2% developed symptoms. The marginal odds ratio was 0.71 (95% CI 0.58 to 0.87; P=0.001). The absolute risk difference was −3.2% (95% CI −5.2% to −1.3%; P<0.001).
Commentary
The easy interpretation of this study is: masks work to decrease respiratory infections during cold and flu season in Norway. The endpoint is a good clinical one that we care about: “Do you feel sick?” Sure, it is not hospitalization or death but that would require a giant study.
Does this change anything I would personally do or recommend to my patients? Probably not. The cost of wearing a mask for two weeks, for me personally, is higher than the benefit of a 3% reduction in the risk of getting a cold. If someone had something that they absolutely had to be well for in two weeks – a wedding, the trip of a lifetime, the 100 M butterfly at the Paris Olympics – might I recommend it? Probably not, but at least I can now give some numbers.
I expect Dr. Prasad would agree with me here.
Where we disagree is that he seems to take nothing from this trial. After graciously complimenting the authors and thanking them for doing an RCT, he goes on to dismiss the results:
Given, that many people believe masks work, and given that there is no placebo control or sham control in this study, it is entirely possible the result is driven by the placebo effect alone. If you wear a mask, you feel like you do better.
This is true, but remember, these were not committed mask wearers. To get these results every participant who usually wore a mask would need to imagine they got sick during these two weeks. And also, these are people who voluntarily entered a study in which there was a 50% chance of having their masks removed. Also, it is not just “feel better,” it is actually develop symptoms of an upper respiratory infection.3
Two other things to point out from Dr. Prasad’s post:
Given, that many people believe masks work, and given that there is no placebo control or sham control in this study...
A better endpoint for future mask studies would be some objective measure of viral spread, like serial swabbing, or sero-prevalence.
What troubles me here is the proverbial moving of the goalpost. The data on masking thus far has been pretty underwhelming – both in design and in results. Many people have had clinical experiences that put them in the anti-Vinay camp, believing that masks, although far from a panacea, have some utility.4 We have asked for RCTs of masking, now we have a good one, and Vinay raises the bar. Sham masking Vinay, really? Serial swabbing? Who cares if a swab is positive if you feel well or negative if you feel sick. I should also note that Dr. Prasad made exactly the opposite argument on page 137-138 of Ending Medical Reversal when we discussed data supporting Tamiflu.5
I do think this is a good example of how our priors affect our interpretation of the medical literature. Vinay and I have discussed (argued about?) our different takes on masking to prevent the spread of respiratory viruses on the podcast. The TLDR version is he thinks it is nearly worthless and I think that “worthless” is just a little too strong.6 If you go into this article convinced that masking is worthless, you can dismiss the results as Vinay did. If you go in thinking that masking probably provides a very small benefit, this article supports your view: a mask is not an impenetrable shield but might decrease the chance that you will feel sick — by about 3%, especially if you are Norwegian, in late winter, and think masking reduces your risk of getting sick “to some extent”, but can’t be bothered to mask.
Oh wait, I need to add an ad hominem attack. I do not like Vinay’s black Kurt Cobain t-shirt.
And, a little further reading. This post reminded me about a good previous one from a contributor.
You’ll notice that I actually link to the paper here. Something not done by my colleague. Suspicious…
I refuse to spell randomize, randomize, even though this paper was published in the BMJ. I don’t know what the Brits are thinking mucking with our language.
I do wish this was better defined in the study.
Yeah, I know, not a perfect parallel but I am making an argument here.
See reference 4. Speaking only for myself, though I think we agree, I feel that community mask mandates are, at best, counterproductive. Masking in healthcare facilities has been beaten to death on this site.
Both Prasad AND Cifu are wrong: this study tells us nothing about masks. It should not change anyone’s opinions about masking. We should use this study as a case of how not to design medical trials.
1. The treatment is not wearing mask. The treatment is an emailing someone and asking them to wear a mask and providing them an online gift card to buy a 50 pack of masks.
2. The control is emailing someone and explicitly telling them not to mask.
3. The outcome is not illness. The outcome is a complex formula gleaned from an online symptom survey that some participants filled out 14 days later.
4. The obsession with the “3% ARR” is bad trial analysis. Only 80% of the treatment group filled out the online survey, while 87% of the control group did. So apparently, we should conclude that masks cause a 7% reduction in people’s willingness to fill out online surveys. It’s the only sensible conclusion, right?
I just can’t get over that we are discussing a trial where the study staff never had in person contact with any participants. We have zero evidence that anyone ever wore a mask! We just have emails. Replace “mask” with any other intervention and you will see that this trial only induces arguing among partisans.
I am not an epidemiologist, much less an expert in public health (in fact, I am a psychiatrist), but I am completely amazed every time I read these studies and the controversies that follow. No exception is this latest discussion. Where do you live, dear friends and colleagues? On the moon or on some remote atoll in the Pacific? But do you not realize that not one, I repeat, not one, of the arguments you use stands up to careful scrutiny in the light of real life? How many people wear their masks properly? How many change them with the necessary regularity? How many avoid touching it with unclean hands? How many follow the rules for proper use of these protective devices, especially among children, young people, and the elderly? My answer, simply by observation of the people around me, in everyday life, in airports, in trains, in the grocery store, is: hardly anyone. So if an observational study finds some benefit from using the masks, I do not doubt that it is true -- the researchers, like Brutus, are all honorable men -- but I conclude that it will be due to the placebo effect (as Vinay Presad suspects) or perhaps astral influence, if not voodoo magic. However, I feel certain to rule out that it is due to the protective barrier of the masks. Thank you for your attention and excuse the long comment.