Children, Evidence-based Medicine and the Windmills
Drs Camilla Alderighi and Raffaele Rasoini write about an important project to improve health literacy.
Sensible Medicine is pleased to present this work from the wife and husband team of Camilla Alderighi and Raffaele Rasoini. I met them in Rome in January of 2020 when I was there to speak.
Camilla and Raffaele are general cardiologists based in Florence, Italy. They represent the Informed Health Choices Project in Italy.
What follows is more than a report on their attempts to bring health literacy to young minds, but also a temptation to be optimistic. Optimistic because if Giacomo, the fifth grader they tell us about, can understand dubious evidence, imagine what is possible on a grand scale.
I love their work because part of improving the state of critical appraisal of evidence is broadcasting it to a more informed audience. John Mandrola
“Some researchers studied a drug named hydroxychloroquine against COVID-19. They selected 26 patients who were in hospital with COVID-19 and gave them this drug. Another 16 COVID-19 patients who refused to take the drug served as controls. After a few days, 70% of the patients receiving the drug had virus-free throat swabs, compared to 12.5% of the not treated patients. What would you say about this study?”
“I would say it doesn’t look like a fair comparison at all: assignment to groups wasn’t done by chance, since folks chose whether they would take the drug or not. This means the two groups are not similar, and the results may be unreliable. Plus, too few people were involved in this study.”
This answer was given by Giacomo, a fifth-grade student from Florence, Italy. We posed this question to him as part of a final test in a critical health literacy project called Informed Health Choices. (Screenshot of the website below.)
It was June 2020, and we were engaged in online discussions with 10–11 years old students about the pitfalls of a French study on the use of hydroxychloroquine for COVID-19.
Critical health literacy is a set of skills that enable people to critically appraise information relevant to health. The past few years have put both the lack of these skills and their value in the spotlight.
The pandemic has also initiated an anxious infodemic of studies unsuitable for use in making well-informed health choices. Yet a well-informed layperson would be able to dismiss low-value research and even decline a recommended treatment if they believe it is not supported by reliable evidence.
Several years ago, the British researcher Iain Chalmers said, “You don’t need to be a scientist to think critically and ask good questions.” Along with Andrew Oxman, they studied ways to convey the key concepts of evidence-based medicine to the public.
They quickly realized how hard it was to teach health literacy to adults. Unlike children, adults have limited time, many prejudices, and entrenched narratives that make most of them impervious to new concepts, especially when old concepts feel jeopardized.
Chalmers and Oxman felt they needed to start early on to teach people how to think critically about health claims and health choices, and they did so.
After some pilot experiences, they decided to investigate the effects of teaching a set of key concepts of critical health literacy to primary school children in Uganda, a low-income country.
The idea might sound quixotic, but it was not.
They used a cluster randomized controlled trial (RCT), a study design suited to rigorously test differences in complex environments, such as in education or public health.
In 2016, Oxman, Chalmers, and other researchers randomly assigned 120 primary schools in Uganda (comprising more than 10,000 children) to traditional learning or to lessons aided by a comic-based textbook covering several key concepts of critical health literacy.
For example, children were taught that “in fair comparisons people should not know which treatment they get” or that “fair comparisons with few people or outcome events can be misleading.”
The trial, published in The Lancet, found that 10- to 12-year-old children who were taught with these key concepts developed a better critical and decisional attitude about health claims and treatments than those who did not use these learning resources.
Raffeale and I believe that we should draw inspiration from that approach—for instance, we think it possible to design and conduct cluster RCTs on non-drug interventions for COVID-19 (e.g., behavioral interventions such as facemasks) that have the potential to affect entire countries.
The low number of RCTs on non-drug interventions for COVID-19 conducted up to date is embarrassing. On the BESSI Collaboration website, only 18 RCTs on non-drug treatments have been registered so far.
While some adults are choosing to embrace non-evidence-based attitudes by prohibiting the in-person school attendance of children who do not follow masking guidelines, a basic questions remains: do the net benefits of masking kids in school outweigh the downsides?
In sum, we believe in teaching children the key concepts of evidence-based medicine. With this foundation, they might have an easier time approaching it as adults, when it comes time to make their own health decisions and participate in public health debates.
Let’s just hope that we don’t come across too many windmills along the way.
We’ve joined our colleagues at Sensible Medicine because we share the aim to provide people with medical information that has not been influenced by other interests, which then fosters a more neutral approach to the truth.
On Twitter, Camilla is @camialderighi and Raffaele is @RasoiniR
Here was my critical thought process as a patient.
Ivermectin and HCQ have been prescribed safely over billions of doses and have an extremely high safety profile. Even if they don’t work, they won’t harm me.
So that’s why I took ivermectin when I got Covid. Did it help? I basically never had a fever and had the equivalent of a head cold for about a week. Maybe it did nothing. Maybe it was placebo. Maybe it helped. It definitely didn’t harm me though.
I have consistently been wary of all new drugs and medical technology until it is proven over a period of years and not withdrawn from the market or hit with a black box warning. Long before the pandemic came along.
This is why I was afraid of the mRNA vaccines and did not take them, although I have nothing against people who chose to, like my spouse.
I still feel that I made the most reasonable choice. Could I have been an unlucky one who died from Covid? Yes, and I accepted that risk and felt it was lower than the risks of adverse reactions taking new medicines like mRNA or paxlovid. If I had been wrong, I would have accepted that I miscalculated the risk and admitted I made a terrible error and dealt with the health consequences.
The kids did a great job of evaluating that hydroxychloroquine study. As a follow-up, the kids should be presented with a second study where hydroxychloroquine was utilized in the manner in which it has been found to be efficacious, ie. at the initial onset of symptoms. The students should understand that studies are also published where it is clearly the goal of the trial designers for the drug to fail.