A Letter to Health Journalism Students

Drs. Alderighi and Rasoini are back with some tips for budding health journalists. Their short essay helps everyone learn key tips and tricks of reading observational studies

Dear Journalism Student,

Most of the information that people get from health studies comes from what journalists write about them. It is important work.

Both doctors and journalists need critical skills for appraising health research. Yet earning these skills is given less room than deserved in both medical and journalism schools.

We write to you from the perspective of doctors who use critical appraisal of medical research to provide people with reliable information and help them make well-informed health choices.

In this post, we share some tips for assessing an observational study and writing a news article about it.

As a case study, we chose a paper entitled “Long-term neurological outcomes of COVID-19,” recently published in Nature Medicine.

This was a retrospective cohort study aimed at characterizing the post-acute (1–12 months) neurological outcomes of COVID-19. Researchers extracted data from the US Department of Veteran Affairs databases and compared the neurological outcomes of 154,068 individuals with previous COVID-19 diagnosis against two control cohorts without previous COVID-19 diagnosis; one control group included ≈ 5.6 million (contemporary controls) and another included ≈ 5.8 million (historical controls) individuals.

The authors observed that people with previous COVID-19 had higher rates of several neurological ailments (e.g., memory disorders, strokes, seizures). Rates of neurological ailments paralleled COVID-19 severity.

Tips:

Mind causality

The chief limitation of observational studies is the difficulty of inferring causation from associations between variables. Consider the observation that people with previous COVID-19 had more neurological outcomes than people without previous COVID-19. For some individuals, this association could be causal. But there are other explanations as well. For example, people diagnosed with COVID-19 may have undergone more testing and, consequently, may have received more diagnoses.

Alternatively, we may hypothesize that people who caught COVID-19 were more vulnerable for reasons that the study didn’t capture, and these vulnerabilities might have predisposed them to develop both COVID-19 and other illnesses.

To protect readers from the causality trap, you could plainly communicate that this study is observational; therefore, we cannot be sure if the associations identified are causal. Writers should avoid causal language.

Be prudent with recommendations

Experts often recommend medical practices based on findings from observational research. However, if you can’t be sure that an exposure causes an outcome, you should refrain from making recommendations.

Journalists tend to report the recommendations they find in study papers, press releases, or expert interviews.

Our general advice: Be prudent about all of these. Recommending actions can have a range of adverse consequences for people if the recommendation is incorrect. For example, people shouldn’t be advised to check their neurological state if they have had no symptoms after SARS-CoV-2 infection.

Ask basic questions about the study

“What are the study patients like?” In our case study, all patients were veterans, and the majority were male, white, in their 60s, unvaccinated, and infected in 2020 by alpha or delta variants. Would the study results be generalizable to a Black, fully vaccinated, 35-year-old woman who catches omicron in late 2022?  (Editors’ note: this is called external validity or generalizability, and it is a super-important issue—in all studies.)

“What is the comparison group like?” In this case, a cohort with previous COVID-19 was compared with two cohorts without COVID-19 diagnosis. One could ask, why not use a comparison group who had another respiratory viral disease? Without such a comparison, we remain uncertain about the possibility that COVID-19 and other viral diseases are associated with similar outcomes.

Focus on context

“Are there other studies on the same topic?”  In this case, previous studies observed increased rates of neurological outcomes after COVID-19, even though most of these studies had limitations, e.g., retrospective designs, limited to hospitalized people, narrow selection of neurologic outcomes, or absence of control groups.

Also, one prospective cohort investigated some sequelae of COVID-19 up to one year after infection by comparing people with and without previous COVID-19. Unlike our case study, the assessment of cognitive, emotional, motor, and sensory functions found no difference between the two groups except for generalized anxiety disorder, which was more frequent in people with previous COVID-19.

Reporting the context of a study—including discrepancies in the existing literature—allows readers to take a glimpse of the whole (but uncertain) picture instead of jumping to premature conclusions. 

Give voice to different viewpoints

Most news articles covering medical studies quote experts, generally the studies’ authors. But there’s a catch. Authors generally emphasize the most appealing sides of their work because it’s something they believe in and because future research funding may depend on their publication’s impact.

If you quote only the study authors, your readers won’t generally be served the whole picture.  

The good news is that, whenever a mainstream study is published, it’s a matter of hours until discussion takes off on social media. Our advice: Get involved in the discussion and include quotes from credible, though different, viewpoints in your article. That will provide balance and offer a broader view of your article.    

Avoid exaggerated and emotional writing

While writing your article, you should stay away from spin, which is a way of reporting news that exaggerates a study’s findings. In our case study, for example, people who had mild COVID-19 had much lower rates of neurological outcomes than those hospitalized for COVID-19 or admitted to intensive care. This gradient should be clearly reported.

You should also avoid fear mongering, which is the use of terms or images that evoke concern or alarm in readers. Fear mongering may help gain clicks, but it won’t help readers better understand the study findings. Instead, it could distort the way results reach them.

In the end, we want to highlight that critically reporting on a paper about the sequelae of COVID-19 does not make you someone who downplays the existence of these sequelae. This simply means that you feel responsible for your medical reporting and are aware of the effects it can have on the audience.

Don’t forget that, among your readers, there are people suffering from COVID-19 sequelae. Since there are still so many unanswered questions, we—healthcare professionals and journalists—owe people prudent and realistic information.

We have included a Table of the important points