Paxlovid, Long COVID, Preprints and Churnalism

My last churnalism post discussed NPR’s coverage of the Sister Study’s finding that there is a association between hair straightening and endometrial cancer. Some of our sharp readers took me to task for my dedication to “left of center” media and even pointed out my own media blind spot. Although I promise future posts about stories from the Murdoch Empire, I have to discuss a story from last week’s New York Times.

As a reminder, we define churnalism as the careless, incurious reporting of poorly done biomedical research. Churnalism trades the real story -- why a study is unimportant or proves something other than it contends -- for the easy headline. There are seven deadly sins of churnalism, outlined in an early Sensible Medicine post.

Last week, The New York Times published an article “Paxlovid May Reduce Risk of Long Covid in Eligible Patients, Study Finds.” I was not optimistic after hearing about this article. The benefit of Paxlovid is a moving target, long COVID remains a poorly defined entity, and I had not seen this article published in a major journal. After reading it, I think the Times piece qualifies as churnalism.

Paxlovid

Paxolovid is a combination of two antivirals, nirmatrelvir and ritonavir. The combination is effective against COVID-19. Paxlovid, though not approaching the benefit of the initial COVID vaccine, was an important advance. Given our record with antiviral medications, I thought it would take years to develop one effective against COVID. I was wrong about this. There have been two studies, both published in the NEJM, that give a good sense of the efficacy of Paxlovid.

First, there is the initial phase 2/3 randomized controlled trial examining the efficacy of Paxlovid published in February 2022. The study randomized 2246 people who were symptomatic with COVID 19, unvaccinated, not hospitalized, and at high risk for complications from the infection. The endpoint in the study was Covid-19–related hospitalization or death from any cause in the 28 days after randomization.

The study showed that, in this population and at this point in the pandemic, Paxlovid was quite effective. Patients assigned to Paxlovid were 6.3 % less likely to be hospitalized or die in the next 28 days than those assigned to placebo. The incidence of hospitalization or death by day 28 was 0.77% (with no deaths) in the Paxlovid group vs. 7.01% (with seven deaths) in the placebo group.[i]

Because this study looked at unvaccinated people and earlier strains of the virus, data that is more recent was necessary. The second study worth noting is an observational study that studied how Paxlovid performed during the omicron wave. Patients in this study were Israelis, over 40 years old, at high risk for progression to severe COVID. Unlike the RCT, most of these patients had some level of immunity (through prior infection or vaccination) – 90% in the Paxlovid treated group vs. 78% in the untreated group. (Like the study covered in the Times article discussed below, you can already see issues with confounding. Patients who received Paxlovid were more likely to have been vaccinated).

In this study, Paxlovid was effective in patients 65 and older. The rate of hospitalization due to Covid-19 was reduced by 73% (adjusted hazard ratio, 0.27; 0.15 to 0.49). The adjusted hazard ratio for death due to Covid-19 was 0.21 (0.05 to 0.82). There was no benefit in patients 40 to 64 years of age. The absolute benefits were much lower in this study than in the RCT. Whereas in the initial study, the absolute benefit was 6.3%, in this study, limiting ourselves to the 65 and older group, the absolute benefit was 1.9%. And remember, this was an observational study, patients who took Paxlovid were different from those who did not, and residual confounding is likely.

The Study Discussed in the New York Times

This study that the Times focused on, Nirmatrelvir and the Risk of Post-Acute Sequelae of COVID-19, appeared as a preprint (which, despite Dr. Mandrola’s brilliant recent post, has to make us cautious as it have not been peer reviewed). The observational study compared patients treated with Paxlovid to those not treated. All patients had a positive COVID test between March 01, 2022 and June 30, 2022, were not hospitalized on the day of the positive test, had at least one risk factor for progression to severe COVID-19 illness, and survived the first 30 days after SARS-CoV-2 diagnosis. All the patients were cared for by the US Department of Veterans Affairs. The authors compared the rate of a prespecified panel of 12 “post-acute COVID-19 outcomes” at 90 days.

The results of the study: Paxlovid use was associated with reduced risk of post-acute COVID outcomes (HR 0.74; 0.69 - 0.81) with an absolute risk reduction of 2.32%. A negative association was found for 10 of the 12 prespecified outcomes.

Paxlovid may, in fact, reduce the risk of these post COVID outcomes. I, honestly, do not know at this point. I, personally, do prescribe Paxlovid to my older and high-risk patients. But, there are reasons be cautious in accepting these results.

1.      This is an observational study. The cause of the difference in the outcomes cannot be assumed to be Paxlovid use.

2.      We cannot make this assumption because of residual confounding. The patients who took Paxlovid were different from those who did not (see table 1 in the supplementary material). The authors did adjust for these differences (table 1 in the article) but, as is always the case, residual confounding is likely.

3.      Being a VA population, the population was overwhelmingly male.

4.      “Long COVID” remains a poorly defined entity. This article did not look at the prevention of Long COVID. It looked at the incidence of 12 symptoms often considered part of the Long COVID syndrome.

5.      Post-acute COVID symptoms were not actually measured. Instead, the authors used ICD-10 codes logged into the VA system.

6.      There was no accounting for care outside the VA, which may have included Paxlovid treatment or care for post-acute symptoms.

The New York Times Article

The Times article did an inadequate job identifying the uncertainties that exist in research into Long COVID. The article certainly did not identify uncertainties that exist despite the findings in this research. The article did do two things well.

1. The author noted that this article is a preprint that has not undergone peer review.

2. The Times article also pointed out the lack of diversity in the population.

I found four important shortcomings

1. There was no reference to this being an observational study that cannot prove causation.

2. The author speaks to only two sources for the article. One, Dr. Michael Peluso, an assistant professor of medicine at UCSF, seems supportive of the findings. The other is the senior author of the paper, Ziyad Al-Aly.

3. The only skepticism that comes up in the article is when Dr. Peluso notes that Paxlovid “did not completely eliminate post-Covid conditions.”

4. There was no discussion about the relationship between the 12 post-acute COVID-19 outcomes studied and Long COVID – of which the article tells us Paxlovid reduces the risk. Nor was there discussion of how the presence of these outcomes were identified; this, as discussed above, is likely a major flaw in this study.

5. A fifth is not terribly important but bugged me. A full three paragraphs were spent discussing bioplausibility. Bioplausibility is important. We don’t test things that are not plausible. We consider plausibility when we weight suprising results - it accounts for our “pre-study probability.” But here, it is used as an argument about why we should accept these results, taking up space that could have been used to consider alternative interpretations.

With these five shortcomings, three of the churnalism sins were committed:

1. Observational studies almost never prove causation

3. Ignoring confounding, selection bias and other epidemiological errors

7. Being incurious

The Remedy

I’d suggest 3 questions that would have made this report health journalism rather than churnalism:

This was an observational study, which cannot prove causation. You found an association between Paxlovid use and a reduction in 10 of 12 “post acute COVID-19 outcomes.” What other explanations are there for your findings?

In this study, you looked at what you call “post acute COVID-19 outcomes” – essentially 12 diagnosis codes used for patients between 30 and 90 days after they had COVID. What do you see as the relationship between this outcome and Long COVID?

In the data we have thus far, Paxlovid seems to be losing efficacy as the population becomes more immune to COVID. Do you expect this trend to continue and, if it does, how would this affect your findings?

---

[i] This ink on this study was hardly dry when physicians began to extrapolate the results and prescribe the drug to low risk patients who had either been previously infected or vaccinated.