Yet Another Story on Re-Analyzing a Major Trial
The Study of the Week once again underscores the value of sharing medical trial data for re-analysis
A group of authors, who are part of an international effort called RIAT, (Restoring Invisible and Abandoned Trials) have published a re-analysis of a major trial of a cholesterol-lowering drug.
Their findings have been called both important and not real science. You are interested now, I hope.
The story begins in 2017, when the New England Journal of Medicine published the FOURIER trial comparing a drug called evolocumab vs placebo in more than 27,000 patients who already had established heart disease and a persistently elevated LDL-C despite taking statins.
Evolocumab is an inhibitor of proprotein convertase subtilisin/kexin type 9. Suffice to say these injections markedly reduce levels of cholesterol—which should be important in patients who already have heart disease.
Here is a picture from the FOURIER trial. You can see that the drug markedly reduced LDL-C over placebo
In cardiology, it’s not enough for a drug just to change a level of something in the blood. You need it to improve outcomes.
In FOURIER, the primary outcome was a composite of 5 things: the first event of cardiovascular death, MI, stroke, unstable angina or the need for coronary revascularization (stent or bypass).
FOURIER reported that evolocumab reduced the incidence of the primary composite endpoint by 15%. We write that as a hazard ratio with 95% confidence intervals (HR 0.85 (95% CI, 0.79 to 0.92)). On the surface, this looked like a win for the drug. FDA approved it for use.
There were, however, some curiosities regarding these results: death due to cardiovascular disease and overall death were slightly higher in the patients who took evolocumab. Not (statistically) significantly higher, but I found it curious that a drug that demolished cholesterol levels, reduced the chance of a heart attack, stroke or the need for stents, did not result in less death due to heart disease or less overall death.
Now to the new report from the RIAT group. These authors, one of whom I know personally, got hold of the Clinical Study Reports—so called CSRs, which provide more details than that published in the NEJM.
CSRs are technical documents prepared by the manufacturer and submitted to regulatory agencies as part of the approval. They contain info about the trial’s protocol, amendments, inclusion and exclusion criteria, outcome definitions and measurement, efficacy and safety results and statistical analysis plan. The main FOURIER CSR is over 25 000 pages.
The first story is how these authors got hold of such documents.
They made numerous attempts to contact the authors and got no response. They asked FDA and FDA said it would take years. They eventually got access from Health Canada.
Here were the main findings of the re-analysis:
FOURIER had 870 deaths; 41% of the time the cause of death determined by the clinical events committee differed from that declared by the local investigator.
The CSR info when compared with the 2017 publication in the NEJM, yielded 11 more deaths from MI in the evolocumab group vs 3 less death from MI in the placebo group.
In the CSR, the number of deaths due to cardiac failure in the evolocumab group was almost double those in the placebo group (31 vs 16).
The reported HR for cardiovascular mortality in the original trial analysis was 1.05 (95% CI 0.88 to 1.25). A 5% increase. In the re-analysis, it was now 20% higher in the evolocumab group. Technically, this was not statistically significant because the 95% confidence intervals ranged from 0.95 (a 5% decrease) to 1.51 (a 51%) increase.
The RIAT authors concluded that clinicians should be skeptical about prescribing evolocumab for patients with established atherosclerotic cardiovascular disease.
Comments:
There are two views of this new study.
One is that there is nothing to see. A group of authors who had less information than the trialists looked at the CSR’s and came up with differences in causes of death. And, statistically speaking, the results of the trial were upheld.
Journalist Patrice Wendling has incredible coverage on theHeart.org | Medscape Cardiology website. Primary author of FOURIER, Marc Sabatine, said this of the new paper:
It's hard to call this science. I think it lacks all scientific rigor and is fundamentally flawed and, because their process was flawed, it has led them to erroneous conclusions."
Another view of this matter is that there is a lot more “flux” in the results of a major trial than I would have expected.
If the re-analysis, using CSRs, is correct, then cardiovascular death rates are even higher in the evolocumab group. The higher rate was weird in the main trial, and it’s weirder yet, in the re-analysis.
Finally, a colleague of mine from Egypt, Professor Ahmed Bendary, Tweeted that this might be the biggest story of 2023:
The linked paper reported that mice without the molecule that evolocumab inhibits may be more susceptible to heart failure with normal heart function. It’s an animal study so let’s not over-react. But…
I take two main messages from this story:
Yes, it is possible that that the higher death rates from cardiac disease (in both the main trial and reanalysis) were just noise.
But perhaps it’s because drug effects are complex and cholesterol reduction may be only one effect of these drugs. The higher rates of cardiac failure in the reanalysis does comport with the animal study on PCSK9-deficient mice.
It is really hard to understand how this much cholesterol reduction, and MI and stroke reduction, and reduction in the need for stents or bypass, does not improve survival.
The other message is more wide-ranging—and more important:
Independent and transparent scrutiny of medical literature is crucial—especially when it leads to changes in practice.
Source data should be made available for independent review. Why would the FDA and authors refuse to share this data?
I’ve always held that great results will pass muster in independent analyses.
It shocks me that there is so little independent scrutiny of major medical evidence.
“ Independent and transparent scrutiny of medical literature is crucial—especially when it leads to changes in practice.
Source data should be made available for independent review. Why would the FDA and authors refuse to share this data?”
Amen! Some medical reformers have been sounding this alarm for years. I cannot believe that major studies get published in major journals with almost zero transparency. I understand why a pharma company wants to hide its “massaged” numbers, but why on earth do the journals agree to go along with it? I have to believe that after everything the past couple years there will finally be a stronger push to release full trial data to the public, not have it cherry-picked and filtered through gatekeepers…
Like I wrote the other day, the silver lining of the total loss of public trust is that hopefully it will finally prompt these people to change their ways.
https://gaty.substack.com/p/2023-preview-the-end-of-medicine
Frankly all conclusions will be off the mark when assuming pcsk9 / statins and LDL-C reduction is protective, and frankly this is an area that requires extreme caution and scepticism as i fail to see any evidence since Ancel Keys' fraudulent so called 7 countries study that high blood lipid is a problem at all. Just because they see firemen at every house fire doesn't mean the firemen are the cause, yet here we are still embracing this theory because ... food and drug industry. It stinks John because it's rotten. Yes i am absolutely aligning with such as Dr Malcolm Kendrick, and until I see robust evidence, I continue to be. Keep it up John, love your work and yes it should be obvious we need raw data, complete transparency. The only conclusion one can make is that the aim is to hide data, and negative data at that.