The Solution to Low Power is Not to Add Weak Endpoints
Lessons from the PARACHUTE HF trial in patients with Chagas Cardiomyopathy. You can't replicate PARADIGM HF by simply changing endpoints
JAMA has published the PARACHUTE-HF trial comparing the angiotensin receptor-neprilysin inhibitor (ARNI) sacubitril/valsartan to enalapril in patients with heart failure with reduced ejection fraction (HFrEF) caused by Chagas disease. In Chagas disease, heart failure is a late sequelae from infection with the protozoan parasite Trypanosoma cruzi.
The main lesson today is that you cannot fix power problems with endpoints.
By power, I mean the ability to detect signal from noise. To wit, if you want to tell whether a coin is fair, you need more than 10 tosses. In medical trials that measure clinical events (such as MI, HF, stroke or death), you need enough of them to show differences between two treatment arms.
PARACHUTE-HF authors aimed to recapitulate the 2014 PARADIGM HF trial of sacubitril/valsartan vs enalapril in patients with HFrEF.
On paper, the trials look similar. Both trials tested the same drug combination; both trials enrolled patients with impaired LV systolic function (LVEF 29%), and both trials used elevations of the biomarker plasma B-type natriuretic peptide (BNP) level for study entry.
PARADIGM HF found that sacubitril/valsartan resulted in a statistically robust 20% reduction in the primary endpoint of cardiovascular death (CV) death or hospitalization for HF (HHF) compared to enalapril. Both components of the endpoint, as well as all-cause death, favored the ARNI drug.
While there have been criticisms of PARADIGM HF, including different run-in periods and suboptimal dosing of enalapril, the trial established sacubitril/valsartan as a commonly used drug in patients with HFrEF.
In the PARACHUTE-HF trial, sacubitril/valsartan also fared better than enalapril. The trial was (technically) positive. Should we therefore make sacubitril/valsartan the preferred drug in patients with HF due to Chagas cardiomyopathy?
Before you answer that question, consider that sacubitril/valsartan has no generic and is a costly brand-name drug.
One more background point in the matter of measuring efficacy in patients with HFrEF. The seminal trials of ACE-I, beta-blockers and spironolactone all used one endpoint: death. Those three drug classes were so effective, that new agents could no longer budge the endpoint of death. More recent trials of HFrEF have therefore had to add the endpoint hospitalization for heart failure or HHF.
The Differences in the Two Trials
In PARADIGM HF, more than 8,400 patients were randomized to the two treatment arms. This resulted in about 1000 endpoint events (CV death or HHF) per arm (914 and 1117, to be exact). The 20% reduction (hazard ratio = 0.80) came with 95% confidence intervals ranging from 0.73-0.87 and p-value of < .001. In other words, if there was no difference in the two drugs, finding a 20% lower rate of events would be a very surprising result.
In PARACHUTE-HF, 922 patients were randomized to the two treatment arms. Pause there. That’s a huge difference. The drug dosing is similar; patients are similar too. But PARACHUTE randomizes about 1/5th as many patients. That there are far fewer patients with Chagas disease-related heart failure is expected. Chagas may affect millions of patients—mostly, but not exclusively—in South and Central America, but it is only a tiny fraction of the total population of patients with heart failure.
How do the PARACHUTE-HF investigators deal with the smaller number of patients? They changed the endpoint and the analytic method. Namely, they add an endpoint to the two components CV death and HHF. PARACHUTE-HF also measured a relative change in NT-proBNP from baseline to 12 weeks. BNP is a biomarker that goes up when there is volume overload; it can be useful in patients who present with shortness of breath. If normal in this setting, heart failure is a less likely cause. But BNP is a biomarker, not a clinical event like death or hospitalization.
PARACHUTE-HF authors also changed the analytic method. They used a hierarchical procedure called a win ratio. This is a computer heavy analysis were each patient pair is compared first on CV death, then HHF, then change in BNP. In each patient pair, one outcome is a win, the other is tie. The ties move to HHF, and wins are counted and ties move on to change in BNP.
The upsides of a win ratio are that it maximizes use of the data and it prioritizes important outcomes. The downside of the win ratio is that less important outcomes often dominate the ratio. (As is the case in this trial.)
Here is a picture from the paper.
As above, there is little difference in CV death—wins are 18.3% and 17.5%, sacubitril/valsartan vs enalapril. There is also little difference in wins for HHF—7.7% vs 6.9%. The driver of the “positive” result was wins in change in BNP—22.5% vs 7.2%, respectively. In other words, sacubitril/valsartan was more effective in lowering a biomarker. The total win ratio was 1.52 with 95% confidence intervals of 1.28-1.82).
When we look at the components of the endpoint, there were 110 vs 117 CV death events and 102 vs 111 hospitalizations for heart failure (sac/val vs enalapril). These are essentially the same; hazard ratios were 0.95 and 0.92. To compare the trials, PARADIGM HF had ≈ 1000 endpoints per treatment arm—nearly 10x more than in PARACHUTE.
To the authors credit, or perhaps the peer reviewers or editors, the conclusion in JAMA reads:
Conclusions and Relevance In patients with HF due to Chagas disease, there was no significant difference in clinical outcomes between sacubitril/valsartan and enalapril, but there was a greater reduction in NT-proBNP at 12 weeks in patients in the sacubitril/valsartan group.
Comments and Teaching Points
We want evidence to guide our practice. Trials are the way to gather evidence. The problem in Chagas cardiomyopathy is that there are not enough patients to sort out differences in clinical outcomes with ARNI drugs vs standard ACE-I.
PARADIGM HF showed that you needed about 8000 patients to have confidence in the result.
PARACHUTE-HF studied the two drugs, but there were simply not enough outcome events to determine a difference. The hazard ratio for CV death was 0.95, a 5% lower rate on sacubitril/valsartan, but the 95% confidence intervals went from 0.73 to 1.23. Which means it could be 27% better for CV death or 23% worse. Translation: we don’t know.
I think you’d be correct in asking why such a trial was done. The authors must have known that there were not enough patients to detect clinical differences. They would also know that ARNI drugs are better at lowering BNP, but this is only a biomarker—not an endpoint.
In the statistical plan, they declared that PARACHUTE-HF would be positive if both the win-ratio was positive and the HR for CV death and HHF was less than 1.0. And it was; at 0.91 (95% CI 0.73-1.13).
Yet I have already told you that there were not enough CV death or HHF events to sort out signal from noise. They were actually quite similar in the two groups.
The makers of sacubitril/valsartan (Entresto) Novartis funded the trial. Many of the authors declared conflicts of interest with Novartis. So I can’t help thinking this might be a marketing effort as much as a scientific effort. I hate to think that, because we want the best for patients who have Chagas disease.
The editorialists get it correct when they write:
…we are concerned that a preferential emphasis on ARNI therapy for Chagas cardiomyopathy would be distracting, with cognitive and financial opportunity costs.
Of course this trial is not enough to make ARNI the preferred drug in Chagas cardiomyopathy. The trouble I have is that this was surely known before anyone entered into an experiment.
The lesson for Sensible Medicine readers is that you can’t rescue low power with meaningless endpoints and win ratio analyses.