A Cautionary Tale of Clinical (Mis)translation
Spironolactone use and hyperkalemia hospitalizations following publication of the RALES trial in Canada
On a recent This Fortnight in Medicine podcast, John, Vinay, and I discussed the recent POLY-HF trial where a polypill containing metoprolol succinate, spironolactone, and empagliflozin was compared to enhanced usual care, which involved the use of individual agents for guideline directed medical therapy (GDMT) for patients with systolic heart failure. In my opinion, the trial only applies to a small, highly selected group of heart failure patients, at best, and the results are subject to multiple limitations but I have no doubt that it will be promoted breathlessly and applied inappropriately for all heart failure patients.
This is a reminder that in our zeal to help patients we can do more harm than good by uncritically applying clinical trial results to daily patient care.
The Randomized Aldactone Evaluation Study (RALES) compared spironolactone to placebo in 1,663 patients with heart failure with a left ventricular ejection fraction of </= 35% and a New York Heart Association (NYHA) class of III or IV. Patients were followed for an average of 2 years. Spironolactone reduced death by a whopping 11% (number needed to treat 9) and hospitalizations for cardiac causes by 8%, and there was no statistically significant difference in cases of serious hyperkalemia.
In the words of my two and a half year old son, you could drive a big old truck through the separation in the survival curves.
On a per-patient basis, RALES showed the largest absolute difference in survival for any large-scale clinical trial testing drug therapy in patients with chronic systolic heart failure.
RALES was published in September of 1999. Several years following its publication, Juurlink et al. conducted a population-based time-series analysis to examine trends in the rate of spironolactone prescriptions and the rate of hospitalizations for hyperkalemia before and after the publication of RALES.
The authors linked prescription-claims data and hospital-admission records for more than 1.3 million adults in Ontario, Canada, for the period from 1994 through 2001.
Among patients recently hospitalized for heart failure, the spironolactone prescription rate was 34 per 1,000 patients in 1994 and increased immediately after the publication of RALES, to 149 per 1,000 patients by late 2001.
Over that time, hospitalizations for hyperkalemia rose from 2.4 to 11 per 1,000 patients and the associated mortality rose from 0.3 to 2.0 per 1,000 patients. The publication of RALES was NOT associated with significant decreases in the rates of readmission for heart failure or death.
What gives? How could increased use of a drug that performed so well in a clinical trial not translate to better results in the real world?
To understand this, you must understand the concepts of internal validity and external validity and appreciate the difference between the two.
Internal validity is the extent to which a clinical trial accurately proves that the tested treatment directly caused the observed results. It confirms that the results are due to the intervention itself and not to bias, confounding variables, or measurement errors.
Like most drug trials published in high-impact journals, RALES has high internal validity. I’m confident that if spironolactone is used in patients, like those included in RALES, and the same procedures are applied (dose initiation, up-titration, clinical and laboratory monitoring, etc.), that similar results would be achieved. They wouldn’t necessarily be observable on an individual basis, but they would be detectable at a population level.
But internal validity is not nearly as important as external validity when it comes to clinical translation. External validity is the extent to which the study’s results can be generalized to the broader, “real-world” population of patients with the clinical condition of interest. It determines whether the treatment’s success will hold true when applied to everyday medical practice involving the “target population.”
Who was the target population in RALES? Fundamentally speaking, it was patients with systolic heart failure. If you drill down a little further, the average age was 65. In the Ontario population studied by Juurlink et al., the average age was 78 and 10% resided in a long-term care facility. This is a massive difference.
Remember this: patients in clinical trials tend to represent the most robust, ambulatory, educated, and highly motivated of those in the broader target population. This is true whether explicit inclusion/exclusion criteria are placed on these conditions or not.
Then there is the matter of trial procedures. Monitoring conditions in clinical trials is difficult to replicate in the real world.
In RALES, follow-up evaluations and laboratory measurements were conducted every 4 weeks for the first 12 weeks, then every 3 months for up to 1 year and every 6 months thereafter. Additional clinical laboratory tests were also performed at weeks 1 and 5. That’s a lot of monitoring—and it’s hard to do this in regular clinical practice.
Clear differences in age structure between RALES and the Ontario population studied by Juurlink, which will necessarily encompass significant differences in illness severity, frailty, multimorbidity, polypharmacy, etc., and differences in procedures likely drove the unexpected findings by Juurlink. It is a classic cautionary tale but one that is not appreciated nearly enough these days.
In my own teaching, I have developed a set of rules for applying clinical trial results to patient care:
When you know your patient would not be included in an RCT, proceed cautiously (if at all)
Advanced age, frailty, multimorbidity, or meets explicit exclusion criteria for trial
When you are unsure if your patient would be included, proceed cautiously (consider reducing the treatment effect by half and doubling the risk of harms and think, “Does this still seem like an attractive option?”)
When you are confident your patient would be included in an RCT, attempt to replicate RCT procedures (i.e., dose initiation, up-titration schedule, safety monitoring, etc.)
If you cannot, proceed cautiously
When a run-in phase is employed, start your patient on the medicine like they are in a run-in phase, and if they do poorly, stop
Pay attention to entry conditions in clinical trials.
Appreciate that certain patients are not included in RCTs because investigators know they WILL NOT be able to show benefit – so why would you try it on them?
Editor’s note: This may be the most important post of the year. I could not agree more strongly with Andrew. For many reasons, not least quality measures, clinical translation of trials is massively misunderstood in the real-world. We thank Andrew for this incredibly important piece. JMM
 | A guest post by
|
Beautifully direct, Dr. Foy, thank you.
Importantly relevant to virtually all daily meds. Most statin trials, for instance, used run-out phases in which 20-40% dropped out, yet virtually no one is carefully monitored for their first months on a statin. This is why side effects and intolerance are often up to TEN times higher in real world experience than in trials, making guidelines and published AE rates deeply misleading. Thank you again, beautifully explained.
This strikes me as a nearly perfect article: so congratulations. It’s simple and straightforward and it captures a very serious problem that we were routinely discount, the difference between internal validity and external validity.
I go on rounds and Residents refer to “goal directed management” of heat failure with almost no real appreciation of just how far away our patients lie from those in the trials