Randomized Trials Should Determine Causality and Be Useful in the Clinic—But Sometimes Accomplish Neither

The recent RCT comparing lactated ringers vs normal saline fails to answer the decades-old question. Here is why.

One of the most common and valid criticisms of a randomized clinical trial is external validity—or generalizability. The point of any RCT is that we can apply the results to a patient we see in the office.

A trial that recruits highly selected (perfect) patients is hard to apply to the “regular” everyday patient. A classic example is the DanGer-Shock trial of the microaxial flow pump in patients with cardiogenic shock. Primary investigator Jacob Moller took nearly 11 years to randomize 360 patients and in these perfect patients the pump lowered mortality.

Trial procedures can also influence generalizability of a trial. Take EAST AF, which found that early rhythm control lowered major adverse effects vs rate control in patients with new atrial fibrillation. The early rhythm control group had many more interactions with the healthcare team than the rate control arm. This created a likely performance bias.

The opposite of a highly selected trial is the so-called pragmatic trial. Pragmatism, of course, exists on a scale from somewhat pragmatic to highly pragmatic. The Chlorthalidone vs Hydrochlorothiazide trial was a highly pragmatic trial of the two drugs. The trial found no differences, but the pragmatic aspect of the trial created enough noise to render the results less than useful.

---

Today I discuss perhaps the most pragmatic trial I have ever seen.

The Canadian Critical Care Trials Group set out to answer the longstanding question of best IV fluid (IVF) to use in patients ill enough to need IVF. As background, there are many diseases and conditions that require IVFs. There are two common IVF: normal saline and lactated ringers. LR has slightly lower sodium content as well as other components, such as potassium, calcium, chloride and lactate. LR is called a balanced crystalloid.

Both types of fluids have been used for decades. Doctors argue about which one is best. There have been many RCTs comparing the two fluids in patients with different conditions. There are potential reasons why one fluid may better in certain conditions.

The Canadian Trial

The background is important. The large number of trials have given way to numerous meta-analyses of these trials, which have included more than 30,000 patients.

Let’s stop there and pause. If there are double-digit numbers of trials, and more than 30,000 patients in meta-analyses, you would think we know the best fluid replacement. Right?

Well, we don’t. NEJM published one of the most recent meta-analysis of 13 RCTs comparing the fluids in “critically ill” adults and found no difference in mortality. The numbers were 17.4% death with LR vs 18.2% with saline; relative risk, 0.96; 95% confidence interval [CI], 0.91 to 1.01.

The authors of this meta-analysis did something interesting: instead of saying that since the CI included benefit and harm, there was no statistical difference, they did what’s called a Bayesian analysis.

I won’t go into the details of Bayesian analyses but suffice to say this technique outputs the probability that one treatment reduces mortality. (I co-authored this review.) This differs from the normal frequentist analysis in which authors describe how surprising the data are given the made-up assumption that there is no difference in the treatments.

In the Bayesian analysis, the authors found an 89% probability that LR lowered mortality. Boom. A “negative” trial now looks positive.

This was the equipoise used to justify the Canadian group’s new trial.

The Trial Procedures

In this trial, patients were not randomized. Instead hospitals were randomly assigned to two 12-week periods where every patient who needed IV fluids received LR or NS. Then they did a washout period, then another 12-week period using the opposite fluid. This is called a cluster-randomized crossover trial.

The primary outcome was a composite of death or readmission to the hospital in the next 3 months. Administrative databases provided the outcomes. This trial estimated the effect of the two fluids on the hospital level. They call this the “estimand,” and it is the effect of LR vs NS averaged across the seven participating hospitals.

The estimand here is not the effect of the two IV fluids on individual patients, rather the effect being measured is on the hospital policy of using the two fluids.

Trial Results

About 40% of patients were considered “medical” patients, 42% were “surgical” and 17% were Ob-Gyn. The mean co-morbidity scores were similar.

Primary outcome events were available in more than 43,000 patients.

The mean (±SD) incidence of the composite of death or readmission to the hospital within 90 days after the index admission was 20.3±3.5% with lactated Ringer’s solution and 21.4±3.3% with normal saline (adjusted difference, −0.53 percentage points; 95% confidence interval, −1.85 to 0.79; P=0.35) All secondary outcomes were similar. And no heterogeneous treatment effects were evident when looking at subgroups of patients.

The authors concluded that the hospital-wide policy of giving either LR or NS did not result in any differences in death or readmission.

Overall Comment:

I cover this trial because it has all the markers of a definitive trial: multiple centers, many thousands of patients, unbiased endpoints and two simple treatments. You would think it would help answer the decades-long question of the best fluid replacement.

But it does not. It does not because it’s too pragmatic. It tries to answer a policy question when the right question is an individual patient question. When you are sick enough to need IV fluids, you want a trial that informs the decision for the best fluid.

In this trial, the huge pragmatism (every patient for 12 weeks got one or the other IV fluids) led to extreme amounts of noise that precludes reliable knowledge of the best IV fluid.

The ultimate problem is that the causal chain from fluid choice to death or readmission has countless other factors.

The first disrupter to causation is that the trial included vastly different conditions, ranging from pregnancy to sepsis to post-op states. The effect of two treatments could have different effects in different conditions.

The second obvious disrupter of the causal chain is that some patients received small amounts of fluid and others received lots. This surely bears on the treatment effect.

Third was that patients could have other treatments and surgeries—each of which could affect outcomes.

Fourth was that only 78% adhered to LR vs 93% to normal saline. That’s a big difference.

Finally, the authors powered the trial to find a 1% difference in mortality, and even with 43,000+ patients, the confidence interval went from -1.8% better to +0.8% harm.

Final Comment

When I look at the previous data, it’s hard to find a major difference in outcomes between the two fluids. Perhaps there isn’t a difference.

But if there was a difference, a trial looking at hospital policy is the wrong way to answer the question.

Hospital policies can and should be studied.

But not for specific treatment interventions. There is way too much noise in this trial to learn anything useful.

If you are a patient with a condition requiring IV fluids, this trial does not help you.

---

P.S. This is an unusual trial. If you are an expert and disagree, please say so, or contact us, and you can write a follow-up post. JMM