‘I See It Work’ Is Bad Evidence

Here is Dr Wil Ward on the downsides of anecdote as data, He uses the LifeVest wearable cardiac defibrillator and VEST trial as an example. This is really good. JMM

Can we say LifeVest saves lives when a trial found it doesn’t?

We have an uncanny urge to label treatments as “life-saving”, even when incredible uncertainty surrounds such a claim.

• We claim screening colonoscopy can “save lives” even when trial data fails to show improvement of all-cause mortality

• The CDC claimed pediatric COVID vaccination prevented death, even when such a causal relationship was impossibly small to quantify and implausible

• The American Psychiatric Association called antidepressants “lifesaving” during pregnancy, even though they increase suicidal ideation under 24 years old

Perhaps our eagerness to help leads us to forget that trust is needed for medicine to save lives.

LifeVest Counseling Gone Wrong

Tom was a 56-year-old with a history of incarceration and meth use admitted for new heart failure. He came to the Emergency Department when worsening dyspnea prevented him from walking across his house. He took no medications and preferred to avoid doctors.

He had bilateral lower extremity edema and an echocardiogram found his LVEF to be 12%. It was a case of methamphetamine-induced heart failure.

Over several days, the edema resolved with diuresis. His low blood pressure only allowed for low-dose heart failure medication therapy. We discussed maximizing his survival odds through methamphetamine abstinence, medication treatment, physical activity, and close follow up.

I learned that the cardiologist would offer him a LifeVest. I encouraged Tom to ask questions.

‘Why are they offering this LifeVest?’

‘Well, with advanced heart failure, you have a risk of a fatal heart rhythm. The LifeVest shocks you if it detects a dangerous rhythm. Theoretically, this should save your life.’

‘Why do you say “theoretically”?’

‘Well, in a trial, LifeVest did not conclusively save lives.’

‘So why would someone recommend it?’

‘When we monitor people with the device, it appears to work as intended. But a clinical trial was not convincing. I would consider LifeVest a lower priority compared to the other interventions we discussed.’

I reassured him that his remaining treatment plan was sound. I checked in later:

‘What did the cardiologist say?’

‘That if I have a fatal heart rhythm, there is a 90% chance the LifeVest will save my life.’

LifeVest Evidence

There is a troubling gap between the narrative pitched to Tom (which is a common belief) and what the best evidence shows. When I presented weak trial data to the consulted cardiologist his response was typical,

‘But I see it work. It delivers shocks appropriately, so obviously it saves lives.’

This mechanistic reasoning is plausible. But is it sufficient?

Let’s review the Vest Prevention of Early Sudden Death Trial (VEST). It is the only LifeVest RCT ever conducted. It randomized (2:1 ratio) 2,302 participants with an LVEF <35% and a recent myocardial infarction to LifeVest or no LifeVest. The data is unconvincing for a few reasons:

1. VEST failed the primary outcome, arrhythmic death (1.6% vs 2.4%, p=0.18)

2. All-cause mortality was lower in the LifeVest arm (3.1% vs 4.9%, p=0.04), but not significant after multiple comparison adjustment. An anomalous stroke imbalance (4 fatal strokes in control, 0 in device, p=0.01) likely explains this difference.

3. The per-protocol analysis shows device adherence was correlated with lower all-cause death (p=0.01). But such thinking is also flawed. Might adherent participants have better overall health, contributing to all-cause longevity? (Editor’s note: intention-to-treat is even more important in this trial because the vest is difficult to comply with. It’s invasive, requires energy, and often beeps false alarms. Poor adherence therefore is inherent in the Rx. It would be like a huge-sized pill that was difficult to swallow or caused esophageal pain.JMM)

The “life-saving” nature of LifeVest is unsupported. Yet, it remains recommended by guidelines and heralded as a standard intervention for heart failure with reduced ejection fraction.

Who Lost Trust in Whom?

Tom followed up for a cardiology appointment. Staff reported to me that he was angry, defiant, and unlikely to ever follow-up again. This depiction contradicted the gentle and introspective Tom that I knew. I felt a pang of guilt and remembered Tom saying:

Prison is supposed to rehabilitate people. But many people you meet in prison only intend to hurt you. I learned in prison who I could and could not trust.

Had I caused him to abandon care?

I called him.

He was taking all his medications and felt much better. At his follow-up cardiology appointment, he declined the LifeVest a second time. And despite his explicit refusal, a LifeVest arrived at his doorstep a few days later. Tom left it in the box and arranged follow-up in the next closest town.

Tom did not lose trust in medicine. The cardiologist did not trust that Tom could make an informed decision.

It All Turns On Humility

Guidelines, plausibility, and surrogate endpoints often suggest an intervention is effective when it is not. The LifeVest was approved then recommended by the AHA before VEST was published. When VEST failed its primary endpoint, Dr. Mandrola called for guideline revision. Seven years later, the AHA has not responded.

New-onset heart failure is an overwhelming diagnosis requiring multiple new medications and frequent follow-up. Adding the LifeVest compounds the burden: it is expensive, inconvenient, and anxiety-inducing. The device makes physicians’ jobs harder too. Delivery can delay hospital discharge. And patients often refuse it unless provided far-fetched claims.

We should celebrate VEST’s negative results. A well-designed RCT involving thousands of patients did not find it saved lives by shocking fatal arrhythmias. This should make heart failure management simpler. Tom understood that trial data should overrule plausibility. I think the rest of us should, too.

Wil Ward, MD is a family medicine physician and hospitalist in rural Iowa.