Finding Bias Before the First Patient is Enrolled

Last Monday, we considered potential bias in data analysis. Today, the Study of the Week considers another source of bias—trial design.

Before I show you an example of trial-design bias, it is worth thinking about why it occurs.

Children learn that the core idea of science is to answer a question about nature using experimental methods. It matters not how an experiment turns out. In fact, most experiments don’t unlock mysteries; science is supposed to be hard.

Medical science is different, and it has a lot to do with its funding. For better or worse, industry funds much of medical science.

Industry does not fund studies to answer questions about nature. Industry funds (and designs) studies to show that their product works. This is not nefarious. Society can reap huge rewards from the confluence of interests. Industry-sponsored studies have greatly improved my field of electrophysiology.

But. But. This duality of interest forces consumers of medical evidence to be careful when interpreting industry-funded medical evidence.

The ATLAS trial compared two types of implantable cardioverter defibrillators, or ICDs. It allows a shining example of trial-design bias.

I have to explain a little about the two ICD types. I will keep it simple. Here is a picture.

Transvenous ICD: The implant involves placing a wire(s) into the veins in the chest, that then goes across a valve into the right ventricle. This lead senses the heartbeat, and if the rate is too fast (like that which occurs in ventricular fibrillation), the device shocks the heart.

The downside of this system is that the leads are in the blood vessels and heart, and they eventually scar in place. Infection can occur, and if it does, it is often in the heart. That is bad. And if the lead fails after years, it can require specialized extraction procedures to remove it. These procedures come with serious risks.

Subcutaneous ICD: The S-ICD does not have leads in the heart or blood vessels. The lead sits just outside the ribs, close enough to the heart to sense the rate. The generator sits in the side of the chest just south of the armpit.

The potential advantage is that it can provide the same shock capability but without the risk of infection within the heart. The downsides of this system are that not having a lead in the heart makes sensing more challenging. Inappropriate shocks are higher with the S-ICD and the S-ICD cannot pace the heart for slow rates—like the TV-ICD. Also, not having leads in the heart can make defibrillation more difficult.

---

Since the implant techniques of the two systems are different, the potential complications differ as well.

The traditional ICD gained approval because it was used in numerous clinical trials that showed survival was better than standard medical therapy. The S-ICD gained approval because it was shown to be able to terminate VF in a series of patients in an EP lab.

A head-to-head comparison of these life-saving devices is clearly an important scientific question.

---

Now to the ATLAS trial. This trial included 544 patients who were randomly assigned to receive either of the two ICD types. The authors tell us the purpose of the trial was to assess “key ICD performance measures including inappropriate shocks or failed ICD shocks.”

Their choice of primary outcome—and this is the key point of this column—were five bad things that occur within 6 months of implant. I made a slide.

Notice that none of these endpoints assess what the authors say was the key reason for the trial—inappropriate or failed shocks.

The results of this trial found that only 0.4% of those in the S-ICD vs 5% of those in the TV-ICD had a primary outcome. This difference easily met statistical significance.

Their first sentence in the discussion:

The ATLAS trial demonstrates a 92% reduction in lead-related complications using the S-ICD.

---

This is a true conclusion. But this slide shows the problem: 4 of the 5 primary endpoints can only occur in the transvenous arm

This Tweet, from the makers of the S-ICD, came shortly after the trial was presented in May at the Heart Rhythm Society meeting.

BostonSci Cardiology @BSCCardiology

Have you heard? 12X fewer serious complications using EMBLEM™ S-ICD vs. TV-ICD. Read the full results of the ATLAS trial bit.ly/3kTiqTa

---

My Conclusions:

Here is a major clinical trial, presented as a featured study in an important medical meeting, sponsored by industry, conducted by eminent researchers and it was guaranteed to show a clear win for the type of device made by the sponsor.

Four of the five primary endpoints could only occur in TV-ICD arm!

On occasion, I have implanted the S-ICD because of its potential advantages. Emphasis is on potential.

The ATLAS study does nothing to inform the important scientific question as to what occurs at two, five, ten, or 20 years.

The take-home message is that when you read medical studies, you need to consider potential biases in their design.

This is a classic case. Others will be more subtle.