Explain This One to Me

The Study of the Week goes back to the beginning of left atrial appendage occlusion to look at the PREVAIL trial

Next weekend at the Heart Rhythm Society meeting I will debate left atrial appendage occlusion. I’ve lost track of how many times I have tried to convince my colleagues that the evidence supporting this procedure is weak.

The makers of one of the more popular devices, called Watchman, advertise on their website that 400,000 devices have been implanted worldwide. They, like the first professor I ever debated, use the number of procedures done as supporting evidence.

Since readers of Sensible Medicine are well versed in the tragic story of anti-arrhythmic drugs to treat PVCs after MI or hormone replacement therapy in post-menopausal women to prevent cardiac events, I know that you know that acceptance of a therapy by the medical establishment does not equal supporting evidence.

Today I will show you one of the most curious things I have seen since studying medical evidence.

It is the story of how left atrial appendage occlusion got approved and then accepted.

It’s always important to study the seminal trials. (This discussion involves percutaneous left atrial appendage closure; surgical LAA closure has had positive results when done at the time of cardiac surgery.)

The story begins with a trial called PROTECT AF. The Watchman vs warfarin trial reported that the device easily met non-inferiority against warfarin. The rate ratio for the primary endpoint of stroke, systemic embolism or cardiovascular/unexplained death was 0.60 with credible intervals from 0.41-1.05. Since the worst-case scenario of 5% worse was far less than the non-inferiority margin, non-inferiority was easily met.

That paper sits in JAMA as a hugely positive trial. But. FDA rejected it. There were many issues with the trial. Some issues were internal validity concerns and some were high procedural complication rates.

FDA basically said do another trial.

---

The PREVAIL trial was the second trial.

The five-year results of PREVAIL were published in 2017. The trial enrolled patients with atrial fibrillation at higher risk for stroke.

FDA allowed two co-primary endpoints. The first primary endpoint was still stroke, systemic embolism and cardiovascular/unexplained death. Pause and think before I tell you the second co-primary endpoint. It was stroke, systemic embolism excluding the week after implant. The authors called this the “proof-of-concept” endpoint. I call it strange, because what patient ever can exclude the week after a procedure?

Here are the results of the first co-primary endpoint:

In every analysis, the rate of events was higher in the Watchman arm. In the final results, the Watchman arm had a 33% higher rate of events. You can see the that the upper bound (worst-case) allows for a more than doubling (2.13) of stroke, systemic embolism or CV/unexplained death. The agreed upon non-inferiority margin was 75% worse, so the device failed to meet even this lax standard of non-inferiority.

Here are the results of the made-up endpoint excluding mishaps in the week after the implant. One positive of this endpoint was that it included only stroke or systemic embolism.

Patients in the Watchman arm sustained a near doubling of clot-related issues (stroke/systemic embolism) vs warfarin. The worst-case (upper bound of the confidence interval) was nearly 5x worse. This did not meet the non-inferiority margin when analyzed by relative rate. But the FDA had also allowed non-inferiority to be judged by rate difference (absolute rates). Here it just barely met the agreed-upon difference of 2.75. ** (see footnote)

Here is my next slide in my debate:

---

These data went before FDA and the device was approved. I know. I can’t explain it.

Here is the wording of the FDA approval: (emphasis mine)

…for patients with AF at high-stroke risk and are recommended for anticoagulation, suitable for anticoagulation and have an appropriate rationale to seek non-drug alternative to warfarin taking into account the safety and efficacy of the device vs warfarin.

The major payer in the US, CMS had slightly different wording:

A suitability for short-term warfarin but deemed unable to take long term oral anticoagulation following the conclusion of shared decision making, as LAAC is only covered as a second line therapy to oral anticoagulants.

The CMS wording is the most curious because patients who could not take oral anticoagulation over the long-term were excluded from PROTECT and PREVAIL trials. Nearly a decade has passed since CMS started paying for Watchman and we still have no compelling data in these sorts of patients. We just don’t know.

What’s more, we now routinely use direct acting oral anticoagulants which have been proven superior to warfarin. Translation: Watchman missed against an inferior anticoagulant.

---

That’s the story of the seminal trials.

Current interventional cardiology fellows spend many hours learning how to do this procedure. Non-invasive imaging fellows spend similar hours learning how to measure the left atrial appendage solely to guide implants. Patients see oodles of advertisements for this device. They come to me asking about it.

And the device had nearly double the rate of strokes vs an inferior anticoagulant in the seminal trials. There have been no randomized trials in patients deemed ineligible for long-term oral anticoagulation.

Yet my colleagues have implanted 400,000 of these devices. It’s a head scratcher.

---

** Non-inferiority testing is appropriate in the case of LAAO because… if the device worked, it would be a way to avoid long-term oral anticoagulation.

Regulators and authors agree upon a non-inferiority margin: basically, how much worse the new therapy can be than the established standard.

Event rates are then compared in the two groups. You always take the upper bound (or worst-case) of the 95% confidence intervals.

It can be compared using rate ratios—which is relative risk. Or it can be compared using rate differences—which is absolute risk.

The problem with rate difference is that if the observed rates come out lower than the expected rates, it is much easier to meet the non-inferiority margin. Think that a rate difference between say 20 and 18 is a lot different than 4 and 2. This is what happened in PREVAIL.