Millions of people have AF and take oral anticoagulation. It's one of Medicine's most evidence-based treatments. It would take strong data to upend this treatment.
On your 'bad practice' point John. I have been trying for decades to stop students/people saying that if two options perform equally on a criteria we can drop that criteria from the analysis ('noise' as you call it). A criteria should be included on the basis of its importance and the greater its importance the greater the importance of keeping it in, whether or not all options perform equally on it. The more important the criteria on which the two available options perform equally well the closer the verdict is to a toss-up between them, and it should be accepted as such. CV death seems a pretty important criteria to me,.
Maybe I’m mathematically incorrect.. if you told the patient as part of shared decision making- we have two items for cost comparison one cost - $10,000 and the other is $13,999. The way the studies work they consider both those prices to be the same. The device not only didn’t cost less than the standard therapy. It could cost $3999 more before it would be considered more expensive.
Thank you for once again navigating so deftly through the dense fog that can permeate clinical trial design and complicate interpretation of the data derived.
As a stroke doc who participated as a PI in SPAF I, II, and III, I was complicit in helping to establish the effectiveness of anticoagulation (AC) for preventing stroke in patients with AF. It is the nature of clinical practice that I cannot identify patients in whom AC has prevented a disabling cardioembolic stroke but do see patients who have experienced significant and even fatal complications of AC. As such, I would welcome the Watchman or any other device that provided AF patients the efficacy of AC without the associated bleeding risk. Unfortunately, we have yet to identify and confirm the utility of such a device.
A closing word about AC…
Especially since the arrival of the DOACs, AC for AF has evolved to become routine… perhaps even a bit reflexive. Not good– reflexive medicine can be harmful. Not all AF patients require AC, and even the DOACs convey risk.
In a study we presented at the 2024 European Academy of Neurology meeting, of 117 consecutive AF patients seen in a general neurology clinic, 37% were calculated to have an annual risk of a significant AC-related bleeding complication over 1 % greater than their annual risk of stroke if off AC.
Perhaps this does represent one clinical circumstance where shared decision-making may be appropriate.
Upper bound CI of watchman allowed to be 4.8% worse…than a control arm with an event rate of 4.8%. It’s incredible that NEJM would publish a paper that is allowed to suggest that “up to 100% worse” or “up to 2X event rate” is something that is “non-inferior”. It turned out to be up to 1.66X worse, which still stretched the credulity of the concept.
At least the accompanying editorial demonstrated restraint and did not participate in the cheerleading. It also mentioned that the protocol was designed by sponsor, and data was analyzed by sponsor. Maybe NEJM should have added an “advertorial” disclaimer instead of the “original research” label to the manuscript.
John - do you believe that a patient who is truly not a long-term anticoagulation candidate and who has significant risk for stroke - does the watchmen add benefit over hope?
PS. I have a post that will publish in a few weeks about shared decision-making and the information asymmetry between patients and doctors. The main point is that the phrase “shared decision-making” often is a more palatable way of telling patients what the physicians already has decided. I don't say that cynically. It's just what happens when the phrase gets tossed around and put in guidelines without thinking carefully about what it actually requires. I then discuss the SHARE approach and what sharing decisions really requires. Thanks, John for this post.
As our friend and brilliant methodologist Sanjay Kaul: commented elsewhere “Good trial design balances feasibility with rigor; when it’s designed to deliver a win, it signals more about clever protocol writing and trial design than breakthrough efficacy.” I am ashamed of the NEJM for publishing a paper on a study as flawed as this one - all the points you made, but especially, excluding protocol related bleeding, open label design, the excess of people with PAF who statistically did much better with OAC, and not required the authors to discuss the excess of strokes and the details of the non-inferiority bounds, as you laid out. Gregory M. Marcus's editorial was an insightful breath of fresh air and I highly recommend it.
Really good. Isn’t it remarkable what money can buy? It can corrupt the most honest of physician researchers. Watchman has been around for 20 years and they’re still trying to justify it.
I remember taking Combinatorics/Probability class as an engineering major. Thought I would breeze through it. The math wasn't hard AT ALL. But some of the ramifications and how to process the data was MUCH HARDER than I thought. It almost seemed illogical. Full of paradoxes. It sounds like the authors are counting on that for these types of studies.
On your 'bad practice' point John. I have been trying for decades to stop students/people saying that if two options perform equally on a criteria we can drop that criteria from the analysis ('noise' as you call it). A criteria should be included on the basis of its importance and the greater its importance the greater the importance of keeping it in, whether or not all options perform equally on it. The more important the criteria on which the two available options perform equally well the closer the verdict is to a toss-up between them, and it should be accepted as such. CV death seems a pretty important criteria to me,.
Maybe I’m mathematically incorrect.. if you told the patient as part of shared decision making- we have two items for cost comparison one cost - $10,000 and the other is $13,999. The way the studies work they consider both those prices to be the same. The device not only didn’t cost less than the standard therapy. It could cost $3999 more before it would be considered more expensive.
Thank you for once again navigating so deftly through the dense fog that can permeate clinical trial design and complicate interpretation of the data derived.
As a stroke doc who participated as a PI in SPAF I, II, and III, I was complicit in helping to establish the effectiveness of anticoagulation (AC) for preventing stroke in patients with AF. It is the nature of clinical practice that I cannot identify patients in whom AC has prevented a disabling cardioembolic stroke but do see patients who have experienced significant and even fatal complications of AC. As such, I would welcome the Watchman or any other device that provided AF patients the efficacy of AC without the associated bleeding risk. Unfortunately, we have yet to identify and confirm the utility of such a device.
A closing word about AC…
Especially since the arrival of the DOACs, AC for AF has evolved to become routine… perhaps even a bit reflexive. Not good– reflexive medicine can be harmful. Not all AF patients require AC, and even the DOACs convey risk.
In a study we presented at the 2024 European Academy of Neurology meeting, of 117 consecutive AF patients seen in a general neurology clinic, 37% were calculated to have an annual risk of a significant AC-related bleeding complication over 1 % greater than their annual risk of stroke if off AC.
Perhaps this does represent one clinical circumstance where shared decision-making may be appropriate.
Upper bound CI of watchman allowed to be 4.8% worse…than a control arm with an event rate of 4.8%. It’s incredible that NEJM would publish a paper that is allowed to suggest that “up to 100% worse” or “up to 2X event rate” is something that is “non-inferior”. It turned out to be up to 1.66X worse, which still stretched the credulity of the concept.
At least the accompanying editorial demonstrated restraint and did not participate in the cheerleading. It also mentioned that the protocol was designed by sponsor, and data was analyzed by sponsor. Maybe NEJM should have added an “advertorial” disclaimer instead of the “original research” label to the manuscript.
A triumph of technology over reason.
John - do you believe that a patient who is truly not a long-term anticoagulation candidate and who has significant risk for stroke - does the watchmen add benefit over hope?
When I started training in the 70s, the NEJM was not a medical journal but “the Bible“ – now it’s the “Enquier“ -(sensational rag )Ben Hourani MBA
What an excellent analysis. Thank you.
PS. I have a post that will publish in a few weeks about shared decision-making and the information asymmetry between patients and doctors. The main point is that the phrase “shared decision-making” often is a more palatable way of telling patients what the physicians already has decided. I don't say that cynically. It's just what happens when the phrase gets tossed around and put in guidelines without thinking carefully about what it actually requires. I then discuss the SHARE approach and what sharing decisions really requires. Thanks, John for this post.
As our friend and brilliant methodologist Sanjay Kaul: commented elsewhere “Good trial design balances feasibility with rigor; when it’s designed to deliver a win, it signals more about clever protocol writing and trial design than breakthrough efficacy.” I am ashamed of the NEJM for publishing a paper on a study as flawed as this one - all the points you made, but especially, excluding protocol related bleeding, open label design, the excess of people with PAF who statistically did much better with OAC, and not required the authors to discuss the excess of strokes and the details of the non-inferiority bounds, as you laid out. Gregory M. Marcus's editorial was an insightful breath of fresh air and I highly recommend it.
Great article
Really good. Isn’t it remarkable what money can buy? It can corrupt the most honest of physician researchers. Watchman has been around for 20 years and they’re still trying to justify it.
I remember taking Combinatorics/Probability class as an engineering major. Thought I would breeze through it. The math wasn't hard AT ALL. But some of the ramifications and how to process the data was MUCH HARDER than I thought. It almost seemed illogical. Full of paradoxes. It sounds like the authors are counting on that for these types of studies.