"The authors chose as the primary endpoint to measure four outcomes: death, need for tricuspid surgery, hospitalization for heart failure, or quality of life as measured by the Kansas City Cardiomyopathy Questionnaire (KCCQ)."
The first hierarchical outcome was death OR tricuspid valve surgery -- a composite. The whole point of the win ratio analysis is to account for differential importance of clinical events. Death is worse than heart attack, heart attack is worse than unplanned revascularization, etc. The authors went to the trouble of doing a win ratio analysis, but then functionally eliminated the benefit by analyzing death and surgery as statistical equals! How can this be? No one could disagree that death is categorically worse than surgery. Moreover, proceeding with TV surgery is a clinician's decision. Such outcomes in an unblinded trial are hard to interpret.
So, even though TEER was numerically WORSE for all-cause death (and heart failure hospitalization, and stroke, and major bleeding), a fact tucked away in table S13 in the supplementary appendix, this analysis conveniently disguises that. Somehow TEER comes out with more "wins" for death or TV surgery. This is the worst of both analytical techniques (win ratio and composite outcome).
Furthermore, since this is an UNmatched win ratio analysis (https://academic.oup.com/eurheartj/article/33/2/176/439013) with relatively low numbers of clinical events, it magnifies the placebo effect. The analysis involves 175*175=30,625 pairwise comparisons (175 patients in each arm). The lowest ranked outcome in the hierarchical analysis is 15 point improvement on KCCQ. But since most of the pairs did not experience any of the higher ranked clinical endpoints, the non placebo-controlled symptomatic improvement dominates the analysis.
As I read more about past trials and studies for various things related to the medical field, there is definitely the propensity to set these things up to satisfy biases and conflicts of interest. Data is fudged, manipulated, or negative data erased and conclusions worded in such ways that a bad or indifferent trial result is now a good trial result. As long as it gets past the FDA.
Helping further the science of medicine and developing trust is of no consequence. Of course, in any true scientific environment, a thousand experimental efforts must be made before one hits the jackpot. These days, they want the jackpot first and foremost...and they will do anything to get it.
FDA recently rejected Omecamtiv mecarbil for HFrEF (cuz it’s data from Galactic HF trial was underwhelming)…so they are capable of saying No. We will see if the FDA is similarly underwhelmed by the results summarized here. Let’s hope so. We have to rely on the gatekeepers to hold the line against flimsy science, as that is the only way to compel sponsors to design trials with more robust methodology. Sadly, the FDA’s history on this front (as evidenced by your prior discussions about Protect/Prevail, and Guide-HF) appears checkered at best.
I find this sort of bias, which is rampant in the medical literature, almost more sickening than outright fraud in the broader world. Those who sign off on trials such as this are inches away from medicine with a great service. Instead, we are left not only with the absence of something useful, but with an accredited document that can (will) be used to extract money from the system and result in patients being exposed to needless procedural risks
Maybe we could specify that the first characteristic sensible medicine is that it be honest medicine. "Gaming the system," is something that is routine and rewarded in many areas of America's competitive life, including law, business and medicine. Unfortunately, this form of corruption is both rarely punished and dangerously corrosive. Our systems for combatting it are mostly absent, ineffective or easily circumvented. Thanks for this clear example of the problem!
This is a great discussion. Thank you. It is very disconcerting that seasoned medical researchers would choose the incorrect control group design. The list of disclosures is also disconcerting as it suggests, as you said, these authors were bought, henceforth influenced, by industry. This is not unique. I look at disclosures for most papers I read after reading the abstract and methods. It seems like half of all studies published in NEJM have significant author disclosures attached to them. Does that suggest NEJM is possibly “captured” by industry because they publish industry supported studies or is it simply a case that NEJM requires authors to disclose and publishes the disclosures along with the paper.
A small but important point. You say that:
"The authors chose as the primary endpoint to measure four outcomes: death, need for tricuspid surgery, hospitalization for heart failure, or quality of life as measured by the Kansas City Cardiomyopathy Questionnaire (KCCQ)."
The first hierarchical outcome was death OR tricuspid valve surgery -- a composite. The whole point of the win ratio analysis is to account for differential importance of clinical events. Death is worse than heart attack, heart attack is worse than unplanned revascularization, etc. The authors went to the trouble of doing a win ratio analysis, but then functionally eliminated the benefit by analyzing death and surgery as statistical equals! How can this be? No one could disagree that death is categorically worse than surgery. Moreover, proceeding with TV surgery is a clinician's decision. Such outcomes in an unblinded trial are hard to interpret.
So, even though TEER was numerically WORSE for all-cause death (and heart failure hospitalization, and stroke, and major bleeding), a fact tucked away in table S13 in the supplementary appendix, this analysis conveniently disguises that. Somehow TEER comes out with more "wins" for death or TV surgery. This is the worst of both analytical techniques (win ratio and composite outcome).
Furthermore, since this is an UNmatched win ratio analysis (https://academic.oup.com/eurheartj/article/33/2/176/439013) with relatively low numbers of clinical events, it magnifies the placebo effect. The analysis involves 175*175=30,625 pairwise comparisons (175 patients in each arm). The lowest ranked outcome in the hierarchical analysis is 15 point improvement on KCCQ. But since most of the pairs did not experience any of the higher ranked clinical endpoints, the non placebo-controlled symptomatic improvement dominates the analysis.
As I read more about past trials and studies for various things related to the medical field, there is definitely the propensity to set these things up to satisfy biases and conflicts of interest. Data is fudged, manipulated, or negative data erased and conclusions worded in such ways that a bad or indifferent trial result is now a good trial result. As long as it gets past the FDA.
Helping further the science of medicine and developing trust is of no consequence. Of course, in any true scientific environment, a thousand experimental efforts must be made before one hits the jackpot. These days, they want the jackpot first and foremost...and they will do anything to get it.
FDA recently rejected Omecamtiv mecarbil for HFrEF (cuz it’s data from Galactic HF trial was underwhelming)…so they are capable of saying No. We will see if the FDA is similarly underwhelmed by the results summarized here. Let’s hope so. We have to rely on the gatekeepers to hold the line against flimsy science, as that is the only way to compel sponsors to design trials with more robust methodology. Sadly, the FDA’s history on this front (as evidenced by your prior discussions about Protect/Prevail, and Guide-HF) appears checkered at best.
With results like these, I doubt Medicare will pay for it.
And think about the patients’ personal bias who undergo (sometimes) painful procedures who WANT to have a positive outcome. Because don’t we all?
I find this sort of bias, which is rampant in the medical literature, almost more sickening than outright fraud in the broader world. Those who sign off on trials such as this are inches away from medicine with a great service. Instead, we are left not only with the absence of something useful, but with an accredited document that can (will) be used to extract money from the system and result in patients being exposed to needless procedural risks
Maybe we could specify that the first characteristic sensible medicine is that it be honest medicine. "Gaming the system," is something that is routine and rewarded in many areas of America's competitive life, including law, business and medicine. Unfortunately, this form of corruption is both rarely punished and dangerously corrosive. Our systems for combatting it are mostly absent, ineffective or easily circumvented. Thanks for this clear example of the problem!
This is a great discussion. Thank you. It is very disconcerting that seasoned medical researchers would choose the incorrect control group design. The list of disclosures is also disconcerting as it suggests, as you said, these authors were bought, henceforth influenced, by industry. This is not unique. I look at disclosures for most papers I read after reading the abstract and methods. It seems like half of all studies published in NEJM have significant author disclosures attached to them. Does that suggest NEJM is possibly “captured” by industry because they publish industry supported studies or is it simply a case that NEJM requires authors to disclose and publishes the disclosures along with the paper.