Cardiology trialists have found the saint graal of the outcomes: composite outcomes and hospitalizations by specific causes.
Someone must urgently study, write and publish about this issue. A RS searching if they are good surrogates for other patient oriented outcomes, for example. A perspective manifesto. I don’t know, something. It may not change it but at least it could inform not so attentive clinical physicians.
We see the same phenomenon in trials of new anti-cancer drugs - control arms which are drugs which have not been the standard of care for years. I this were boxing the control arms would be termed "palookas". Then these trials are designed for obtaining drug approval, not for comparative effectiveness.
This trial, without a sham control arm, produced a pointless and meaningless result.
The Orbita trials showed that sham control in cardiac interventional studies is entirely feasible. It should be the new standard in cardiology, for any trial that uses, or includes, soft “patient symptom driven” endpoints. If this trial was started before orbita was published, then I suppose the investigators could not have known what they didn’t know; but if this was conceived after orbita was published, then shame on those investigators. (I don’t blame Edwards….their fiduciary duty is to their shareholders, so they’re gonna do what they gotta do, and they’re gonna do what the cardiology community lets them do).
There is no easy solution to improving the quality of trial design. In my younger days, I would have hoped that the scientific committees of major meetings (like TCT) would decline presentation slots to poorly done studies; or that major journals (like NEJM) would refuse to publish manuscripts unless they were contrite in recognition of their study’s flaws and limitations. But I’m less naive now, and recognize that meetings and journals are merely mouthpieces now, and not far removed from being extensions of the PR departments of big pharma and device companies. I don’t expect much resistance from guideline committees either, since the members on those writing panels are beholden to the same “incentive” structure as meetings and journals.
And so, the solution and the bulwark against this comes back to the individual physician. And a hefty dose of caveat emptor. To resist the fashion of the day; to insist on evidence that is sufficient to reject the null; to demand good reasons to intervene on patients because we should, and not merely because we can.
There was a day before "evidence-based" medicine that physicians could review the research and make their own decisions about what treatments to offer, based upon their experience treating patients. The notion of the godliness of any research study is a travesty to the practice of medicine.
"To kill an error is as good a service as, and sometimes even better than, the establishing of a new truth or fact." -- Charles Darwin
Sadly, there are serious reputational and financial factors that can distort the truth-finding mission of science. Those need to be called out (as is done so well here at Sensible Medicine! 👏) without becoming cynical about the whole enterprise.
Your analysis was spot on but I don't agree with the conclusion that the trial was a waste. We did learn that TAVR in asymptomatic patients provides no real benefit in the things that really matter over a 5 year followup period. Pharmaceutical companies and medical device manufacturers will always try to find some soft end points to weave into a composite that will make their product look useful. As long as all the hard end points are clearly given and can be disentangled from the composites, I don't have any objection. Obviously the end points and characteristics of the study population need to be rigorously defined in order to draw any meaningful conclusions. For example, in this case, the criteria for the label "asymptomatic" should be clearly defined.
Haven’t sat on NIH review panels I don’t see how we can leave decisions about spending massive amounts of money on studies to peer review and the NIH. Companies take all the risk, they’re the ones who need to decide about the massive sums they spend.
But more importantly: do you recommend TAVR to your patients with asymptomatic severe AS? Or not?
Is it not obvious the incentives of pharma and device manufacturers? The landscape is riddled with poorly designed studies and flawed analysis because a “positive result” (e.g. one favorable to the economic interests of the study sponsor) was desired. NIH is no better. The revolving door of govt officials to private sector along with the financial interests of professionals within NIH shouldn’t give us more confidence of unbiased research.
I agree that evidenced based practice should be the standard. However, to simply conclude patients “converted or were faith healed” because of their known study category is erroneous. Just because patients did or did not have a cardiac event or stroke doesn’t mean they were not symptomatic.
What of the patient who has severe fatigue? Or, who develops chronic daily headaches? Or, who displays increasing frequency of reactive airway disease? These concerns can also be displayed in the presence of moderate to severe aortic valve stenosis or disease.
I don't really understand the specific concern about the wrong primary endpoint. Sure, some readers may be "fooled", but cardiologists are pretty good at looking at these things, and the relevant data are available.
Apart from the notion that the main cause of the positive result was that getting a TAVR decreases the need for a future TAVR, there is the other common problem with many composite endpoints used in cardiology trials: the different components have very different levels of importance. "Death" is a big deal, "strokes" can be major or minor, "unplanned hospitalization" is of little importance.
However, the data are in Table 2, where the composite of "Death or Disabling Stroke" is shown (and is a reasonable composite in my estimation). The trial shows a point estimate for the HR of 0.87 for early TAVR, with a CI from 0.58 to 1.31.
We need to move to absolute differences to understand what that means. If 1000 people with asymptomatic critical AS do not get early TAVR, we have a point estimate that 15 more will have death or disabling stroke over about four years. However, this ranges from about 47 fewer such events to 35 more such events.
If you would not want early TAVR to avoid a 4.7% additional risk over four years of death or disabling stroke, then these data are adequate -- you should not get early TAVR. If this would lead you to get early TAVR, this trial is underpowered and a larger trial is needed. Even then, though, this trial will provide help in estimating the size of an adequately powered trial and will be able to be included in a meta-analysis.
This trial does not answer all the questions I might have about TAVR in asymptomatic critical AS, but neither is it a clear waste of time and resources. I know more about patient-important outcomes based on this decision than I did before the trial was performed. However, it also remains plausible that early TAVR is sufficiently beneficial to be worth performing or that it has few benefits or is even harmful.
As an anesthesiologist in a community hospital that is not equipped with open heart surgery capabilities, we see these “asymptomatic” patients with severe AS, for all kinds of elective and urgent/emergent procedures requiring general anesthesia. The preload reduction that often comes with anesthesia, most profoundly with spinal anesthesia is very difficult to reconcile, and when these patients code in the OR, they are impossible to resuscitate. One cannot give a cardiac anesthetic to a patient with a kidney stone and everyone (patient, surgeon, family) is shocked when we tell them they likely need ICU postop.
Perhaps one of the outcome measures followed should be surgical survival/denial in the control and treatment arms.
“another potential solution is that users of medical evidence equip themselves with enough skills in critical appraisal so that they are not easily bamboozled by such trials.”
The question is how, after the corrupt Covid incentives from insurance co’s, gov’t, and big Rx destroyed any confidence pts may have in most physicians who understand the dangerous nature of mRNA vax w/ LNP’s. 30-40 yrs ago doctors would have NEVER touched that vax based on animal studies alone.
Tim 6:10 For the love of money is the root of all kinds of evil. Praying new admin will select you to address these serious concerns you most certainly understand.
You have answered your own question. Drug and device developers cannot be trusted to design trials that generate reliable information. They regularly stack the deck. They should pay the NIH or a similar trusted body to do the trial. The problem you describe is widespread. Only about a third of the patients in the Entresto trial for reduced ejection fraction heart failure were on an aldosterone blocker. (mineralocorticoid receptor antagonist). Had they been on optimal medical therapy, the positive results of the trial would have been dramatically blunted. Spironolactone and eplerenone are much less expensive than Entresto and we could save a ton of money by using them earlier. Medical literacy is not good enough for your other option to work.
I agree. I don't think pharma conducting their own trials have any credibility at all anymore. If there was a body like NIH that conducted all trials, we would need to be very sure to remove perverse incentives. They would need to be completely uncoupled and independent of any pharma ties and funding. Members would need to have a significant amount of time when leaving employment before one could be employed by a pharma company. 10 years? Maybe even that's not enough?
Public funding of all research sounds ideal, but even then I’m not sure how you can eliminate perverse incentives, some of which you alluded to. Further, if tax dollars went to NIH to fund a study that showed an Edwards product to be “beneficial”….which results in Edwards selling more product for private profit…I’m not sure that’s a great use for tax dollars either.
It’s thus EXTREMELY common for the makers of such-and-such to sponsor studies of the thing they make (and would like to sell). It’s in fact the norm. But not all sponsorship is created equal. Ideally, while the sponsor provides funding, there should be no sponsor “involvement” in protocol design, study execution, data analysis, and manuscript generation. The other extreme is when studies are done by in-house employees, which one can basically ignore.
Great insight; how do you make asymptomatic better? Seems like a fool’s errand. The 2 and 4 year survival of severe asymptomatic AS is 93% and 89 % respectively. Since AS affects mainly older individuals (75+) with an annual mortality of 3 to 4% what value does the intervention offer?
As far as funding of research, I think a pool of funds from Pharma/ device manufacturers could be established to distribute to researchers by a non conflicting agency or entity, governed by strict separation requirements.
Fantastic analysis. Our academic medical leaders have to stop accepting these endpoints. Ultimately, this trial just showed that having a planned TAVR reduces your probability of needing an unplanned TAVR within 6 months- a conclusion which is self-evident.
Cardiology trialists have found the saint graal of the outcomes: composite outcomes and hospitalizations by specific causes.
Someone must urgently study, write and publish about this issue. A RS searching if they are good surrogates for other patient oriented outcomes, for example. A perspective manifesto. I don’t know, something. It may not change it but at least it could inform not so attentive clinical physicians.
I would be glad to collaborate.
Dubious references from Stalinist Russia: "Give me the man and I will give you the case against him"
In medicine, a PI could say: "Give me the device (drug) and I will give you a positive trial for it"
So if we do not include "unplanned cardiac hospitalizations", we can conclude?
We see the same phenomenon in trials of new anti-cancer drugs - control arms which are drugs which have not been the standard of care for years. I this were boxing the control arms would be termed "palookas". Then these trials are designed for obtaining drug approval, not for comparative effectiveness.
This trial, without a sham control arm, produced a pointless and meaningless result.
The Orbita trials showed that sham control in cardiac interventional studies is entirely feasible. It should be the new standard in cardiology, for any trial that uses, or includes, soft “patient symptom driven” endpoints. If this trial was started before orbita was published, then I suppose the investigators could not have known what they didn’t know; but if this was conceived after orbita was published, then shame on those investigators. (I don’t blame Edwards….their fiduciary duty is to their shareholders, so they’re gonna do what they gotta do, and they’re gonna do what the cardiology community lets them do).
There is no easy solution to improving the quality of trial design. In my younger days, I would have hoped that the scientific committees of major meetings (like TCT) would decline presentation slots to poorly done studies; or that major journals (like NEJM) would refuse to publish manuscripts unless they were contrite in recognition of their study’s flaws and limitations. But I’m less naive now, and recognize that meetings and journals are merely mouthpieces now, and not far removed from being extensions of the PR departments of big pharma and device companies. I don’t expect much resistance from guideline committees either, since the members on those writing panels are beholden to the same “incentive” structure as meetings and journals.
And so, the solution and the bulwark against this comes back to the individual physician. And a hefty dose of caveat emptor. To resist the fashion of the day; to insist on evidence that is sufficient to reject the null; to demand good reasons to intervene on patients because we should, and not merely because we can.
There was a day before "evidence-based" medicine that physicians could review the research and make their own decisions about what treatments to offer, based upon their experience treating patients. The notion of the godliness of any research study is a travesty to the practice of medicine.
"To kill an error is as good a service as, and sometimes even better than, the establishing of a new truth or fact." -- Charles Darwin
Sadly, there are serious reputational and financial factors that can distort the truth-finding mission of science. Those need to be called out (as is done so well here at Sensible Medicine! 👏) without becoming cynical about the whole enterprise.
Your analysis was spot on but I don't agree with the conclusion that the trial was a waste. We did learn that TAVR in asymptomatic patients provides no real benefit in the things that really matter over a 5 year followup period. Pharmaceutical companies and medical device manufacturers will always try to find some soft end points to weave into a composite that will make their product look useful. As long as all the hard end points are clearly given and can be disentangled from the composites, I don't have any objection. Obviously the end points and characteristics of the study population need to be rigorously defined in order to draw any meaningful conclusions. For example, in this case, the criteria for the label "asymptomatic" should be clearly defined.
Haven’t sat on NIH review panels I don’t see how we can leave decisions about spending massive amounts of money on studies to peer review and the NIH. Companies take all the risk, they’re the ones who need to decide about the massive sums they spend.
But more importantly: do you recommend TAVR to your patients with asymptomatic severe AS? Or not?
Is it not obvious the incentives of pharma and device manufacturers? The landscape is riddled with poorly designed studies and flawed analysis because a “positive result” (e.g. one favorable to the economic interests of the study sponsor) was desired. NIH is no better. The revolving door of govt officials to private sector along with the financial interests of professionals within NIH shouldn’t give us more confidence of unbiased research.
I agree that evidenced based practice should be the standard. However, to simply conclude patients “converted or were faith healed” because of their known study category is erroneous. Just because patients did or did not have a cardiac event or stroke doesn’t mean they were not symptomatic.
What of the patient who has severe fatigue? Or, who develops chronic daily headaches? Or, who displays increasing frequency of reactive airway disease? These concerns can also be displayed in the presence of moderate to severe aortic valve stenosis or disease.
I don't really understand the specific concern about the wrong primary endpoint. Sure, some readers may be "fooled", but cardiologists are pretty good at looking at these things, and the relevant data are available.
Apart from the notion that the main cause of the positive result was that getting a TAVR decreases the need for a future TAVR, there is the other common problem with many composite endpoints used in cardiology trials: the different components have very different levels of importance. "Death" is a big deal, "strokes" can be major or minor, "unplanned hospitalization" is of little importance.
However, the data are in Table 2, where the composite of "Death or Disabling Stroke" is shown (and is a reasonable composite in my estimation). The trial shows a point estimate for the HR of 0.87 for early TAVR, with a CI from 0.58 to 1.31.
We need to move to absolute differences to understand what that means. If 1000 people with asymptomatic critical AS do not get early TAVR, we have a point estimate that 15 more will have death or disabling stroke over about four years. However, this ranges from about 47 fewer such events to 35 more such events.
If you would not want early TAVR to avoid a 4.7% additional risk over four years of death or disabling stroke, then these data are adequate -- you should not get early TAVR. If this would lead you to get early TAVR, this trial is underpowered and a larger trial is needed. Even then, though, this trial will provide help in estimating the size of an adequately powered trial and will be able to be included in a meta-analysis.
This trial does not answer all the questions I might have about TAVR in asymptomatic critical AS, but neither is it a clear waste of time and resources. I know more about patient-important outcomes based on this decision than I did before the trial was performed. However, it also remains plausible that early TAVR is sufficiently beneficial to be worth performing or that it has few benefits or is even harmful.
As an anesthesiologist in a community hospital that is not equipped with open heart surgery capabilities, we see these “asymptomatic” patients with severe AS, for all kinds of elective and urgent/emergent procedures requiring general anesthesia. The preload reduction that often comes with anesthesia, most profoundly with spinal anesthesia is very difficult to reconcile, and when these patients code in the OR, they are impossible to resuscitate. One cannot give a cardiac anesthetic to a patient with a kidney stone and everyone (patient, surgeon, family) is shocked when we tell them they likely need ICU postop.
Perhaps one of the outcome measures followed should be surgical survival/denial in the control and treatment arms.
“another potential solution is that users of medical evidence equip themselves with enough skills in critical appraisal so that they are not easily bamboozled by such trials.”
The question is how, after the corrupt Covid incentives from insurance co’s, gov’t, and big Rx destroyed any confidence pts may have in most physicians who understand the dangerous nature of mRNA vax w/ LNP’s. 30-40 yrs ago doctors would have NEVER touched that vax based on animal studies alone.
Tim 6:10 For the love of money is the root of all kinds of evil. Praying new admin will select you to address these serious concerns you most certainly understand.
Medicine needs to stop trying to make humans immortal instead of having happy productive lives.
You have answered your own question. Drug and device developers cannot be trusted to design trials that generate reliable information. They regularly stack the deck. They should pay the NIH or a similar trusted body to do the trial. The problem you describe is widespread. Only about a third of the patients in the Entresto trial for reduced ejection fraction heart failure were on an aldosterone blocker. (mineralocorticoid receptor antagonist). Had they been on optimal medical therapy, the positive results of the trial would have been dramatically blunted. Spironolactone and eplerenone are much less expensive than Entresto and we could save a ton of money by using them earlier. Medical literacy is not good enough for your other option to work.
I agree. I don't think pharma conducting their own trials have any credibility at all anymore. If there was a body like NIH that conducted all trials, we would need to be very sure to remove perverse incentives. They would need to be completely uncoupled and independent of any pharma ties and funding. Members would need to have a significant amount of time when leaving employment before one could be employed by a pharma company. 10 years? Maybe even that's not enough?
Public funding of all research sounds ideal, but even then I’m not sure how you can eliminate perverse incentives, some of which you alluded to. Further, if tax dollars went to NIH to fund a study that showed an Edwards product to be “beneficial”….which results in Edwards selling more product for private profit…I’m not sure that’s a great use for tax dollars either.
It’s thus EXTREMELY common for the makers of such-and-such to sponsor studies of the thing they make (and would like to sell). It’s in fact the norm. But not all sponsorship is created equal. Ideally, while the sponsor provides funding, there should be no sponsor “involvement” in protocol design, study execution, data analysis, and manuscript generation. The other extreme is when studies are done by in-house employees, which one can basically ignore.
Great insight; how do you make asymptomatic better? Seems like a fool’s errand. The 2 and 4 year survival of severe asymptomatic AS is 93% and 89 % respectively. Since AS affects mainly older individuals (75+) with an annual mortality of 3 to 4% what value does the intervention offer?
As far as funding of research, I think a pool of funds from Pharma/ device manufacturers could be established to distribute to researchers by a non conflicting agency or entity, governed by strict separation requirements.
Fantastic analysis. Our academic medical leaders have to stop accepting these endpoints. Ultimately, this trial just showed that having a planned TAVR reduces your probability of needing an unplanned TAVR within 6 months- a conclusion which is self-evident.