The problem is that despite this manipulation that is similar to the one used in EXCEL (repeat revascularization was removed for the first time in CABG vs PCI trials) and a custom made MI definition (not detailed in the trial plan submitted to clinical trials.gov) was used, this paper will certainly be used in future Guidelines recommendations
I think Dr. Mandrola is being too generous with his upper bounds for declaring “nefarious behaviour “. Changing the primary endpoint, even if prespecified, is an absolute no-no. If this was initially planned as a 36 month endpoint study and they did that, there would be an outcry.
This is yet another in a long line of studies by people who push catheters, wanting to justify their reasons for pushing more catheters. There is ample prior data to suggest that revasc (more than other endpoints) is particularly poor for PCI (Vs CABG) as time goes by. It would be entirely predictable (physiology guided PCI or not) to see that drive the primary composite more and more with progressive follow up. And that is in fact what was shown. That they have obfuscated this fact by changing the endpoint (and that CIRC allowed them to do so without more overt indication of this in the abstract and text) is egregious abuse of the scientific process by both the authors and the journal.
And of course, the other elephant in the room is that med Rx alone was not studied. This part is at least excusable, since this study came before Ischemia. But it doesn’t eliminate or resolve the dilemma of “just because we can (revascularize in some way for stable CAD) doesn’t mean we should”.
It’s and excelent example about data manipulattion.. Nowadays, this trial should not be considered appropiate in chronic coronary artery disease. The right choice would be a conservative strategy and no revascularization, unless patients where or get symptomatic. At those days previous to COURAGE and ISCHEMIA trials, that was not so clear, but at least, a third branch for medical treatment should be included. It’s a good example too about how, prejudice plays a role in trial designs, usually taking for granted that an invasive strategy is a better choice for high risk patients.
It would also be interesting to see the percentages and/or raw numbers for the various outcomes. That might shed some light on the question of whether the number of repeat interventions in the three year follow-up was dropped for nefarious reasons.
Excellent points in the comments by Thomas Fariss and David Rind. Another problem with multiple endpoints is that they come in varying degrees of hardness. Total (all cause) mortality is always the hardest endpoint and where I always place most of the weight in judgement of a study. MI is somewhat softer depending on the rigor of the criteria used to make the diagnosis. In 40 years of practice I have seen quite a number of MIs diagnosed on the basis of equivocal ECG changes and tiny bumps in enzymes. Repeat interventions depend upon the thresholds for intervention of the various cardiologists and the willingness of the individual patients to undergo them. These factors make these endpoints considerably softer than total mortality. Granted that large numbers may smooth out some of the differences but we seldom see large numbers of outcomes in these types of studies. I don't understand why stroke is used as an outcome in a study that deals with interventions that only involve the coronary circulation.
How can you be generous and say the authors did nothing wrong when they clearly moved the goalposts, seemingly to shine PCI in a better light?. First impressions from your article are that they had a significant bias at a minimum but also possibility of nefarious intentions.
A health economist - were this public funded healthcare as in the UK - might well look at opportunity costs. What did each procedure cost, and what were the gains in QALYs? Then compare that with, say, building a protected crossing for children outside their high school, or feeding them a free orange a day.
Interesting to consider a comparison between these interventions and best medical therapy. The absolute risk reduction from either surgery or PCI is very small, something most patients (and even doctors) don’t seem to acknowledge.
Really nice discussion of this. Something to consider is that composite endpoints with outcomes that matter differently to patients are problematic to begin with and even more so when you start discussing noninferiority. I might well accept a doubling (or tripling) of the risk of an additional procedure to avoid first having a CABG, but I'd be less likely to accept a doubling of the risk of death.
Great comment and subsequent discussion
The problem is that despite this manipulation that is similar to the one used in EXCEL (repeat revascularization was removed for the first time in CABG vs PCI trials) and a custom made MI definition (not detailed in the trial plan submitted to clinical trials.gov) was used, this paper will certainly be used in future Guidelines recommendations
I think Dr. Mandrola is being too generous with his upper bounds for declaring “nefarious behaviour “. Changing the primary endpoint, even if prespecified, is an absolute no-no. If this was initially planned as a 36 month endpoint study and they did that, there would be an outcry.
This is yet another in a long line of studies by people who push catheters, wanting to justify their reasons for pushing more catheters. There is ample prior data to suggest that revasc (more than other endpoints) is particularly poor for PCI (Vs CABG) as time goes by. It would be entirely predictable (physiology guided PCI or not) to see that drive the primary composite more and more with progressive follow up. And that is in fact what was shown. That they have obfuscated this fact by changing the endpoint (and that CIRC allowed them to do so without more overt indication of this in the abstract and text) is egregious abuse of the scientific process by both the authors and the journal.
And of course, the other elephant in the room is that med Rx alone was not studied. This part is at least excusable, since this study came before Ischemia. But it doesn’t eliminate or resolve the dilemma of “just because we can (revascularize in some way for stable CAD) doesn’t mean we should”.
It’s and excelent example about data manipulattion.. Nowadays, this trial should not be considered appropiate in chronic coronary artery disease. The right choice would be a conservative strategy and no revascularization, unless patients where or get symptomatic. At those days previous to COURAGE and ISCHEMIA trials, that was not so clear, but at least, a third branch for medical treatment should be included. It’s a good example too about how, prejudice plays a role in trial designs, usually taking for granted that an invasive strategy is a better choice for high risk patients.
It would also be interesting to see the percentages and/or raw numbers for the various outcomes. That might shed some light on the question of whether the number of repeat interventions in the three year follow-up was dropped for nefarious reasons.
Excellent points in the comments by Thomas Fariss and David Rind. Another problem with multiple endpoints is that they come in varying degrees of hardness. Total (all cause) mortality is always the hardest endpoint and where I always place most of the weight in judgement of a study. MI is somewhat softer depending on the rigor of the criteria used to make the diagnosis. In 40 years of practice I have seen quite a number of MIs diagnosed on the basis of equivocal ECG changes and tiny bumps in enzymes. Repeat interventions depend upon the thresholds for intervention of the various cardiologists and the willingness of the individual patients to undergo them. These factors make these endpoints considerably softer than total mortality. Granted that large numbers may smooth out some of the differences but we seldom see large numbers of outcomes in these types of studies. I don't understand why stroke is used as an outcome in a study that deals with interventions that only involve the coronary circulation.
Wait, how is this not nefarious??
How can you be generous and say the authors did nothing wrong when they clearly moved the goalposts, seemingly to shine PCI in a better light?. First impressions from your article are that they had a significant bias at a minimum but also possibility of nefarious intentions.
A health economist - were this public funded healthcare as in the UK - might well look at opportunity costs. What did each procedure cost, and what were the gains in QALYs? Then compare that with, say, building a protected crossing for children outside their high school, or feeding them a free orange a day.
Interesting to consider a comparison between these interventions and best medical therapy. The absolute risk reduction from either surgery or PCI is very small, something most patients (and even doctors) don’t seem to acknowledge.
Really nice discussion of this. Something to consider is that composite endpoints with outcomes that matter differently to patients are problematic to begin with and even more so when you start discussing noninferiority. I might well accept a doubling (or tripling) of the risk of an additional procedure to avoid first having a CABG, but I'd be less likely to accept a doubling of the risk of death.
Great analysis. The authors did nothing wrong but I appreciate you pointing out that the writing is misleading.
How is changing endpoints not wrong?