Two weeks ago, I wrote about how brave orthopedic surgeons saved many people from needless knee surgery. Yet cardiologists may be the bravest group of doctors when it comes to use of placebo trials
The subjects wore headphones playing music, but we still said loudly & per protocol “We’re turning on the laser now.”
The IRB (institutional review board, the hospital committee deciding whether the study was ethical) labored over my application to do this trial for *over a year.* Finally they approved, and we began enrolling. At the first interim review, the message was essentially, “Wait, we are concerned that you are performing sham procedures,” and we had to pause again.
The power of placebo seems to be especially strong when it comes with the gee-whiz factor of a laser or a stem cell.
Of course, sham procedures make scientific sense and can support or reject an invasive treatment's efficacy. I'm sure our profession is riddled with untested or inadequately tested treatments that are regarded as established. Clearly, individuals who are in a procedural sham group have a risk of direct harm, different from a conventional placebo cohort that is left alone. For this reason, the informed consent process needs to be robust, perhaps with the involvement of independent professionals who serve as neutral participants. Investigators may harbor unconscious or even overt bias with regard to recruitment. Fidelity to medical ethics is essential.
Placebo sham surgery is not needed for procedures where the outcome is quite visible to the surgeon and patient alike. Example: sham macular hole surgery where the central vision can recover to 20/20 vision and closure of the previously visible macular hole would have been unethical once cases were reported to be successful. OTOH natural history studies were subsequently done that showed that not all macular holes need surgery, so there was that difficulty that needed to be analyzed after the surgery was pioneered by Kelly and Wendel. There is no one size fits all for trials in Medicine, therein lies the problem.
We must be careful not to compare apples to oranges and decide fruit is bad.
Catheter based "TMLR" is not equivalent to surgical TMLR as Marty Leon himself noted in the quoted 2005 article, dismissing clear surgical advantages in TMLR lesion localization, density, and depth, and ascribing published improved clinical outcomes to a sort of no-pain no-gain placebo philosophy. Surgical TMLR, whether transthoracic minimally invasive or transsternal seeks full transmural lesion creation, while LV perforation is carefully avoided in the catheter-based "percutaneous" approach. Of note was a CVA risk inherent in manipulations across the aortic valve.
More difficult to explain are published short- and mid-term clinical improvements in studies comparing concomitant CABG-TMLR and CABG alone as well as evidence of improved vascularity found with perioperative PET imaging. Placebo differences would be neutralized since both arms are surgical, usually transsternal.
My point is that whether one is a "believer" or not, each approach must be assessed on its own.
Placebo-controlled trials remain the gold standard for surgical interventions. Ethical considerations must balance patient safety with the need for rigorous evidence. Verification of therapeutic benefit is essential.
Another great example of why any procedure which purportedly improves soft (symptom based) outcomes should be tested via sham placebo controlled trials.
I won't pretend that I have any true expertise in cardiac procedures. However, I'm somewhat concerned with the "this settles it" nature of this post for a number of reasons.
First, after reviewing the literature, there are many studies out there using many outcomes. The FDA has become stricter on the use of surrogate outcomes. We know pain is a perception with a psychological component. We have to accept it for what it is even though it often might not have strong physical correlates in patients. We often see very strong placebo effects in psychotropic drug trials but that doesn't mean that the treatment didn't do anything. For invasive procedures with real risks, “hard" outcome measures are important but Phase 3 trials are usually powered for 1 or 2 primary endpoints. Exercise tolerance and myocardial blood flow are interesting measures. The first more subject to placebo effects, the second less prone, but is it a good outcome measure for angina?
"The big reversal trial” you cite was powered to detect a 1 minute increase in exercise tolerance, which was the primary endpoint. There are 6 pre-specified secondary endpoints and I'm concerned that there may be some bias in these measures being reported without the trial having sufficient power to see real differences. The authors use ANOVA to test for differences in both safety and efficacy between the 3 groups. I find this worrisome given the <100 subjects in each test group. The within group variance numbers provided are quite high across measures which makes me want to see the distributions and judge whether certain patients should have been excluded. We need to start looking more at the extent to which individual patients respond to a therapy (or placebo!) and try to understand why. If we can match patients to treatments successfully, we most likely will have new criteria for deciding what works.
To summarize, I respect the author raising this issue and the fact that he raises it with clinical experience. However, TMR is still approved and used in select cases and I don't believe that what is presented here warrants the “had no benefit" conclusion as stated. There are many more "hard” positive study outcomes that need to be addressed first. As for the use of shams and placebos, I think care must be taken. The lawyer Aaron Siri keeps making the claim that “without true placebo controls, we don't know whether a vaccine is safe and effective". This is a blatantly false statement. We have established baselines for most metrics and we know what good and bad look like. Deaths are investigated even if there are more in the placebo arm. Likewise, when we get a speeding ticket, the excuse "I was going the same speed as everyone else” doesn't cut it. Comparisons to standard of care are absolutely reasonable in many cases and the only ethical approach. Medical device data from quality trials should be scrutinized (and in full disclosure, I have had major medical device clients in the past). Outcomes will sometimes be hard to understand and the FDA should ensure proper benefit/risk profiles and proper target patient populations prior to approval.
I appreciate your thorough argument. But what do you mean "The within group variance numbers provided are quite high across measures which makes me want to see the distributions and judge whether certain patients should have been excluded." We can't exclude patients from the analysis that just don't match with our expectations (I assume it's not some documentation error). I can imagine here a more fitting or rigorous statistical analysis but somehow I'm not sure I understood you correctly about the exclusion part.
Thanks for your comment/question. We were provided with the standard deviation for the exercise tolerance primary outcome and other measures as well. The baseline for the 3 groups is roughly between 6 and 6.5 minutes with a standard deviation of about 2.5 minutes. That's quite a spread considering the experiment was powered to detect a 1 minute difference in exercise duration. Whenever we test means using an ANOVA, we're making assumptions about the underlying distributions. I'm curious if we may see some patients improving but other patients performing worse in the post period. Looking for an increase in exercise tolerance of 1 minute seems odd to me given that some other studies have reported significant decreases in hospital admissions or increases in survival. My comment on inclusion was inclusion of patients in the trial, not the inclusion of their data once they were in the trial. The only time that would be an issue is if they suffered a comorbidity or other event once the trial was started. I'm expressing curiosity more than anything else but there are many ways to get to a mean and care must be taken when we say "no one got any benefit" when that isn't really what we're testing.
Setting aside the details of this specific example, the more fundamental problem here is that new elective surgical procedures should require strong proof of effectiveness before being deployed to the public. It seems rather strange to me that chemical/biological interventions that can typically be easily stopped or reversed have a higher standard of proof than surgery, which typically cannot.
5-9% mortality is an INSANE level of risk to treat pain and discomfort. I can't quite even get my head around that, even if we assume patients DID get some objective benefit. What on earth were they even thinking? I had open heart surgery at Cleveland Clinic to repair a bad mitral valve and their mortality rate for that operation was less than 1 in a thousand.
But the patients have to be correctly informed about that. Somehow I doubt that in this particular case people truly understood the risk of this procedure.
Aussie Med Student, you are the first medically-trained person I've encountered who appears to understand just what that bland appellation "quality of life" truly means.
If you're still a student, I sincerely hope that you ace your finals and residency, are wildly successful with a meteoric rise in well-earned prestige and are destined for a prominent and highly influential career.
This wasn't the first time in the history of cardiology that this happened: first, there was internal mammary artery ligation.
In the 1950s, this surgery was wildly popular for treating angina pectoris (severe chest pain caused by reduced blood flow to the heart). The theory was that by tying off the internal mammary arteries in the chest, blood would be forced to detour through other pathways and increase the blood supply to the heart muscle. Patients routinely reported massive improvements, fewer chest pains, and less reliance on nitroglycerin.
However, a couple of pioneering researchers—most notably Dr. Leonard Cobb in Seattle (published in 1959) and Dr. E. Grey Dimond in Kansas City (published in 1960)—decided to test the procedure using a sham surgery as a control.
Here is exactly how that historic placebo experiment worked:
• The Setup: Patients were given local anesthesia and had their chests cut open so the arteries were fully exposed.
• The Placebo: Once the chest was open, the surgeon would open a randomized envelope. If it said "ligation," they tied off the arteries. If it said "sham," the surgeon simply sat there for several minutes, did absolutely nothing to the arteries, and then stitched the patient back up.
• The Result: The patients who received only the skin incision reported the exact same dramatic relief from their chest pain as the patients who actually had their arteries tied off. Neither group showed any actual improvement on objective electrocardiogram (ECG) treadmill tests.
Once these double-blind trials revealed that the benefits were entirely a placebo effect, the procedure was abandoned almost overnight.
Very interesting. The internal mammary ligation was a little before my time, but during the late 1960s and early 1970s, its polar opposite was briefly popular. This was direct implantation of one or both internal mammary arteries into the myocardium---known as the Vineberg procedure after the surgeon that pioneered it. I don't remember whether any scientific studies were done but it was soon eclipsed by saphenous vein bypass surgery. It is interesting that the rationale was the direct opposite of that claimed by the advocates of mammary artery ligation.
AMEN, well said and well explained. Unfortunately most elective procedures done today, particularly in orthopedics, were not tested in trials before being unleashed on the public. Sham-controlled trials now exist to challenge practice, not develop it—clearly backwards. Thank you again for the excellent piece.
This is exactly why sham controlled trials matter in procedural medicine. Biological plausibility and subjective symptom improvement are not enough because placebo responses in invasive interventions can be remarkably strong, and TMR became a powerful example of how rigorous blinding protects patients from unnecessary risk despite convincing early observational data.
Unfortunately, our medical systems fail to recognize that a "placebo effect" is often (always?) an effect not on the patient, but on the doctor. The surgeon perceives that the operation was a success, when in fact there was a placebo effect on the surgeon. When we examine the mysteries of placebo effect without considering the placebo effect on the physician, we easily misunderstand.
I graduated from medical school back when God was a baby. For a period of time, I worked in Franz Halberg's chronobiology lab at the University of Minnesota, and later went into Family Practice. There are two components to evaluating anything in medicine:
1. Empirical observations where events seemed to be tied together.
All those mice gave Halberg the basis for his chronobiology theory, which eventually became the worldwide standard. What are you doing now? I am retired and living north of Boston.
I suspect he was. Perhaps I should come and visit you to discuss in more detail! We have homes in Greece, Boston, Minnesota, Mexico, and Florida, and I think Hawaii would make a good addition!
If you could convince insurance companies to pay the same for sham procedures as for real procedures, more physicians could do safe and effective sham procedures without losing income. Just think of the new horizons in healthcare that would open up… Sham surgery, sham radiation, sham drug protocols, sham vaccinations, and more, all reimbursed at the same rate as real interventions but without patient harm.
The subjects wore headphones playing music, but we still said loudly & per protocol “We’re turning on the laser now.”
The IRB (institutional review board, the hospital committee deciding whether the study was ethical) labored over my application to do this trial for *over a year.* Finally they approved, and we began enrolling. At the first interim review, the message was essentially, “Wait, we are concerned that you are performing sham procedures,” and we had to pause again.
The power of placebo seems to be especially strong when it comes with the gee-whiz factor of a laser or a stem cell.
Of course, sham procedures make scientific sense and can support or reject an invasive treatment's efficacy. I'm sure our profession is riddled with untested or inadequately tested treatments that are regarded as established. Clearly, individuals who are in a procedural sham group have a risk of direct harm, different from a conventional placebo cohort that is left alone. For this reason, the informed consent process needs to be robust, perhaps with the involvement of independent professionals who serve as neutral participants. Investigators may harbor unconscious or even overt bias with regard to recruitment. Fidelity to medical ethics is essential.
Almost 10% mortality? If so then the benefits are irrelevant, no elective surgery should be allowed with 10% mortality, surely! That's enormous.
Placebo sham surgery is not needed for procedures where the outcome is quite visible to the surgeon and patient alike. Example: sham macular hole surgery where the central vision can recover to 20/20 vision and closure of the previously visible macular hole would have been unethical once cases were reported to be successful. OTOH natural history studies were subsequently done that showed that not all macular holes need surgery, so there was that difficulty that needed to be analyzed after the surgery was pioneered by Kelly and Wendel. There is no one size fits all for trials in Medicine, therein lies the problem.
We must be careful not to compare apples to oranges and decide fruit is bad.
Catheter based "TMLR" is not equivalent to surgical TMLR as Marty Leon himself noted in the quoted 2005 article, dismissing clear surgical advantages in TMLR lesion localization, density, and depth, and ascribing published improved clinical outcomes to a sort of no-pain no-gain placebo philosophy. Surgical TMLR, whether transthoracic minimally invasive or transsternal seeks full transmural lesion creation, while LV perforation is carefully avoided in the catheter-based "percutaneous" approach. Of note was a CVA risk inherent in manipulations across the aortic valve.
More difficult to explain are published short- and mid-term clinical improvements in studies comparing concomitant CABG-TMLR and CABG alone as well as evidence of improved vascularity found with perioperative PET imaging. Placebo differences would be neutralized since both arms are surgical, usually transsternal.
My point is that whether one is a "believer" or not, each approach must be assessed on its own.
Placebo-controlled trials remain the gold standard for surgical interventions. Ethical considerations must balance patient safety with the need for rigorous evidence. Verification of therapeutic benefit is essential.
Another great example of why any procedure which purportedly improves soft (symptom based) outcomes should be tested via sham placebo controlled trials.
I won't pretend that I have any true expertise in cardiac procedures. However, I'm somewhat concerned with the "this settles it" nature of this post for a number of reasons.
First, after reviewing the literature, there are many studies out there using many outcomes. The FDA has become stricter on the use of surrogate outcomes. We know pain is a perception with a psychological component. We have to accept it for what it is even though it often might not have strong physical correlates in patients. We often see very strong placebo effects in psychotropic drug trials but that doesn't mean that the treatment didn't do anything. For invasive procedures with real risks, “hard" outcome measures are important but Phase 3 trials are usually powered for 1 or 2 primary endpoints. Exercise tolerance and myocardial blood flow are interesting measures. The first more subject to placebo effects, the second less prone, but is it a good outcome measure for angina?
"The big reversal trial” you cite was powered to detect a 1 minute increase in exercise tolerance, which was the primary endpoint. There are 6 pre-specified secondary endpoints and I'm concerned that there may be some bias in these measures being reported without the trial having sufficient power to see real differences. The authors use ANOVA to test for differences in both safety and efficacy between the 3 groups. I find this worrisome given the <100 subjects in each test group. The within group variance numbers provided are quite high across measures which makes me want to see the distributions and judge whether certain patients should have been excluded. We need to start looking more at the extent to which individual patients respond to a therapy (or placebo!) and try to understand why. If we can match patients to treatments successfully, we most likely will have new criteria for deciding what works.
To summarize, I respect the author raising this issue and the fact that he raises it with clinical experience. However, TMR is still approved and used in select cases and I don't believe that what is presented here warrants the “had no benefit" conclusion as stated. There are many more "hard” positive study outcomes that need to be addressed first. As for the use of shams and placebos, I think care must be taken. The lawyer Aaron Siri keeps making the claim that “without true placebo controls, we don't know whether a vaccine is safe and effective". This is a blatantly false statement. We have established baselines for most metrics and we know what good and bad look like. Deaths are investigated even if there are more in the placebo arm. Likewise, when we get a speeding ticket, the excuse "I was going the same speed as everyone else” doesn't cut it. Comparisons to standard of care are absolutely reasonable in many cases and the only ethical approach. Medical device data from quality trials should be scrutinized (and in full disclosure, I have had major medical device clients in the past). Outcomes will sometimes be hard to understand and the FDA should ensure proper benefit/risk profiles and proper target patient populations prior to approval.
I appreciate your thorough argument. But what do you mean "The within group variance numbers provided are quite high across measures which makes me want to see the distributions and judge whether certain patients should have been excluded." We can't exclude patients from the analysis that just don't match with our expectations (I assume it's not some documentation error). I can imagine here a more fitting or rigorous statistical analysis but somehow I'm not sure I understood you correctly about the exclusion part.
Thanks for your comment/question. We were provided with the standard deviation for the exercise tolerance primary outcome and other measures as well. The baseline for the 3 groups is roughly between 6 and 6.5 minutes with a standard deviation of about 2.5 minutes. That's quite a spread considering the experiment was powered to detect a 1 minute difference in exercise duration. Whenever we test means using an ANOVA, we're making assumptions about the underlying distributions. I'm curious if we may see some patients improving but other patients performing worse in the post period. Looking for an increase in exercise tolerance of 1 minute seems odd to me given that some other studies have reported significant decreases in hospital admissions or increases in survival. My comment on inclusion was inclusion of patients in the trial, not the inclusion of their data once they were in the trial. The only time that would be an issue is if they suffered a comorbidity or other event once the trial was started. I'm expressing curiosity more than anything else but there are many ways to get to a mean and care must be taken when we say "no one got any benefit" when that isn't really what we're testing.
Setting aside the details of this specific example, the more fundamental problem here is that new elective surgical procedures should require strong proof of effectiveness before being deployed to the public. It seems rather strange to me that chemical/biological interventions that can typically be easily stopped or reversed have a higher standard of proof than surgery, which typically cannot.
No argument from me there!
5-9% mortality is an INSANE level of risk to treat pain and discomfort. I can't quite even get my head around that, even if we assume patients DID get some objective benefit. What on earth were they even thinking? I had open heart surgery at Cleveland Clinic to repair a bad mitral valve and their mortality rate for that operation was less than 1 in a thousand.
If the morbidity is such that the patient would rather be dead than continue to suffer, then I'd tolerate high mortality rates even for a placebo ;)
Are you arguing for a third arm? Treatment; control; euthanasia?
But the patients have to be correctly informed about that. Somehow I doubt that in this particular case people truly understood the risk of this procedure.
Aussie Med Student, you are the first medically-trained person I've encountered who appears to understand just what that bland appellation "quality of life" truly means.
If you're still a student, I sincerely hope that you ace your finals and residency, are wildly successful with a meteoric rise in well-earned prestige and are destined for a prominent and highly influential career.
Godspeed.
Triumph of technology over reason.
This wasn't the first time in the history of cardiology that this happened: first, there was internal mammary artery ligation.
In the 1950s, this surgery was wildly popular for treating angina pectoris (severe chest pain caused by reduced blood flow to the heart). The theory was that by tying off the internal mammary arteries in the chest, blood would be forced to detour through other pathways and increase the blood supply to the heart muscle. Patients routinely reported massive improvements, fewer chest pains, and less reliance on nitroglycerin.
However, a couple of pioneering researchers—most notably Dr. Leonard Cobb in Seattle (published in 1959) and Dr. E. Grey Dimond in Kansas City (published in 1960)—decided to test the procedure using a sham surgery as a control.
Here is exactly how that historic placebo experiment worked:
• The Setup: Patients were given local anesthesia and had their chests cut open so the arteries were fully exposed.
• The Placebo: Once the chest was open, the surgeon would open a randomized envelope. If it said "ligation," they tied off the arteries. If it said "sham," the surgeon simply sat there for several minutes, did absolutely nothing to the arteries, and then stitched the patient back up.
• The Result: The patients who received only the skin incision reported the exact same dramatic relief from their chest pain as the patients who actually had their arteries tied off. Neither group showed any actual improvement on objective electrocardiogram (ECG) treadmill tests.
Once these double-blind trials revealed that the benefits were entirely a placebo effect, the procedure was abandoned almost overnight.
Very interesting. The internal mammary ligation was a little before my time, but during the late 1960s and early 1970s, its polar opposite was briefly popular. This was direct implantation of one or both internal mammary arteries into the myocardium---known as the Vineberg procedure after the surgeon that pioneered it. I don't remember whether any scientific studies were done but it was soon eclipsed by saphenous vein bypass surgery. It is interesting that the rationale was the direct opposite of that claimed by the advocates of mammary artery ligation.
AMEN, well said and well explained. Unfortunately most elective procedures done today, particularly in orthopedics, were not tested in trials before being unleashed on the public. Sham-controlled trials now exist to challenge practice, not develop it—clearly backwards. Thank you again for the excellent piece.
This is exactly why sham controlled trials matter in procedural medicine. Biological plausibility and subjective symptom improvement are not enough because placebo responses in invasive interventions can be remarkably strong, and TMR became a powerful example of how rigorous blinding protects patients from unnecessary risk despite convincing early observational data.
Unfortunately, our medical systems fail to recognize that a "placebo effect" is often (always?) an effect not on the patient, but on the doctor. The surgeon perceives that the operation was a success, when in fact there was a placebo effect on the surgeon. When we examine the mysteries of placebo effect without considering the placebo effect on the physician, we easily misunderstand.
I graduated from medical school back when God was a baby. For a period of time, I worked in Franz Halberg's chronobiology lab at the University of Minnesota, and later went into Family Practice. There are two components to evaluating anything in medicine:
1. Empirical observations where events seemed to be tied together.
2. Placebo controlled double blind trials.
Both are equally important.
I worked in that Halberg lab too...what ever became of his work? Did we sacrifice those many thousands of mice in vain?
All those mice gave Halberg the basis for his chronobiology theory, which eventually became the worldwide standard. What are you doing now? I am retired and living north of Boston.
Still practicing in Hawaii
I often thought Halberg was studying us,given that we often went without sleep to carry those studies out, sometimes for a whole week .
I suspect he was. Perhaps I should come and visit you to discuss in more detail! We have homes in Greece, Boston, Minnesota, Mexico, and Florida, and I think Hawaii would make a good addition!
If you could convince insurance companies to pay the same for sham procedures as for real procedures, more physicians could do safe and effective sham procedures without losing income. Just think of the new horizons in healthcare that would open up… Sham surgery, sham radiation, sham drug protocols, sham vaccinations, and more, all reimbursed at the same rate as real interventions but without patient harm.