Two weeks ago, I wrote about how brave orthopedic surgeons saved many people from needless knee surgery. Yet cardiologists may be the bravest group of doctors when it comes to use of placebo trials
Another great example of why any procedure which purportedly improves soft (symptom based) outcomes should be tested via sham placebo controlled trials.
I won't pretend that I have any true expertise in cardiac procedures. However, I'm somewhat concerned with the "this settles it" nature of this post for a number of reasons.
First, after reviewing the literature, there are many studies out there using many outcomes. The FDA has become stricter on the use of surrogate outcomes. We know pain is a perception with a psychological component. We have to accept it for what it is even though it often might not have strong physical correlates in patients. We often see very strong placebo effects in psychotropic drug trials but that doesn't mean that the treatment didn't do anything. For invasive procedures with real risks, “hard" outcome measures are important but Phase 3 trials are usually powered for 1 or 2 primary endpoints. Exercise tolerance and myocardial blood flow are interesting measures. The first more subject to placebo effects, the second less prone, but is it a good outcome measure for angina?
"The big reversal trial” you cite was powered to detect a 1 minute increase in exercise tolerance, which was the primary endpoint. There are 6 pre-specified secondary endpoints and I'm concerned that there may be some bias in these measures being reported without the trial having sufficient power to see real differences. The authors use ANOVA to test for differences in both safety and efficacy between the 3 groups. I find this worrisome given the <100 subjects in each test group. The within group variance numbers provided are quite high across measures which makes me want to see the distributions and judge whether certain patients should have been excluded. We need to start looking more at the extent to which individual patients respond to a therapy (or placebo!) and try to understand why. If we can match patients to treatments successfully, we most likely will have new criteria for deciding what works.
To summarize, I respect the author raising this issue and the fact that he raises it with clinical experience. However, TMR is still approved and used in select cases and I don't believe that what is presented here warrants the “had no benefit" conclusion as stated. There are many more "hard” positive study outcomes that need to be addressed first. As for the use of shams and placebos, I think care must be taken. The lawyer Aaron Siri keeps making the claim that “without true placebo controls, we don't know whether a vaccine is safe and effective". This is a blatantly false statement. We have established baselines for most metrics and we know what good and bad look like. Deaths are investigated even if there are more in the placebo arm. Likewise, when we get a speeding ticket, the excuse "I was going the same speed as everyone else” doesn't cut it. Comparisons to standard of care are absolutely reasonable in many cases and the only ethical approach. Medical device data from quality trials should be scrutinized (and in full disclosure, I have had major medical device clients in the past). Outcomes will sometimes be hard to understand and the FDA should ensure proper benefit/risk profiles and proper target patient populations prior to approval.
Setting aside the details of this specific example, the more fundamental problem here is that new elective surgical procedures should require strong proof of effectiveness before being deployed to the public. It seems rather strange to me that chemical/biological interventions that can typically be easily stopped or reversed have a higher standard of proof than surgery, which typically cannot.
5-9% mortality is an INSANE level of risk to treat pain and discomfort. I can't quite even get my head around that, even if we assume patients DID get some objective benefit. What on earth were they even thinking? I had open heart surgery at Cleveland Clinic to repair a bad mitral valve and their mortality rate for that operation was less than 1 in a thousand.
This wasn't the first time in the history of cardiology that this happened: first, there was internal mammary artery ligation.
In the 1950s, this surgery was wildly popular for treating angina pectoris (severe chest pain caused by reduced blood flow to the heart). The theory was that by tying off the internal mammary arteries in the chest, blood would be forced to detour through other pathways and increase the blood supply to the heart muscle. Patients routinely reported massive improvements, fewer chest pains, and less reliance on nitroglycerin.
However, a couple of pioneering researchers—most notably Dr. Leonard Cobb in Seattle (published in 1959) and Dr. E. Grey Dimond in Kansas City (published in 1960)—decided to test the procedure using a sham surgery as a control.
Here is exactly how that historic placebo experiment worked:
• The Setup: Patients were given local anesthesia and had their chests cut open so the arteries were fully exposed.
• The Placebo: Once the chest was open, the surgeon would open a randomized envelope. If it said "ligation," they tied off the arteries. If it said "sham," the surgeon simply sat there for several minutes, did absolutely nothing to the arteries, and then stitched the patient back up.
• The Result: The patients who received only the skin incision reported the exact same dramatic relief from their chest pain as the patients who actually had their arteries tied off. Neither group showed any actual improvement on objective electrocardiogram (ECG) treadmill tests.
Once these double-blind trials revealed that the benefits were entirely a placebo effect, the procedure was abandoned almost overnight.
Very interesting. The internal mammary ligation was a little before my time, but during the late 1960s and early 1970s, its polar opposite was briefly popular. This was direct implantation of one or both internal mammary arteries into the myocardium---known as the Vineberg procedure after the surgeon that pioneered it. I don't remember whether any scientific studies were done but it was soon eclipsed by saphenous vein bypass surgery. It is interesting that the rationale was the direct opposite of that claimed by the advocates of mammary artery ligation.
AMEN, well said and well explained. Unfortunately most elective procedures done today, particularly in orthopedics, were not tested in trials before being unleashed on the public. Sham-controlled trials now exist to challenge practice, not develop it—clearly backwards. Thank you again for the excellent piece.
This is exactly why sham controlled trials matter in procedural medicine. Biological plausibility and subjective symptom improvement are not enough because placebo responses in invasive interventions can be remarkably strong, and TMR became a powerful example of how rigorous blinding protects patients from unnecessary risk despite convincing early observational data.
Unfortunately, our medical systems fail to recognize that a "placebo effect" is often (always?) an effect not on the patient, but on the doctor. The surgeon perceives that the operation was a success, when in fact there was a placebo effect on the surgeon. When we examine the mysteries of placebo effect without considering the placebo effect on the physician, we easily misunderstand.
I graduated from medical school back when God was a baby. For a period of time, I worked in Franz Halberg's chronobiology lab at the University of Minnesota, and later went into Family Practice. There are two components to evaluating anything in medicine:
1. Empirical observations where events seemed to be tied together.
All those mice gave Halberg the basis for his chronobiology theory, which eventually became the worldwide standard. What are you doing now? I am retired and living north of Boston.
I suspect he was. Perhaps I should come and visit you to discuss in more detail! We have homes in Greece, Boston, Minnesota, Mexico, and Florida, and I think Hawaii would make a good addition!
If you could convince insurance companies to pay the same for sham procedures as for real procedures, more physicians could do safe and effective sham procedures without losing income. Just think of the new horizons in healthcare that would open up… Sham surgery, sham radiation, sham drug protocols, sham vaccinations, and more, all reimbursed at the same rate as real interventions but without patient harm.
The childhood vaccine schedule has never been tested "with a proper sham control arm." Or even with a comparison of a vaccinated population vs an unvaccinated population.
Thanks for the shout-out for what is still one of the most important and impactful trials that I've been involved in. One minor correction... the trial was actually enrolled between 1998-1999 and was scheduled for a LBCT presentation at TCT 2001, which was interrupted by the events on 9/11. After that, we got a little distracted for a while before the paper was finally written and published.
Or some patients who love doctors who do surgery found it helpful! I have patients who are convinced surgery is the answer. I have stopped trying to tell them it isn't so! As a PT, I do my best and when someone swears they need surgery, I walk into my office, bang my head at the wall and tell the patient, I will see you postop. It's maddening but the way people are.
As see more in your space you are so right - proper technique and rehab can help a lot. Even arthritis - benefits from movement. Have 2-patients for whom spine surgeons insisted they needed repair but it was Parkinson’s though.
Another great example of why any procedure which purportedly improves soft (symptom based) outcomes should be tested via sham placebo controlled trials.
I won't pretend that I have any true expertise in cardiac procedures. However, I'm somewhat concerned with the "this settles it" nature of this post for a number of reasons.
First, after reviewing the literature, there are many studies out there using many outcomes. The FDA has become stricter on the use of surrogate outcomes. We know pain is a perception with a psychological component. We have to accept it for what it is even though it often might not have strong physical correlates in patients. We often see very strong placebo effects in psychotropic drug trials but that doesn't mean that the treatment didn't do anything. For invasive procedures with real risks, “hard" outcome measures are important but Phase 3 trials are usually powered for 1 or 2 primary endpoints. Exercise tolerance and myocardial blood flow are interesting measures. The first more subject to placebo effects, the second less prone, but is it a good outcome measure for angina?
"The big reversal trial” you cite was powered to detect a 1 minute increase in exercise tolerance, which was the primary endpoint. There are 6 pre-specified secondary endpoints and I'm concerned that there may be some bias in these measures being reported without the trial having sufficient power to see real differences. The authors use ANOVA to test for differences in both safety and efficacy between the 3 groups. I find this worrisome given the <100 subjects in each test group. The within group variance numbers provided are quite high across measures which makes me want to see the distributions and judge whether certain patients should have been excluded. We need to start looking more at the extent to which individual patients respond to a therapy (or placebo!) and try to understand why. If we can match patients to treatments successfully, we most likely will have new criteria for deciding what works.
To summarize, I respect the author raising this issue and the fact that he raises it with clinical experience. However, TMR is still approved and used in select cases and I don't believe that what is presented here warrants the “had no benefit" conclusion as stated. There are many more "hard” positive study outcomes that need to be addressed first. As for the use of shams and placebos, I think care must be taken. The lawyer Aaron Siri keeps making the claim that “without true placebo controls, we don't know whether a vaccine is safe and effective". This is a blatantly false statement. We have established baselines for most metrics and we know what good and bad look like. Deaths are investigated even if there are more in the placebo arm. Likewise, when we get a speeding ticket, the excuse "I was going the same speed as everyone else” doesn't cut it. Comparisons to standard of care are absolutely reasonable in many cases and the only ethical approach. Medical device data from quality trials should be scrutinized (and in full disclosure, I have had major medical device clients in the past). Outcomes will sometimes be hard to understand and the FDA should ensure proper benefit/risk profiles and proper target patient populations prior to approval.
Setting aside the details of this specific example, the more fundamental problem here is that new elective surgical procedures should require strong proof of effectiveness before being deployed to the public. It seems rather strange to me that chemical/biological interventions that can typically be easily stopped or reversed have a higher standard of proof than surgery, which typically cannot.
No argument from me there!
5-9% mortality is an INSANE level of risk to treat pain and discomfort. I can't quite even get my head around that, even if we assume patients DID get some objective benefit. What on earth were they even thinking? I had open heart surgery at Cleveland Clinic to repair a bad mitral valve and their mortality rate for that operation was less than 1 in a thousand.
Triumph of technology over reason.
This wasn't the first time in the history of cardiology that this happened: first, there was internal mammary artery ligation.
In the 1950s, this surgery was wildly popular for treating angina pectoris (severe chest pain caused by reduced blood flow to the heart). The theory was that by tying off the internal mammary arteries in the chest, blood would be forced to detour through other pathways and increase the blood supply to the heart muscle. Patients routinely reported massive improvements, fewer chest pains, and less reliance on nitroglycerin.
However, a couple of pioneering researchers—most notably Dr. Leonard Cobb in Seattle (published in 1959) and Dr. E. Grey Dimond in Kansas City (published in 1960)—decided to test the procedure using a sham surgery as a control.
Here is exactly how that historic placebo experiment worked:
• The Setup: Patients were given local anesthesia and had their chests cut open so the arteries were fully exposed.
• The Placebo: Once the chest was open, the surgeon would open a randomized envelope. If it said "ligation," they tied off the arteries. If it said "sham," the surgeon simply sat there for several minutes, did absolutely nothing to the arteries, and then stitched the patient back up.
• The Result: The patients who received only the skin incision reported the exact same dramatic relief from their chest pain as the patients who actually had their arteries tied off. Neither group showed any actual improvement on objective electrocardiogram (ECG) treadmill tests.
Once these double-blind trials revealed that the benefits were entirely a placebo effect, the procedure was abandoned almost overnight.
Very interesting. The internal mammary ligation was a little before my time, but during the late 1960s and early 1970s, its polar opposite was briefly popular. This was direct implantation of one or both internal mammary arteries into the myocardium---known as the Vineberg procedure after the surgeon that pioneered it. I don't remember whether any scientific studies were done but it was soon eclipsed by saphenous vein bypass surgery. It is interesting that the rationale was the direct opposite of that claimed by the advocates of mammary artery ligation.
AMEN, well said and well explained. Unfortunately most elective procedures done today, particularly in orthopedics, were not tested in trials before being unleashed on the public. Sham-controlled trials now exist to challenge practice, not develop it—clearly backwards. Thank you again for the excellent piece.
This is exactly why sham controlled trials matter in procedural medicine. Biological plausibility and subjective symptom improvement are not enough because placebo responses in invasive interventions can be remarkably strong, and TMR became a powerful example of how rigorous blinding protects patients from unnecessary risk despite convincing early observational data.
Unfortunately, our medical systems fail to recognize that a "placebo effect" is often (always?) an effect not on the patient, but on the doctor. The surgeon perceives that the operation was a success, when in fact there was a placebo effect on the surgeon. When we examine the mysteries of placebo effect without considering the placebo effect on the physician, we easily misunderstand.
I graduated from medical school back when God was a baby. For a period of time, I worked in Franz Halberg's chronobiology lab at the University of Minnesota, and later went into Family Practice. There are two components to evaluating anything in medicine:
1. Empirical observations where events seemed to be tied together.
2. Placebo controlled double blind trials.
Both are equally important.
I worked in that Halberg lab too...what ever became of his work? Did we sacrifice those many thousands of mice in vain?
All those mice gave Halberg the basis for his chronobiology theory, which eventually became the worldwide standard. What are you doing now? I am retired and living north of Boston.
Still practicing in Hawaii
I often thought Halberg was studying us,given that we often went without sleep to carry those studies out, sometimes for a whole week .
I suspect he was. Perhaps I should come and visit you to discuss in more detail! We have homes in Greece, Boston, Minnesota, Mexico, and Florida, and I think Hawaii would make a good addition!
If you could convince insurance companies to pay the same for sham procedures as for real procedures, more physicians could do safe and effective sham procedures without losing income. Just think of the new horizons in healthcare that would open up… Sham surgery, sham radiation, sham drug protocols, sham vaccinations, and more, all reimbursed at the same rate as real interventions but without patient harm.
The childhood vaccine schedule has never been tested "with a proper sham control arm." Or even with a comparison of a vaccinated population vs an unvaccinated population.
Thanks for the shout-out for what is still one of the most important and impactful trials that I've been involved in. One minor correction... the trial was actually enrolled between 1998-1999 and was scheduled for a LBCT presentation at TCT 2001, which was interrupted by the events on 9/11. After that, we got a little distracted for a while before the paper was finally written and published.
The only sham-controlled study of a device for mitral regurgitation (REDUCE-FMR) is now being followed up by a larger study - EMPOWER.
Surprising how something this dangerous got approved. Thankfully some patient loving physicians decided to make sure it was helpful.
Or some patients who love doctors who do surgery found it helpful! I have patients who are convinced surgery is the answer. I have stopped trying to tell them it isn't so! As a PT, I do my best and when someone swears they need surgery, I walk into my office, bang my head at the wall and tell the patient, I will see you postop. It's maddening but the way people are.
One more thing. The surgeons are happy to oblige even if the patient is improving.
As see more in your space you are so right - proper technique and rehab can help a lot. Even arthritis - benefits from movement. Have 2-patients for whom spine surgeons insisted they needed repair but it was Parkinson’s though.
Interesting