How Doctors are Fooled
For the next few weeks I will explore the need for having a proper control arm in medical studies
Two major studies presented this weekend at the American College of Cardiology meeting may herald new ways to treat two common cardiac conditions. Yet the foundation of these new approaches rest on dubious evidence.
This starts a multi-part series exploring a common theme in these studies—the control arm.
This first week I will tell a story from my first years of private practice.
Next week, I will explore a study of treating patients with valvular heart disease without open-heart surgery. The authors report the results as “positive,” but the control arm figures in how we interpret the results.
The following week, I will write about the use of an older medicine for a new condition, which may or may not exist, depending on how you think about control arms in studies.
During the first week of my first job as a doctor, at Audubon Hospital, on Poplar Level Road in Louisville, KY, I read the chart of a man who was Day 2 after TMR.
I asked Kim, our rounding nurse, “what is TMR?”
She said, “it’s transmyocardial revascularization—a heart surgeon opens the chest and lasers many holes in the ventricle. It’s done to relieve angina. You’ve never heard of it?”
TMR was a big deal. It required open chest surgery. It took patients many days to recover.
I was baffled. Just weeks before I had finished a top training program. During those years, I had moonlighted at a busy private cardiology group. I had never heard of TMR.
But at Audubon hospital, up on the hill on Poplar Level Road, TMR was common. I asked my partners about it. “John, I know, it seems strange, but patients feel better after it; their angina gets better. We see it.”
I shook my head and carried on. Google did not exist. There were no cell phones to look up studies. There were no private message groups to pose a question to.
And…
It turns out that in the NEJM in 1999, a study comparing TMR to medical therapy in patients with severe angina due to coronary disease found the following:
In the first year of follow-up, 2 percent of patients assigned to undergo TMR were hospitalized because of unstable angina, as compared with 69 percent of patients assigned to medical treatment
So, for years, at Audubon hospital and many other hospitals, patients with severe coronary disease and chest pain had holes drilled into their hearts to improve blood flow. All of these patients faced a long recovery; some had serious complications and even died from the procedure.
Then. Boom.
Nine years later, a group of investigators finally decided to study TMR in a proper way: they included a group of patients who had surgery but did not have holes placed in the heart. We call this a placebo or sham control.
These are charts from the paper.
The pictures clearly show no difference in exercise capacity or angina in any of the three groups. High-dose laser, low-dose laser, or no laser all had the same results.
Opening the chest of patients and lasering holes in the heart did not work any better than a sham.
And that was it for TMR. It was all placebo effect.
We never did TMR again. We sort of forgot about it.
How Were We Fooled?
Earlier studies of TMR against medical therapy plus our experience with patients who felt better after TMR plus the eminence of the surgeons who advocated for the procedure had utterly fooled us.
This happens when you compare a huge procedure (surgery) against something small (medicines); patients tend to feel better after a bigger procedure.
Placebo effects are super-complicated. But they are real. It’s why drug trials use a placebo pill.
This history of medicine is replete with examples wherein a proper control arm reverses a practice. Brian Olshansky, MD, wrote this wonderful review of placebo and nocebo effects—it is open access.
Finally…
It might seem unethical, bordering on barbaric, to operate on a human and not do the surgery. The flip side, however, is the ethics of doing ineffective invasive surgeries on hundreds or thousands of humans—for years.
Tune in next week and I will discuss the big study on treating valvular heart disease without open-heart surgery. Control arms take center stage.
placebo still flaws me periodically. & it has had different definitions. i heart (<3) sham surgical trials (always the knee comes to mind); this one is new on me. incredible!
but if this is what it takes for us to tell the difference, just what is going on? how well has placebo been studied for actually directing of the bodies healing resources? i actually dont know. feel like i should. is it qualia Ltd? ill add it to the list. _JC
Keep up the good work. I’ve learned many things about critical appraisal and EBM based on this and your Medscape TWIC articles.
It would seem, after trials like Orbita and the early false hope of renal denervation studies, that sham control would be mandatory in trial design involving invasive procedures. Sadly, that does not yet seem to be the case.
OTOH, I would submit that the rigor of the control arm should be judged commensurately to the strength of the endpoints. I accept the benefit of TEER in primary and appropriately-selected secondary MR patients for that reason.
But no sham control, AND a benefit only in QOL? Well, that is a different story.