The Importance of Proper Control Arms

The study-of-the-week considers a well-viewed paper on placebo effects for common psychiatric conditions. The lessons transcend psychiatry.

This is a story about placebo effects. I raise the issue because the prominent journal JAMA-Psychiatry published a paper purporting to show that placebos have great effects in many psychiatric conditions.

The erroneous conclusion provides excellent teaching points.

First a Thought Experiment

Say I gave 100 cyclists a red supplement designed to increase muscle contraction and increase pedal power. That’s all I did. No control arm. Then I measured perceived benefits from the cyclist. I might tell you that 50% of cyclists felt stronger.

You’d laugh. You’d rightly say that you can’t make causal conclusions about the red supplement because there was no comparison arm. You want to know how the red pill compares with another red pill with nothing in it. But. It’s actually more complicated than that. Stay tuned.

The JAMA-Psychiatry Review Paper

The authors systematically reviewed 90 randomized placebo-controlled trials that included nearly 10,000 patients who had various psychiatric diagnoses.

They then extracted the placebo effect. To do this, they measured the pre- and post-placebo effect size per the diagnosis it was studied for.

The authors reported that most psychiatric disorders, especially depression and generalized anxiety disorder, had substantial placebo effects.

Here is their conclusion:

Comments

This gives the impression that placebos could be used to treat depression or anxiety or any number of psychiatric disorders.

This is wrong. And the error is instructive. (Though they sort of hedge in that last phrase of the last sentence.)

The problem with this placebo-effect study is that you need a non-placebo arm to assess the placebo-effect. You need an arm that was in the study but received no placebo.

One reason why is that diseases change over time. Not only psychiatric diseases, but blood pressure, chest pain, atrial fibrillation, etc.

Patients enrolled in trials are often at the pinnacle of their disease. The concept of regression to the mean holds that simply doing nothing may allow the patient to get better.

Another reason you need non-placebo arms is that simply being on a tablet can cause a wide variety of effects in the body.

The Best Example of a Proper Non-placebo Arm

In 2021, Imperial College London authors published a study (the SAMSON trial) looking at statin side effects. Here they assessed the nocebo effect, which is the opposite of the placebo effect: feeling worse on the treatment not better.

They enrolled patients who were statin-intolerant. One month, the patients received statin tablets, one month they received statin placebo tablets and one month they received no tablets. The order of months over a year was randomized. Each day the patients recorded how they felt on a smartphone app.

Here is the picture of the main results

Statin-intolerant patients felt best during the months that they took no tablets—the non-placebo arm. There were no differences between the statin-placebo tablet vs the statin tablet.

Inclusion of a non-placebo arm allowed the researchers to conclude that the act of taking a tablet caused symptoms. It did not matter whether the tablet had a statin in it or not.

Back to the Psychiatry Study

The authors write that placebos can have positive effects on many common conditions in psychiatry. But that is just a guess.

They don’t know it to be true because they don’t have non-placebo controls.

Back to the Red Supplement Study

Now let’s say you split the 100 cyclists; 50 got a red supplement; 50 got a red placebo. Both groups improved by 30% perceived benefit. You would conclude that the red supplement was no better than placebo. Or no effect.

Now what if you went back and looked at just the placebo arm.

You could say the red placebo induced 30% benefits from baseline. But that would be wrong because you don’t have a non-placebo arm.

---

The lesson of this study-of-the-week is to think hard about control arms of trials. You cannot conclude a placebo tablet (or procedure) has causal effects unless there is a proper non-placebo arm.

JMM

Reference:

Epidemiologist Dr Eleanor Murray has this technical explainer.