The 10-year Anniversary of Medical Experts Biggest Flub

The Study of the Week goes back in time to visit one of Medicine's most consequential errors. A story of hubris and groupthink run amok

Here is how bad the experts’ error was: in the randomized controlled trial testing the experts’ recommendation, the primary outcome (bad event) occurred in 13.7% of individuals randomized to the established practice vs 1.9% in the new treatment.

For those counting, the new treatment reduced the bad outcome by 86% over the expert’s choice. The p-value was less than 0.001, which means that if there was no difference in the two treatments, obtaining these results would be vanishingly unlikely.

I know of few trials wherein an established therapy performed this poorly. For instance, in Cardiology’s most important trial, the CAST trial of antiarrhythmic drugs (AAD) vs placebo in patients after MI, the established therapy (AAD) was 2.6x worse than placebo. In this trial, the expert recommended therapy was about 7x worse than the new therapy.

How could this be? How could medical experts worldwide be this wrong about something? Not 20% or 50% wrong but 7x wrong.

This is a story of groupthink, of hubris, of placing too much weight on plausibility and mostly the resistance to test theories in trials.