A broken study of time-restricted eating exposes broken thinking amongst Top People
The Study of the Week explores selective criticism of flawed studies
You may have heard the ruckus about the study showing that intermittent fasting was associated with a 91% higher rate of dying of heart disease.
I call it a ruckus because after the AHA sent out a press release about the study, two things happened:
The mainstream health press covered it as they do (breathlessly) for any clickable story.
The Top People of cardiology and health went ballistic in their criticism of the paper. All of sudden, the world got an education on confounding and selection bias, and the difference between an abstract and a peer-reviewed paper. Grin.
The lesson here is not that a flawed study got outsized attention but the nonsense inherent in the reaction from the Top People.
First the background. The AHA hosts a meeting called the Epidemiology, Prevention, Lifestyle and Cardiometabolic congress. Or AHA EPI for short. It’s a relatively small meeting. Before any medical meeting, the organizers often send out press releases of “interesting” studies. In reality, “interesting” equates to clicks.
This press release blew up because the study found an inconvenient association: that time restricted eating (or intermittent fasting) associated with a 91% higher rate of cardiovascular death.
You see…time restricted eating as a weight loss or health delivery strategy has many advocates. This finding made them uncomfortable. The fact that major news outlets covered the story with dumb headlines enhanced the outrage.
So. Boom. The criticism was harsh and swift. I won’t link to them all. Just do an advanced Twitter search for time restricted eating. This will lead you to Top People educating the world about flawed science.
Readers of Sensible Medicine, however, would hardly be surprised about the weak methods because we highlight this game plan so often.
In this study, the research team used an NHANES database. NHANES is called a longitudinal study—basically it follows adults over time. They took adults who self-reported their time windows of eating and then linked them to another database to learn future outcomes.
Less than an 8-hour window of eating was considered the intermittent fasters (the active comparator arm); those that reported eating over 12-16 hours was the control arm—of sorts.
The study had the façade of science. There were 20,000 individuals. The authors used multivariable cox proportional hazard models and reported hazard ratios.
When they did the correlations with self-recall of diet patterns on two average days, they found a 91% higher rate of cardiac-related death in the intermittent fasters. They did not find the same association with all-cause death or cancer.
But. But.
Of course this was a fatally flawed study. However you feel about intermittent fasting, it surely doesn’t (nearly) double your risk of death from heart disease. I’m pretty neutral on the strategy since the highest quality studies don’t show much benefit. See NEJM and JAMA-IM.
The problems with this study were obvious and typical of observational nutritional science: the authors did correlations from a 2-day recall of eating patterns. Non-random comparisons are problematic because those that report time-restricted eating are different from those who do not report it. Mostly, though, there’s no plausible reason for this outsized finding. It’s surely noise not signal. What’s more, the paper has not been vetted. It’s just a poster.
These specific criticisms are not the point. Medical meetings overflow with these sorts of studies.
Last month, an equally flawed study found that women may benefit more from regular physical activity than men. This one made it through review at a big journal.
And. The message better aligned with the Top People—so the typical observational flaws were ignored. I critiqued the exercise study on the This Week in Cardiology podcast (transcript here).
That’s the story today. Flawed studies get hammered if the Top People don’t like the result but not if they like the result.
My hope in writing this piece is that a) researchers stop doing comparison studies with non-random observational methods; b) journals and meetings stop accepting them; c) health journalists use these papers to teach the public about methodological weaknesses; and d) the Top People of medicine unleash their critical appraisal fury regardless of the results of the study.
Since all of these are unlikely to happen, keep reading Sensible Medicine. We will strive to be the neutral judges. JMM
Please consider subscribing or supporting our work.
Have mainstream doctors always been this devoid of critical thinking or is this a new phenomenon? I’ve worked as an NP in large academic medical centers for 20 + years and I never really questioned most medical professionals until the pandemic when there were such egregious missteps.
I’m now questioning everything for myself and family. I feel sorry for those that don’t have any medical or science in their background. I feel like we’re back in the snake oil salesmen days.
As a health journalist, I deeply appreciate the free education I've gotten on evaluating studies here on Sensible Medicine, though the little I've learned has made it clear how much more I still have to understand. Would you, Dr. Mandrola, or some of your Sensible Medicine colleagues ever consider offering a webinar or short e-course for health journalists (or whoever is interested) on this topic? I'd gladly pony up for such a thing.