Yesterday a Sensible Medicine post argued that skepticism can go too far. That's true, but the specific examples raised led me to believe that we're talking past each other.
The author holds that claims like ‘coffee can reduce dementia’ may be bolstered by the consistency of the observational finding, the correlation with basic science laboratory experiments, and that triangulation can help us build causal models. Let's talk about this.
Basic science
In defense of the possibility that coffee can prevent dementia, the author writes, “It’s well established from lab research that coffee contains biologically active compounds with plausible effects on neuroinflammation, oxidative stress, amyloid and tau pathways, and related biology. These compounds include caffeine, chlorogenic acids, caffeine acid, and trigonelline.”
If you take any complex compound from soy sauce, to whiskey, to coffee, to tobacco, to ripe tomatoes, you name it, you can take the substance and put it in a distillation column. You can extract compounds of all varieties. Then you can concentrate these compounds and you can run cellular assays or paint the back of a nude mouse with these compounds. I guarantee you you will find compounds that are antioxidants, and reduce inflammation, as well as compounds that increase cancer. All in the same bottle.
Part of the way we're talking past each other is that we have not explained the deficiencies of basic science adequately. Scientists often ask bad questions, run bad experiments, and have bad incentives to claim discovery, but beyond all that —some things lack reducibility. Smoking is on balance harmful, coffee may be on balance harmful, beneficial or neutral-- but there is no grand theory of polyphenols and antioxidants that can explain the net health outcomes of all food stuffs.
Basic science often fails to provide clarity because it can say whatever you want it to. You can compare HIT exercise to yoga and measure Vo2 max and CRP— but that tells me nothing about living longer or better, and what to do if I only have 25 mins free.
Consistency
Observational studies are malleable. If you play around in stata or spss you can get it to give you hazard ratios that are positive, negative or null for most association pairings.
Adam said observational studies agree with randomized trials 80% of the time. That's true, but a coinflip is 50%!
John Ioannidis once said that in many cases, observational studies are more like an opinion poll or self-fulfilling prophecy. What did he mean?
There are topics where medical researchers tend to hold the same views. Women are at least as good doctors or better than men. If you look at all of the observational literature on outcomes by physician gender, you will get this conclusion. Bacon is bad. People exposed to air pollution on 9-11 must have been harmed. The pesticides sprayed on golf courses are bad. The list goes on and on.
No one thinks it was good to breath in dust on 9-11. No one thinks bacon is healthy. Etc.
On these topics, the temptation for researchers to analyze the data and then make small adjustments till the desired conclusion is reached is extreme.
I don't think it's good to breath in dust from a collapsed building. I don't need an observational study linking that to non Hodgkin's lymphoma— a study that may or may not be true.
For nutritional topics where people disagree: coffee and alcohol, it's no surprise we get both answers. That's what John Ioannidis means by opinion poll.
Triangulation
Although it is tempting to concede that study a is bad, and study b is bad, but nevertheless argue the totality of the evidence and directionality can lead one to conclude…
Or that just because a study is ‘worth less, doesn't mean it's worthless’…
The truth is we don't know this to be true.
A thousand pieces of bad evidence that point in one direction is why innocent people go to prison for crimes they didn't commit. One witness says they saw a tall man, this guy is a tall man, one witness says they saw blue car, this guy has a blue car…. Then 5 years later the DNA says someone else did it.
(In this analogy, the DNA is randomization)
Pretest probability
The final way we're talking past each other is we're not acknowledging the pretest probability that things work. We've spent hundreds of billions of dollars on dementia research and nothing has worked. If dementia could be averted by reducing inflammation, Tao protein, or chlorogenic acids— we might have had at least some drugs that do that.
Out of 100 trillion compounds that we run our hands over or pull into our lungs each year, the vast, vast majority have no net effect on human health. That's just the pretest probability. The vast majority of things you're exposed to don't increase or decrease your life expectancy.
Then among the substances that do have an effect, harm is much more likely to be the net direction. That's because natural selection has assiduously rewarded being healthy.
Then there may be a tiny fraction of compounds or molecules that do not occur in nature that have a net beneficial effect, but most of these would have a net beneficial effect in states of malady or maladaption. If indeed, it was good to augment your HMG COA reductase every day of your life in every person, surely that would have been selected for.
Science is organized skepticism
The very definition of science is that it is a form of organized, collective, rational skepticism. I worry that faulting us at sensible medicine for being too skeptical is like someone faulting you at Burning Man for being too sober. Actually it is you all who are a bunch of drug addicts.
Medical scientists and healthcare journalists are a bunch of people who are way too confident that we have made a new discovery, or that everyday substances if taken in slightly different amounts can improve our health. Their pretest probability is broken.
When human beings try to improve their health, they tend to screw up much more than they tend to help.
During the covid-19 pandemic you had thousands of reporters and scientists who quickly concluded that closing schools, beaches, borders, wearing cloth masks, not hanging out with your loved ones, that these all would benefit all people. Imagine the hubris of that. The staggering hubris.
In several thousand years of recorded human history, there have been few human beings so confident that they can proscribe the movements and behaviors for all others. Public health had more in common with Genghis Khan, and Egyptian pharaohs than it did with science. Now of course, with AM sobriety we can see how crazy we were.
Morning sickness is bad, take your thalidomide
Fire is bad, hang up those asbestos curtains
Atrial fibrillation is bad, insert a device into the left atrial appendage and blow it up.
Cancer is bad, take this novel injectable immune therapy in combination with nivolimab that was tested with no control arm.
Dementia is bad, do more crossword puzzles and drink 3 cups of coffee to avoid it.
Miscarriage is bad, take a diethysybestrol to avoid it.
Covid is bad, take paxlovid or ivermectin (depending on your voter registration)
The thing that separates man from the other animals is a desire to take medicine, and a desire to believe that we have figured out something that others didn't. Skepticism is resisting that impulse.
Perverse professional incentives
The last thing I think is worth mentioning is that one of the reasons why John Mandrola,Cifu and I and others have argued that we should have a moratorium on observational research on exercise, or coffee— is that this is funded with taxpayer money.
The author quotes Meir Stampfer from Harvard. With his colleague Frank Hu the two have swallowed massive amounts of NIH funding. Whether or not they've done good for the American people is uncertain, but what is certain is that they've done well for themselves.
I have no problem with people doing coffee research that's of poor quality and doesn't change practice, but I don't know why I should be forced to pay for it.
Similarly, journalism covers these articles endlessly because it's cheap and easy clickbait. That's fine, but then a once in a century pandemic comes along and journalism has bred a populace that's uncritical of reading studies. We also have journalists who are not nearly skeptical enough of the preposterous, and society distorting interventions that are being deployed. The journalist who thinks coffee can prevent dementia becomes the journalist who thinks a cloth mask can work on a 2-year-old.
All you do is criticize studies
Everyday as an oncologist, I prescribe abvd or chop or venetoclax or hundreds of other drugs, devices and interventions that work. Writing a post that says that people with Hodgkin's lymphoma should be treated with ABVD is as uninteresting as saying hungry people should eat food.
The things we talk about in medicine that are worth talking about are the ones where your take is different than others. I see you're all stopping lisinopril and switching to entresto that seems to be pretty misguided and here's why.
The truth is there are massive financial interests that benefit from tricking people into thinking that healthcare is offering a steady stream of miracles. There are some real gains, but loads of fool's Gold. On sensible medicine we're trying to teach how to see the difference.
Skepticism can go too far, just like you can be too thin, but when you look around America the bigger problem is the opposite one.
Finally, I appreciate that sensible medicine publishes posts that stimulate discussions like this.


The 'a coinflip is 50%' line is the one I'd put on a poster. Eighty percent agreement sounds reassuring until you ask what the agreement would be if the literature were guessing — and in nutrition, where the same exposure gets both answers, that's not a hypothetical.
Where I land, from behind a pharmacy counter, is on triangulation — with a wrinkle. A patient brings me the coffee headline and asks whether to drink more of it. The answer isn't 'the evidence is weak'; it's that one cup holds hundreds of compounds pulling in different directions, at doses nothing like the concentrated extracts in the lab work. Drug development demands the randomized step before exposing millions; nutrition never gets one, so the 'totality' stays provisional forever.
Your pretest probability point is the one that sticks. After hundreds of billions spent on dementia, the prior that a pantry staple succeeds where purpose-built molecules failed should be vanishingly small. Skepticism can go too far — but at the counter, the cost of under-skepticism is a patient rearranging their life around a hazard ratio.
Perfect take!