a) the anti vaxxers…for whom their prior of “vaccines cause autism” is set in stone, and no amount of “absence of evidence” (ie negative trials) will affect their posterior probability.
b) the vitamin D disciples, for whom their prior of “vitamin D supplementation does something” is also set in stone, and no amount of “absence of evidence” (ie negative trials) will affect their posterior probability.
"If your vitamin D is normal, more almost certainly does nothing". A quasi-tautology. Exactly what is a 'normal' level? Standard recommendations used to be 20-40ng/ml, but for Covid prevention the recent literature says around 60. It depends on the particular outcome you wish. Given that vitD affects around a thousand human promoters, perhaps you will admit that more research is justified-expecially when the consequences of vitD levels are so critical for health? Or do you know what a normal level is?
An example of an MD thinking he knows more than he does....
Excellent but you see lots of docs/teaching attendings won’t get promoted if they won’t engage in such research. Your blog likely is read and has more of a good clinical impact than poorly run studies but it’s hard to put into a promotion type packet.
How about studies that show pollution is really really bad?
Since we all know these types of studies are not going away, let’s hope that RFK Jr. funds those that have a chance to open up some new paradigms on these topics. For example, saying we don’t need more nutrition studies locks us in the calories-in, calories-out frame for weight management. But what if the carbohydrate-insulin model is the correct one? We need to fund the studies that have a chance to overturn our existing beliefs, and deny funds for useless investigations that can only tell us we’re right about what we already think we know.
We have a lot to learn from studying "Health outcomes by socioeconomic status." I'm not talking about the obvious truth that being poor is bad for you. What I am talking about is investigating what causes people with lower SES to have greater rates of occurrence (incidence, not prevalence) of disorders & diseases of interest. What we learn there could help everyone in all demographic groups.
For example: The CDC's recent Autism and Disability Monitoring (ADDM) shows a greater rate of increase in autism incidence among those with lower SES and certain racial & ethnic groups. For developmental disorders, case incidence is best measured via birth year cohort prevalence, which is exactly what the ADDM reports measure, even though they misleadingly call it simply "prevalence," which it is not.
If the trend in autism case incidence is truly greater among certain groups than others, as it appears to be, we should investigate what is different about those groups that might explain the increased risk. Some risk factors might well apply to all groups.
But we're not doing any of that. MAHA is not helping, either. For the most part, we aren't even recognizing the problem.
There are no good studies on Vitamin D. So some of these examples are errant. A good study of Vitamin D would be very expensive. Monitoring blood levels at various intervals, not just at the beginning or end. Dose response curves. The standard of 1000 units will never move the needle.
So many studies choose the wrong dose at the very beginning.
Your thesis is rather reminiscent of the proposition in 1898. Science has discovered everything. The end of science. Like the end of history.
Fair enough. So why do vitamin D proponents waste everyone’s time by willfully doing lots of bad studies, instead of focussing on doing even just one or two good ones?
Amen on observational studies on exercise and nutrition. Please don't waste your time and our time on trying to parse out whether 10K steps prolongs life better than 8K. Add sleep duration and quality to the list-these observational studies are confounded by the unmeasured variables that cause people to have poor, varied or shorter duration sleep. Also, no more on how marriage or religion makes us live longer.
Great article but I disagree about poverty if done in a comparative context. This is true of social determinants research generally. While not exactly on poverty. The recent Lancet article that finds that the bottom quartile of wealth in old Europe have approximately equal mortality rates to the top wealth quartile in the US is eye opening. I think that is surprising and useful.
Absolutely agree on the dementia prevention point. There are SO MANY aging cohorts throughout the world. We’ve done a fantastic job tracking the natural history of the disease and we know the major risk factors — that’s what observational studies are good for. Now on to the hard work of preventing and treating with RCTs.
Haha... an an exercise physiologist, #2 hurts. But it's also obvious. The big discovery was learning that exercise helps. What the field is trying now is quite incremental and unlikely to change a paradigm.
I would add "behavior change" research to the list. Theory based research on trying to get people to become more healthy has never and will never work. People change some habits while they are in your study. Nothing is likely to last long term.
I guess the question remains: how do we keep our jobs and remain academically productive if the entire field of our expertise is off the table? Or is this advice purely for clinicians and not PhDs?
On #9. The advice on vitamin D is all over the map, from 800 IU to 8000 IU per day. Likewise, the advice on minimum blood levels varies from 20 ng/ml to 80 ng/ml. And virtually all the trials have target osteopooesis or bone strength, and not the immune system. We need more of the latter, because one of the major factors in the death of elderly patients early in the covid epidemic was not having enough vitamin D.
Shouldn't you add studies of the relationship between childhood vaccines and autism to your list? No credible evidence has come from the many large studies that have been done, and one that may soon be underway mandated by RFK Jr and David Geier will surely find a link that the methodology and data do not support.
At least for MMR we don’t need more studies. For the others it’s probably sensible but it’s hard to find people who are truly neutral as to the outcome. RFK Jr certainly isn’t.
When I was an editor at JAMA, I would tell authors don’t send me any papers that tell us that disparities exist. We know disparities exist. Send me papers that show what to do about it. I never got any.
I agree entirely with that list.
However, among them will be:
a) the anti vaxxers…for whom their prior of “vaccines cause autism” is set in stone, and no amount of “absence of evidence” (ie negative trials) will affect their posterior probability.
b) the vitamin D disciples, for whom their prior of “vitamin D supplementation does something” is also set in stone, and no amount of “absence of evidence” (ie negative trials) will affect their posterior probability.
"If your vitamin D is normal, more almost certainly does nothing". A quasi-tautology. Exactly what is a 'normal' level? Standard recommendations used to be 20-40ng/ml, but for Covid prevention the recent literature says around 60. It depends on the particular outcome you wish. Given that vitD affects around a thousand human promoters, perhaps you will admit that more research is justified-expecially when the consequences of vitD levels are so critical for health? Or do you know what a normal level is?
An example of an MD thinking he knows more than he does....
Excellent but you see lots of docs/teaching attendings won’t get promoted if they won’t engage in such research. Your blog likely is read and has more of a good clinical impact than poorly run studies but it’s hard to put into a promotion type packet.
How about studies that show pollution is really really bad?
Voltaire had it right: “common sense is not so common”. And please a moratorium on retrospective studies: how about some new well designed ones?
Great stuff Dr. Cifu, you are a breath of fresh air.
Since we all know these types of studies are not going away, let’s hope that RFK Jr. funds those that have a chance to open up some new paradigms on these topics. For example, saying we don’t need more nutrition studies locks us in the calories-in, calories-out frame for weight management. But what if the carbohydrate-insulin model is the correct one? We need to fund the studies that have a chance to overturn our existing beliefs, and deny funds for useless investigations that can only tell us we’re right about what we already think we know.
We have a lot to learn from studying "Health outcomes by socioeconomic status." I'm not talking about the obvious truth that being poor is bad for you. What I am talking about is investigating what causes people with lower SES to have greater rates of occurrence (incidence, not prevalence) of disorders & diseases of interest. What we learn there could help everyone in all demographic groups.
For example: The CDC's recent Autism and Disability Monitoring (ADDM) shows a greater rate of increase in autism incidence among those with lower SES and certain racial & ethnic groups. For developmental disorders, case incidence is best measured via birth year cohort prevalence, which is exactly what the ADDM reports measure, even though they misleadingly call it simply "prevalence," which it is not.
If the trend in autism case incidence is truly greater among certain groups than others, as it appears to be, we should investigate what is different about those groups that might explain the increased risk. Some risk factors might well apply to all groups.
But we're not doing any of that. MAHA is not helping, either. For the most part, we aren't even recognizing the problem.
There are no good studies on Vitamin D. So some of these examples are errant. A good study of Vitamin D would be very expensive. Monitoring blood levels at various intervals, not just at the beginning or end. Dose response curves. The standard of 1000 units will never move the needle.
So many studies choose the wrong dose at the very beginning.
Your thesis is rather reminiscent of the proposition in 1898. Science has discovered everything. The end of science. Like the end of history.
Fair enough. So why do vitamin D proponents waste everyone’s time by willfully doing lots of bad studies, instead of focussing on doing even just one or two good ones?
Amen on observational studies on exercise and nutrition. Please don't waste your time and our time on trying to parse out whether 10K steps prolongs life better than 8K. Add sleep duration and quality to the list-these observational studies are confounded by the unmeasured variables that cause people to have poor, varied or shorter duration sleep. Also, no more on how marriage or religion makes us live longer.
Thanks for the great article! Tied for first as your best.
Regarding number 8 I would like to figure out a better way to train physicians. I like the way you want to study it.
Great article but I disagree about poverty if done in a comparative context. This is true of social determinants research generally. While not exactly on poverty. The recent Lancet article that finds that the bottom quartile of wealth in old Europe have approximately equal mortality rates to the top wealth quartile in the US is eye opening. I think that is surprising and useful.
Absolutely agree on the dementia prevention point. There are SO MANY aging cohorts throughout the world. We’ve done a fantastic job tracking the natural history of the disease and we know the major risk factors — that’s what observational studies are good for. Now on to the hard work of preventing and treating with RCTs.
Haha... an an exercise physiologist, #2 hurts. But it's also obvious. The big discovery was learning that exercise helps. What the field is trying now is quite incremental and unlikely to change a paradigm.
I would add "behavior change" research to the list. Theory based research on trying to get people to become more healthy has never and will never work. People change some habits while they are in your study. Nothing is likely to last long term.
I guess the question remains: how do we keep our jobs and remain academically productive if the entire field of our expertise is off the table? Or is this advice purely for clinicians and not PhDs?
My pet peeve is clinicians doing useless research rather than caring for people. PhDs should do good work, ideally in research.
On #9. The advice on vitamin D is all over the map, from 800 IU to 8000 IU per day. Likewise, the advice on minimum blood levels varies from 20 ng/ml to 80 ng/ml. And virtually all the trials have target osteopooesis or bone strength, and not the immune system. We need more of the latter, because one of the major factors in the death of elderly patients early in the covid epidemic was not having enough vitamin D.
The COVID vitamin D research is a model of confounded data. Maybe a future SM post.
Shouldn't you add studies of the relationship between childhood vaccines and autism to your list? No credible evidence has come from the many large studies that have been done, and one that may soon be underway mandated by RFK Jr and David Geier will surely find a link that the methodology and data do not support.
At least for MMR we don’t need more studies. For the others it’s probably sensible but it’s hard to find people who are truly neutral as to the outcome. RFK Jr certainly isn’t.
You'd have to specify which ones, certainly not DPT, hepA, and hepB.
When I was an editor at JAMA, I would tell authors don’t send me any papers that tell us that disparities exist. We know disparities exist. Send me papers that show what to do about it. I never got any.
I disagree with Number 7. I think you are ignoring those studies and would prefer not to see them so you don't need to ignore them.
My daily work tells me that poor people do worse. I don't need a study to tell me. I can't ignore what occupies every hour of my working life.