Cheating in study design or just straight up fraud in science is a clear and present danger… likely rampant and would be even more so if the rewards were greater
The antiquated academic mantra (incentive) “publish or perish” significantly dilutes the quality of research in all fields of study—medicine being no exception. Perhaps we have too many research institutions, many of which should focus more on teaching. Perhaps the increasing number of non- inferiority studies is an indicator that too many federal dollars are made available. Can the country (taxpayers) continue to afford research that provides nebulous outcomes? My radical idea is that ALL federal funding of research be put on hold for 5-10 years; instead, promote private and corporate funding of research with tax credit incentives.
In my view, limiting the pool of dollars won’t change these incentives. A smaller pool of dollars will concentrate the research funding in those researchers best at optimizing for these incentives and by any current academic metric these researchers were successful in this endeavor. The problem isn’t the spending of dollars, it’s the incentives that drives the recipients. More or less money with better incentives will mean better research. Much that’s being done is good, but much could be better.
Love the take on EVERDAC. It helps explain why all the ICU docs I talked to it about were unimpressed and didn’t feel like it changed anything. Feels like more studies are like that now.
Agree. Industry incentives are overt and obvious. Nice to have a post that shines a light on academic incentives.
I would say that my generation had “hard outcomes” drilled in as the de facto gold standard, but this is a good reminder that hard outcomes, upon which an intervention has no hope of impacting, will simply make “non inferiority” inevitable. This lesson should come in a box-set, paired with Dr. JMM’s oft-issued refrain where composites of outcomes that move in opposite directions also make non-inferiority the default result.
I wonder if the heuristic might be, if you know the outcome before that trial even starts, then that’s a bogus study.
https://bariweiss.substack.com/p/science-has-a-major-fraud-problem?r=pbogx&utm_medium=ios
This was in the Free Press today. Related to your point about scientific incentives, and as cautionary tales of maladaptive expressions thereof.
Cheating in study design or just straight up fraud in science is a clear and present danger… likely rampant and would be even more so if the rewards were greater
The antiquated academic mantra (incentive) “publish or perish” significantly dilutes the quality of research in all fields of study—medicine being no exception. Perhaps we have too many research institutions, many of which should focus more on teaching. Perhaps the increasing number of non- inferiority studies is an indicator that too many federal dollars are made available. Can the country (taxpayers) continue to afford research that provides nebulous outcomes? My radical idea is that ALL federal funding of research be put on hold for 5-10 years; instead, promote private and corporate funding of research with tax credit incentives.
In my view, limiting the pool of dollars won’t change these incentives. A smaller pool of dollars will concentrate the research funding in those researchers best at optimizing for these incentives and by any current academic metric these researchers were successful in this endeavor. The problem isn’t the spending of dollars, it’s the incentives that drives the recipients. More or less money with better incentives will mean better research. Much that’s being done is good, but much could be better.
Love the take on EVERDAC. It helps explain why all the ICU docs I talked to it about were unimpressed and didn’t feel like it changed anything. Feels like more studies are like that now.
Agree. Industry incentives are overt and obvious. Nice to have a post that shines a light on academic incentives.
I would say that my generation had “hard outcomes” drilled in as the de facto gold standard, but this is a good reminder that hard outcomes, upon which an intervention has no hope of impacting, will simply make “non inferiority” inevitable. This lesson should come in a box-set, paired with Dr. JMM’s oft-issued refrain where composites of outcomes that move in opposite directions also make non-inferiority the default result.
I wonder if the heuristic might be, if you know the outcome before that trial even starts, then that’s a bogus study.
Thank you. I will put this and the JAMA resource into my fall evidence-based practice for nursing course.
I learned something new to me - thanks…!
Well written and I agree with this article.