Why most medical research is likely (still) waste, and less research funding may be one way forward
As always, we are happy to feature articles that expand on ideas discussed on Sensible Medicine – even when the author agrees more with Dr. Prasad than with me. Today, we welcome back Dr. Raudasoja, who last published with us about a year ago.
Adam Cifu
I listened to a recent episode of the Sensible Medicine Podcast in which Vinay argued about that 80% of NIH research is probably waste. Adam thought the number was closer to 20%. I do not have a specific number, but I think I am closer to the “radical right” represented by Vinay. I think most medical research is a waste and often even harmful. This is because of poor research quality and irrelevant research questions.
Most research is poor quality which makes the results unreliable
Poor quality research makes research results unreliable. Probably a familiar issue to Sensible Medicine readers, medicine is filled with unreliable observational research. However, that is not the only source of poor-quality research. Even randomized trials are often poor quality. For example, a meta-research study estimated that only about 29% of randomized trials in cardiovascular disease research were of low risk of bias. The risk of bias makes research results unreliable -- the reader will remain unsure what is the accurate answer to a particular research question after reading a study.
Research often answers irrelevant questions
Risk of bias is not the only reason for wasteful research. Research often asks questions that are not useful. They provide only indirect evidence on the research questions that most people are interested in. There are three sources for poor research questions: irrelevant/indirect study population, irrelevant/indirect intervention, irrelevant/indirect outcomes.
Indirect study population means that the participants recruited by a study do not represent people we see in clinic. A systematic review estimated that clinical trials exclude about 80% of potentially eligible patients. It is therefore uncertain how well research results apply to the patient sitting in front of the clinician.
Second, irrelevant/indirect intervention means that studies tests interventions that are not possible to use outside the study. This may be because the intervention requires special resources or too many resources to be implemented. The intervention may also be so context specific or complex that it is not possible to deploy anywhere else. One example was the Bangladesh trial on community masking. The intervention included several context specific behavioral components to increase mask use in the local communities. By looking at these intervention components, it seems very unlikely that the intervention could be used anywhere in even close to similar manner. And if not used similarly, the results will be uncertain. There is limited external validity. These problems are especially common in research on behavioral interventions.
Third, medical interventions are often tested with indirect outcomes such as progression free survival. It is usually uncertain whether these results translate into an endpoint that is important to patients.
Low quality research is often harmful
The current state of medical research is probably quite close to 1994. Back then a statistics professor and one of the forefathers of EBM, Doug Altman published a BMJ article titled “The scandal of poor medical research.” He wrote, “We need less research, better research, and research done for the right reasons”.
We need to reconsider research funding. Answers to questions do not need to be sought if it is not possible to answer them reliably. Furthermore, we need to ask research questions and study interventions that are relevant to the decision-makers in healthcare. Low quality and irrelevant research are likely to harm decision-making as wrong and/or overconfident judgements from those studies are inevitable -- consider cancer drug approvals based on progression free survival. In EBM, we consider if a treatment provides more benefit than harm for a patient. Similarly, we should consider if research provides more benefit than harm to our societies.
Less research funding may be one of the solutions to reduce waste
I am certain that regardless of your political bias that everyone would find better use for tax-payer money than poor quality research. This may not be the most useful opinion considering my own research career. I think that without clear solutions to increase the quality of funded research, a decrease in funding would often be better for society rather than more funding. Like healthcare, which seems to swallow all resources available without improving outcomes, medical research will also swallow all distributed resources. However, the harms for decision-making from the research conducted with these new resources may outweigh the benefits.
Aleksi Raudasoja is an editor in Finnish Current Care Guidelines and is the responsible editor of Choosing Wisely recommendations. Dr. Raudasoja is finalizing his PhD on de-implementation strategies and low-value care.
Photo Credit: Louis Reed
 | A guest post by
|
Demanding/requesting that medical research studies be conducted with the highest possible level of quality methods and standards is NOT and should NOT be considered a political issue. This is not a “right wing” advocacy issue. Don’t all of us need and want the best treatments and interventions?
One need to go no further than the registrational trials for the Pfizer COVID vaccine, which excluded almost everyone who was at risk, namely, immuno, compromised, those with cancer, those with neurological issues, and yet they were then rolled out to these high risk groups. Furthermore only 5% of the trial group from memory were in the age range most at risk, because at that time, it was well known that the average age of death from Covid was 82 in the United States. This IMO constitutes trial fraud. We also know the end points were for minor reduction in symptoms and end points for transmission, infection, severe disease hospitalization and death were not investigated. This also constitutes IMO trial fraud, as we were then told that these interventions prevented those particular end points. I could go on and on, yet this is neither acknowledged nor sufficiently documented by our research community.