How investigators, journalists, and editors contribute to mistrust in science

I had already scheduled an essay about three new pet peeves of mine for tomorrow when I received this submission from Dr. Bauchner. Tomorrow’s post will offer three pieces of advice to assuage my aggravation. Dr. Bauchner takes a much broader look at problems in publishing today; problems that affect investigators, editors, journalists, funders, academic institutions, and beyond.

Adam Cifu

As a long-time actor in the ecosystem of research and publishing, I have come to believe that the future is uncertain – the weaponization of data, the politicalization of funding, and the dramatic increase in the number of journals and manuscripts – is a threat to the future of science and public belief in it. My focus is on investigators, journalists, and editors – and how some are inadvertently contributing to mistrust in science.

Investigators

Fueled by the need for recognition, grant support, and promotion, as well as the legitimate intent to contribute to science, investigators are publishing more manuscripts. Powered by publicly available data sets, electronic health records, and AI, the increase has been dramatic. Investigators must ask themselves whether their report will benefit science, clinical care, or public policy. Rather than focusing on more publications, a shift in mindset to quality, is necessary. I acknowledge that for medical students and residents, for whom publications have become an important metric, this will be difficult. But for faculty, at all levels, a focus on quality is necessary.

Journalists

I find the reporting of science to be uneven. Some journalists continue to struggle with the difference between hypothesis-generating studies, which are the majority of clinical studies, versus those studies that demonstrate causal relationships, certainly randomized clinical trials, and perhaps meta-analyses and Mendelian randomization studies (if they fulfill all the important scientific criteria). In addition, journalists often don’t comment on the difference between statistical and clinical significance. For RCTs, they rarely report the number needed to treat. Journals are also lacking with respect to the last comment – and we mislead the public by not reporting NNT. For example, in a recent trial published in NEJM that examined outcomes of veterans who received COVID-19 and influenzae vaccines, the COVID-19 vaccine was 64% effective in reducing COVID-19 mortality, statistically significant, with a NNT of 4545. Although both outcomes are true, they are likely to be interpreted differently by people.

Editors

Editors play a powerful role in science. They are the gatekeepers, making thousands of decisions each day regarding what will be published and what will not. Unlike preprint servers, journals place a “stamp of approval” on publications, which legitimizes them. And for the major journals, what they publish is unleashed to a worldwide audience in a matter of seconds. The task has become nearly impossible due to the relentless increase in the number of submitted manuscripts. Editors must embrace AI as an aid to the decisions they make. AI should not replace human decision-making, but AI has the capacity to ensure that a RCT and meta-analysis have been registered and are being reported accurately, are following the appropriate reporting guidelines, employ a valid statistical approach, and verify that references are “real” – just to name a few possibilities. In addition, all editors need to be aware of how journals are funded. This is more complicated for editors of open-access journals in which there is a direct link between how much they publish and the income they generate. If they lose editorial control, they must decide whether to remain as editor.

There are, of course, other actors in this play, including funders, both private and public, academic institutions, and publishers. With respect to publicly funded research, in the US, many of Trump’s policies are abhorrent. The administration professes a commitment to merit, but then announces that more federal grants must be decided based on geography. The recent proposal to ensure that all grants are consistent with the political views of the president, and that senior administrators will sign off on them, is another example of this administration’s commitment to undermine research that does not align with its political views.

Industry

Industry continues to invest billions of dollars in laboratory and clinical research. They support many of the practice-changing clinical trials. Some have expressed concern that some of these trials are designed to ensure superiority (or non-inferiority) of a company’s product, even though these trials are appropriately registered and reported.). In this case, it is the responsibility of journals to ensure that authors justify the design of a trial, and that criticism of the study, if warranted, can appear as an editorial. The drug and device industries are for-profit and will continue to maximize income – usually this serves the common good, but not always.

Academic Institutions

Academic institutions play their part. The coins of the realm for investigators are publications and grants. It is time to de-emphasize the number of publications and emphasize the contribution that publications make to discovery science, clinical care, or health policy. Some institutions already recognize this. When asking for letters of recommendation for promotion, these institutions provide a complete CV, but also require the faculty member to identify 5-10 of their most influential publications with a short description of why they are important, thus emphasizing quality over quantity

Publishers

Not to be forgotten are publishers, who continue to launch more open-access journals, with varying editorial standards, but often high publishing fees. For example, Nature Communications generates approximately $50,000,000 each year for Nature-Springer. I see no end in sight – the large publishers are for-profit entities. Even not-for-profit publishers are launching more titles. Perhaps inappropriately, I have more faith that they will protect editorial independence. Only when investigators stop submitting manuscripts to these journals will their proliferation slow.

It is time to commit to the registration of cohort studies whose intent is to describe, regardless of the words used, a causal relationship and to label hypothesis-generating studies as such in the abstract. Registering of RCTs was transformative in terms of improving editorial integrity and would be for cohort studies. Does anyone believe that the JAMA report that caffeinated coffee is associated with a lower risk of dementia, had it been labeled hypothesis-generating, would have received as much attention? Despite an extensive methods section, the study was not registered, so it is not possible to understand the extent of analyses the investigators had to conduct to produce the results.

Conclusion

It would be helpful if the government allowed the peer-review process to select grants for funding, industry to design studies whose only intent is to improve patient outcomes, and publishers to reduce the number of journals. This is unlikely to happen. So, I have offered suggestions that investigators, editors, and journalists need to own.

Concerns about editorial integrity existed long before the election of 2024, as evidenced by an increasing number of retractions, uncertainty about the replicability and reproducibility of studies, manipulated images, and the lack of registration of both RCTs and meta-analyses, to highlight just some of the issues. Reform has been slow, and the challenges greater because of the policies of the Trump administration, but the ecosystem of research and its reporting needs reform. The key players in the system should reflect on their own behavior if we are to restore faith in science.

Howard Bauchner, MD, is a Professor of Pediatrics and Public Health at the Boston University Chobanian & Avedisian School of Medicine. He is also a Visiting Scholar at the National University of Singapore. Dr. Bauchner is also the former Editor in Chief of JAMA and the JAMA Network and former Editor in Chief of Archives of Disease in Childhood.

Photo Credit Abdulai Sayni