Guidelines: The Good, the Bad, the Ugly

Guidelines have become ubiquitous. In 2024, 289 guidelines were published in the United States, a 29% increase from 2023, with the American Society of Clinical Oncology leading the list with 19. They are influential, often driving citations in journals and influencing clinical care. Recognizing the importance of guidelines, in 2011 the National Academy of Medicine published a report titled Clinical Practice Guidelines We Can Trust, which established 8 core standards for guidelines: transparency; restrict conflict of interest; systematic review; evidence foundations; assessment of quality of evidence; clarity; external review and updating.

Yet increasingly guidelines conflict with one another, or a professional society “objects” to a guideline produced by another professional society. For example, there are now numerous recommendations for screening for breast cancer. The American Cancer Society recommends women be screened yearly for breast cancer between the ages of 45 and 54 (optional beginning at age 40), whereas the United States Preventive Services Task Force (USPSTF) recommends biannual screening between the ages of 40 and 74. Recently, the American College of Radiology (ACR) objected to the new breast cancer screening guidelines of the American College of Physicians (ACP), suggesting they were outdated. Different recommendations from influential societies confuse the public and clinicians.

Guidelines may differ or be controversial for various reasons. First, guidelines may not be addressing the same question. When comparing guidelines on the same subject, readers need to carefully assess the precise issue that is being addressed. Are the guidelines focused on high-risk (and how is that defined) or normal-risk individuals? Are the guidelines assessing the same outcomes?

Second, the time period for the review of the literature may be different. It takes time to produce a guideline. When evidence is evolving, the date of the guideline and the actual “evidence” included in the review might lead to different conclusions.

Third, specialists and generalists often view the same evidence differently. We suspect this is the root cause of the ACR objection to those of the ACP regarding breast-cancer screening. Most specialists focus on case-finding, confirming disease, whereas generalists focus on screening, identifying the potential for disease. These are two very different cognitive activities (specificity vs sensitivity), and likely, over time and with experience, produce different views of the same data.

Fourth, is the concern about financial conflict of interest. In the US, most physicians are now employed and salaried, with fewer in so-called private practice. There are three different potential financial conflicts of interest for salaried physicians. First, many salaried physicians can still receive a bonus based on their clinical activity, especially for procedures such as screening colonoscopies. Second, even if an individual does not directly benefit from performing more diagnostic tests and instituting treatments, the person’s institution benefits. Third, physician experts are used by companies to promote their products, and these relationships are often not fully disclosed. The rigor of disclosure of financial COIs differs substantially among groups issuing guidelines. Do we think that financial conflicts of interest influence guidelines produced by specialty societies — yes. But we do not believe that they are the important reason for differences in guidelines between specialist and generalist societies, or that generalists often find specialty societies’ recommendations to be too aggressive with respect to diagnostic testing and treatment.

It is almost impossible for journals to review guidelines. The process of creating guidelines requires the creation of a multidisciplinary panel, defining the questions to be addressed, conducting a systematic review of the literature, assessing the quality of the evidence, drafting recommendations that are then reviewed by panel members, and finally producing a manuscript for publication or posting on a website. No single individual can do this effectively. Thus, when guidelines appear in journals (or on a society website), often with an accompanying editorial, readers should be aware that these were not “peer-reviewed” in the same manner as original research reports.

The grading of evidence is complicated. The Canadian Task Force on Preventive Health Care (CTFPHC) and USPSTF, launched in 1976 and 1984, respectively, established methodological standards. The USPSTF has published a manual detailing how it grades evidence. The CTFPHC adopted the Grading of Recommendations Assessment, Development and Evaluation (GRADE) in 2010. GRADE has become ubiquitous, with many professional societies using it to develop their guidelines. One challenging issue for all grading systems is whether they are reliable: meaning, would a different group examining precisely the same question with the same data reach the same conclusions? At a time of concern about replicability and reproducibility in science, given the laborious process of developing guidelines, it is not clear how to deal with the lack of data about reliability.

The USPSTF recommendations have traditionally been among the most influential in the US, even more so when they were tethered to the Affordable Care Act. Insurers must pay for A and B recommendations without cost sharing. Although many have championed this policy change, others have raised concerns that it may have inadvertently “politicalized” the USPSTF, with members recognizing the importance of A and B recommendations, regardless of the data, and perhaps influencing how they interpret the evidence.

Improving the Reporting of Published Guidelines

Professional societies and federal agencies will continue to produce guidelines. Indeed, the numbers are likely to increase as societies try to help their members provide high-quality care. The Trump Administration has halted the work of the USPSTF, but hopefully this is temporary. Given that journals, if guidelines are submitted to them, cannot truly peer-review them, what can those who produce and publish them do to improve their quality?

First, guidelines should be dated, with an expiration date, perhaps no longer than 3-5 years. New evidence is constantly emerging, and guidelines need to be updated regularly. Minimally, if no new evidence has emerged, a society could “re-endorse” a guideline. Guidelines that exceeded the expiration date should be removed from websites or at least noted as such.

Second, when publishing a guideline, the journal could insist that the authors follow the current recommendations for guidelines – the AGREE reporting checklist. Admittedly, like all the reporting guidelines, AGREE was developed through a Delphi approach, but it is an attempt to produce high-quality reporting recommendations. Published guidelines in a journal or on a website should include an assessment of adherence with AGREE.

Third, journals should clarify their role in publishing the guideline. Limited peer review should be acknowledged, and the journal should clarify its role as a dissemination vehicle.

Fourth, guidelines should contain a discussion of any recommendations that are different than those of other guidelines. This will at least help clarify for clinicians and the public why guidelines may differ.

Fifth, guidelines should contain information about how committee members were trained in their system of evaluating evidence and producing recommendations. Many guidelines detail their evidence system but not how members were trained to use it. Given that so little is known about reliability, at least some information about training may be valuable.

Sixth, editorials that comment about a newly published guideline should include both a generalist and specialist perspective.

Seventh, professional societies that criticize the guidelines from other groups should do so with evidence, and not simply “object” to them. If we want to restore faith in medicine and science, that needs to be done with data, not simply words.

Guidelines are here to stay and may proliferate given that AI can now conduct literature searches, and perhaps even “grade” the quality of the evidence. It would be interesting to see if different LLMs would produce a similar guideline given a precise question, for example, at what age should an average-risk patient begin screening for breast cancer, or what should be the target for low-density lipoprotein in a high-risk patient. We suspect AI will be helpful in determining if new evidence has emerged and a guideline needs to be updated. Guidelines are important, but they are not the definitive word on how to care for patients – that requires a discussion between patient and clinician – and what is best for that person.

Howard Bauchner, MD, is a Professor of Pediatrics and Public Health at the Boston University Chobanian & Avedisian School of Medicine. He is also a Visiting Scholar at the National University of Singapore. Dr. Bauchner is also the former Editor-in-Chief of JAMA and the JAMA Network (2011-2021) and Archives of Disease in Childhood (2004-2011).

Frederick P Rivara, MD, MPH, is Professor of Pediatrics and Adjunct Professor of Epidemiology at the University of Washington. He is also the former Editor-in-Chief of JAMA Pediatrics (2000-2017) and JAMA Network Open (2018-2024). He continues as an active clinician, mentor, and investigator.

Photo by Javier Allegue Barros on Unsplash