Guidelines, LLMs, and the Laundering of Uncertainty

Are we trading the bitter medicine of ambiguous truth for the hollow satisfaction of false confidence?

Nobody wants to go to the doctor with a question, only to be told, “I don’t know.” Doctors strive for definite answers, but often there are none. In many situations, there is more than one right answer. Doctors seek guidance from mentors, colleagues, clinical guidelines, UpToDate, or, increasingly, AI. There’s no shortage of information and recommendations to choose from. When an answer can be found, patients and doctors are relieved of the burden of uncertainty. And yet, in our quest to quell uncertainty, we are forgoing our ability to grapple with the ambiguities that permeate medicine.

Synthesizing basic medical science, empirical evidence, and the complexities of an individual’s physiology, values, and goals grants medicine its status as both a science and an art, practiced in real time and bespoke for each patient. Yet, despite incredible strides in medical science, medicine remains rife with ambiguities. While fundamental facts are typically stable, science changes. For example, when I was a medical student, I was told that the old way to treat atrial fibrillation had been to control the abnormal rhythm: get everyone back into sinus. But the AFFIRM trial in 2002 challenged this: it turned out that just controlling the rate worked about as well—and avoided the sometimes nasty side-effects of antiarrhythmics. But then, in real time, I watched practice change. Now it appears that controlling the rhythm early with an ablation is often superior.

The scientific paradigm shift is the process through which theories are built, tested, and replaced. Ideally, the good is replaced by the better. But too often, an established practice is found to be pointless or harmful and undergoes reversal. The ongoing proliferation of medical research threatens the levees we’ve built around what we think we know. Making matters worse, a significant portion of scientific research isn’t replicable. Whether due to poor design, incomplete data, or faulty analysis, many manuscripts are relegated to the far corners of PubMed, collecting dust rather than citations. And even high-quality research is not without its controversies: for a paradigmatic example, look no further than the PARADIGM trial.

In the face of this data tsunami, there has been a proliferation of clinical practice guidelines. But there’s something lurking behind the crisp, multicolor tables and lists of citations. The danger lies precisely with their greatest asset: simplification. The full complexity of a clinical question is reduced by a guideline to a recommendation. This is a godsend, but it carries with it a risk: confidence.

Clinical guidelines provide an easy out but disincentivize both research into the topic and scrutiny of how the guidelines themselves were created. By collapsing the jumble of research into an easily interpreted format, the underlying uncertainty and inherent contingency of that research are lost. Decisions of the guideline committee are likewise obscured by the final product. It is through this laundering of uncertainty that guidelines risk disrupting the judicious, patient-centered practice of medicine. In the balance of medical decision-making, strong medical justifications often overshadow unique patient factors. Artificially augmenting the confidence in a recommendation risks alienating care from the individual patient in the pursuit of benefit to the average patient.

Like clinical guidelines, UpToDate provides fast answers to complex questions. By providing articles detailing the diagnosis and treatment of numerous ailments, it has all but supplanted textbooks and the search of the primary literature. What is often forgotten is that UpToDate articles are written by individuals. They are useful not because they are the unerring sum of medical knowledge, but because they are convenient and well presented.

Large language model-based systems have recently joined clinical practice guidelines and UpToDate. Whether driven by unrealistic efficiency expectations in our increasingly corporatized medical systems or by individual physician choice, LLM use has grown rapidly. OpenEvidence, in particular, is popular thanks to its ability to answer complex questions with citations—often to clinical guidelines. At their best, these tools can provide a jumping-off point for deeper reading. But there is an incentive to read the provided answer and stop. Contributing to this bias towards cognitive closure is LLMs’ proclivity for zealous confidence. It remains unclear whether this central tendency toward agreement and obsequiousness can be reversed by software engineers (let alone tolerated by market forces).

There are other concerns with LLMs. Consider the possibility of a literature search revealing a morass of contradictory evidence, while the language model frames a version that appears authoritative and complete. It is not that the information is wrong, but how our view of it is shifted.

Clinical guidelines, UpToDate, and AI are powerful resources. They can distill and present complex medical knowledge into a usable form. It is impossible for physicians to have the knowledge of a medical researcher in every domain in which they practice. Even the best textbook will lag behind the most recent papers; even the most recent paper will lag behind the preprint. My concern is that an over-reliance on these consolidated sources risks trading the bitter medicine of ambiguous truth for the hollow satisfaction of false confidence. I fear that where once there may have been a question, only certainty will remain.

Combatting the trend towards algorithmic medicine should start with medical education. Evaluating scientific literature is a skill, and like any skill, it is learned through practice. Curricula should focus on equipping student doctors with the tools they need to reason clinically: scientific knowledge and a critical eye for research. Basic medical science provides the foundation, while deft literature appraisal allows for lifelong knowledge-building.

Unfortunately, it is becoming increasingly popular for medical schools to shorten their course of study. This is usually justified with reference to the often slow-paced final 6-8 months of undergraduate medical education. And yet many new curricula include cuts to pre-clinical coursework. While more medical knowledge is generated each year, medical students are being given less time to learn it. Even if unnecessary material is removed, introducing a curriculum on evidence appraisal is likely to be net neutral. The study of medical science and the evaluation of medical evidence must be given adequate time, rather than being pushed into evening lectures during busy clinical rotations.

For residents and practicing physicians, solutions are less clear. I suspect that concerns of litigation steer many physicians towards a greater reliance on guidelines. It’s hard to argue a physician acted inappropriately if they followed the recommendations. And even if the guidelines later turned out to be wrong, who could blame them?

What separates modern medicine from superstition is the recognition of what we do not know. Admitting uncertainty builds trust and lends weight to our claims of knowledge. We owe it to our patients and to ourselves to grapple with ambiguity and reject the false comfort of confidence. The alternative is that we become mere confidence men.

Jordan Schneider, MD, is a resident in Internal Medicine at NYU in New York. The perspectives written above are solely his own and do not reflect the views, policies, or positions of the NYU Grossman School of Medicine, NYU Langone Health, NYC Health+Hospitals, the Veterans Administration, or any affiliated organization.

Photo Credit: Annie Spratt