This critique misses a fundamental point: clinical guidelines are not written for patients, and they are not intended to replace individualized clinical judgment. They are written principally by professional societies and expert panels for clinicians and health systems, to synthesize evidence, grade its certainty, define reasonable standards of care, and reduce unwarranted variation.
Good guidelines do not “launder uncertainty.” They explicitly characterize it through evidence grading, conditional recommendations, qualifying statements, and acknowledged evidence gaps. The physician’s job is then to interpret that population-level recommendation in the context of the individual patient.
That is importantly different from an LLM producing a fluent answer to a question. Guidelines can certainly be wrong, outdated, biased, or poorly constructed, and they deserve scrutiny. But equating a professionally developed guideline with an AI-generated response obscures rather than clarifies the real problem: whether clinicians understand the provenance, evidentiary strength, limitations, and intended audience of the information they are using.
My wife who is a rheumatologist said it best, "I know that guideline-based medicine isn't great care, but I don't have the time or skills to look up each condition in the literature." This is the kind of honestly that also should be applauded but won't be well received by patients. I have a niche practice with a far smaller patient load. I have the luxury and maybe the skills to look things up by reviewing papers. I'm probably in the 2% of medical practices that can have 1-1.5 hour appointments per patient. As medical care and our knowledge gets more complex we need more training and more time. But it seems that we've switched to hiring more non-physicians and spending less time per patient.
"Admitting uncertainty builds trust and lends weight to our claims of knowledge." I would love to see more of this in patient/consumer and public health communications.
Any doctor that I consult who will tell me "I do not now because medicine hasn't yet any answer to this, but let's see it there are any options" will immediately win my respect.
I know someone who has stage 4 COPD. He is very frustrated that his doctor will not tell him his life expectancy. I told him that the doctor doesn't tell him because he doesn't have a crystal ball with the answer. There are things that are unknown.
Nicely said. I wish I had your insight when I was a medical resident.
Where once guidelines could be looked upon as a compendium of the state of the science, these days they are merely knowledge translation tools decided upon in an opaque fashion by a group of people with conflicts of interest longer than my arm, and who pretend to know things with certainty when they don’t, whilst demonstrating a gobsmacking inability to critically appraise evidence.
As a result, I find many guidelines these days less “required reading” and more “liner for birdcage”.
A suggestion for medical LLM developers and those interested in such things: train a particular LLM AI tool using medical science publications from different times spans - e.g. training documents selected from material published between 1980 to 2000, 2000 to 2020 - and then ask the tools so trained to solve/answer predetermined clinical problems. What would remain the same or similar between these two eras and what significant differences might emerge…?
First, being in medicine is learning to live with ambiguity. One must understand that. Secondly don't believe every thing that is written. The "new" a fib papers are all funded by device manufacturers. I usually discount industry funded papers by ~30%. A fib ablations are the new money maker the 2020s. Do they really live longer or better? What we do know is that their insurance companies are dinged for $100,000 to put them back into sinus rhythm. And they still have to take anticoagulant for the rest of their lives! Go back to the study from 20 years ago when it showed exactly this. The people with ablations had a little more sinus rhythm, but all I had to show for it was more time in the hospital and more money spent. A lot more money.
After going through all the sieves to get into school, when you’re facing the Head and Neck anatomy and physiology test that covered 350 pages in the textbook, you come to understand where the phrase “ 70 equals MD “ came from.
We need to foster comfort with uncertainty and discourage rigidity in medical education. I teach in the anatomy lab, and I'm always very surprised by how negatively some students will react when told to change course from their dissection guides (because of a surgery in a donor or other variables), even when it comes from the faculty responsible for writing said guide.
I would add (and my work is mostly untangling "why is X not working, I followed the guidelines" questions, when the usual is the obvious, the diagnosis is wrong) - I would add, the usual REASON the diagnosis is wrong is following consensus pathways without the addition of clinical reasoning.
I usually say, "AI 'thinks' like a medical student" (yes, I know it's not actually thinking). What I mean by this is it analyzes diagnosis as if it is a fact-stacking problem and not as an insight-driven engagement USING facts but clarified by the senses and by pattern recognition. Medical students have facts but no patterns, and their senses are not yet trained.
Given the limitation that by the time I am seeing a complex problem AI may have already failed at diagnosis - seeing errors is not uncommon.
In my first week of medical school, in 1972 (when they bring in old guys to tell you what the shot is, before you get down to work), an experienced clinician told us, "You will all want to practice in accord with the literature" (that's what they called EBM back in the day). He went on to say, "There are two problems with that. First - Half of what is in the literature is wrong. Second - we don't know which half".
There is nothing new under the sun, except that the "half is wrong" fraction may be growing.
A thoughtful, clearly articulated and convincing argument for the application of thought to clinical problems.
This is particularly true for residents and subspecialty fellows (not medical students) who focus on learning how their local institution manages particular problems, and, when in doubt, fall back on “authoritative” guidelines that are, at best, compromise documents.
Guidelines are often the refuge of the rushed physician in the trenches, to their patients’ detriment.
Similar issues in nursing school: My students don't tolerate ambiguity. "The doctor ordered it, so it must be okay. The hospital guidelines can't be wrong, etc." My students do not appreciate my questioning established protocols. The underlying issue is how to help students and practitioners become comfortable questioning all medications, interventions, etc. I constantly say to my nursing students, "Is this really doing what we think it is doing, and is there a better way?"
I thank Sensible Medicine for teaching me to critically appraise guidelines: they examined the recent Lipid Lowering Guidelines and exposed the lack of concrete research results underlying even the Class I recommendations. These guidelines state as fact actions that aren’t supported by RCT results. Yet, as this writer states, many take the guidelines at face value without critical appraisal. And medico-legal pressures cause blind adherence to the guidelines. What a mess. And I really worry about the LLM/AI compilations of studies that will also pick out the ones that support big pharma or device makers. What is one to do?
Well said. I think one aspect of the problem is the way we select medical students. People who do well in undergrad science courses and have high MCAT scores are people who are good at memorizing things. That's a very different skill--indeed, a very different mindset--from conducting research, analyzing conflicting information, wrestling with ambiguity.
People who become medical students all are very good at figuring out what "they" want and giving it to them. When they become doctors, part of the socialization process is transferring the identity of "them" to patients and families instead of teachers and test writers.
Unfortunately, this created a vulnerability when they started in with testing after boards, and especially with the shift to employment and the EHR.
Sickness is big, scary, and filled with uncertainty. Dealing with it is the Work. How much more satisfying is it to chase your grade on MOS exams, or be the best at "accurate" coding?
This critique misses a fundamental point: clinical guidelines are not written for patients, and they are not intended to replace individualized clinical judgment. They are written principally by professional societies and expert panels for clinicians and health systems, to synthesize evidence, grade its certainty, define reasonable standards of care, and reduce unwarranted variation.
Good guidelines do not “launder uncertainty.” They explicitly characterize it through evidence grading, conditional recommendations, qualifying statements, and acknowledged evidence gaps. The physician’s job is then to interpret that population-level recommendation in the context of the individual patient.
That is importantly different from an LLM producing a fluent answer to a question. Guidelines can certainly be wrong, outdated, biased, or poorly constructed, and they deserve scrutiny. But equating a professionally developed guideline with an AI-generated response obscures rather than clarifies the real problem: whether clinicians understand the provenance, evidentiary strength, limitations, and intended audience of the information they are using.
My wife who is a rheumatologist said it best, "I know that guideline-based medicine isn't great care, but I don't have the time or skills to look up each condition in the literature." This is the kind of honestly that also should be applauded but won't be well received by patients. I have a niche practice with a far smaller patient load. I have the luxury and maybe the skills to look things up by reviewing papers. I'm probably in the 2% of medical practices that can have 1-1.5 hour appointments per patient. As medical care and our knowledge gets more complex we need more training and more time. But it seems that we've switched to hiring more non-physicians and spending less time per patient.
"Admitting uncertainty builds trust and lends weight to our claims of knowledge." I would love to see more of this in patient/consumer and public health communications.
Any doctor that I consult who will tell me "I do not now because medicine hasn't yet any answer to this, but let's see it there are any options" will immediately win my respect.
I know someone who has stage 4 COPD. He is very frustrated that his doctor will not tell him his life expectancy. I told him that the doctor doesn't tell him because he doesn't have a crystal ball with the answer. There are things that are unknown.
Nicely said. I wish I had your insight when I was a medical resident.
Where once guidelines could be looked upon as a compendium of the state of the science, these days they are merely knowledge translation tools decided upon in an opaque fashion by a group of people with conflicts of interest longer than my arm, and who pretend to know things with certainty when they don’t, whilst demonstrating a gobsmacking inability to critically appraise evidence.
As a result, I find many guidelines these days less “required reading” and more “liner for birdcage”.
A suggestion for medical LLM developers and those interested in such things: train a particular LLM AI tool using medical science publications from different times spans - e.g. training documents selected from material published between 1980 to 2000, 2000 to 2020 - and then ask the tools so trained to solve/answer predetermined clinical problems. What would remain the same or similar between these two eras and what significant differences might emerge…?
First, being in medicine is learning to live with ambiguity. One must understand that. Secondly don't believe every thing that is written. The "new" a fib papers are all funded by device manufacturers. I usually discount industry funded papers by ~30%. A fib ablations are the new money maker the 2020s. Do they really live longer or better? What we do know is that their insurance companies are dinged for $100,000 to put them back into sinus rhythm. And they still have to take anticoagulant for the rest of their lives! Go back to the study from 20 years ago when it showed exactly this. The people with ablations had a little more sinus rhythm, but all I had to show for it was more time in the hospital and more money spent. A lot more money.
After going through all the sieves to get into school, when you’re facing the Head and Neck anatomy and physiology test that covered 350 pages in the textbook, you come to understand where the phrase “ 70 equals MD “ came from.
Great article!
We need to foster comfort with uncertainty and discourage rigidity in medical education. I teach in the anatomy lab, and I'm always very surprised by how negatively some students will react when told to change course from their dissection guides (because of a surgery in a donor or other variables), even when it comes from the faculty responsible for writing said guide.
I would add (and my work is mostly untangling "why is X not working, I followed the guidelines" questions, when the usual is the obvious, the diagnosis is wrong) - I would add, the usual REASON the diagnosis is wrong is following consensus pathways without the addition of clinical reasoning.
I usually say, "AI 'thinks' like a medical student" (yes, I know it's not actually thinking). What I mean by this is it analyzes diagnosis as if it is a fact-stacking problem and not as an insight-driven engagement USING facts but clarified by the senses and by pattern recognition. Medical students have facts but no patterns, and their senses are not yet trained.
Given the limitation that by the time I am seeing a complex problem AI may have already failed at diagnosis - seeing errors is not uncommon.
In my first week of medical school, in 1972 (when they bring in old guys to tell you what the shot is, before you get down to work), an experienced clinician told us, "You will all want to practice in accord with the literature" (that's what they called EBM back in the day). He went on to say, "There are two problems with that. First - Half of what is in the literature is wrong. Second - we don't know which half".
There is nothing new under the sun, except that the "half is wrong" fraction may be growing.
Excellent quote!
A thoughtful, clearly articulated and convincing argument for the application of thought to clinical problems.
This is particularly true for residents and subspecialty fellows (not medical students) who focus on learning how their local institution manages particular problems, and, when in doubt, fall back on “authoritative” guidelines that are, at best, compromise documents.
Guidelines are often the refuge of the rushed physician in the trenches, to their patients’ detriment.
Similar issues in nursing school: My students don't tolerate ambiguity. "The doctor ordered it, so it must be okay. The hospital guidelines can't be wrong, etc." My students do not appreciate my questioning established protocols. The underlying issue is how to help students and practitioners become comfortable questioning all medications, interventions, etc. I constantly say to my nursing students, "Is this really doing what we think it is doing, and is there a better way?"
This excellent article shows a level of understanding that is unusual for one so early in their medical experience.
I thank Sensible Medicine for teaching me to critically appraise guidelines: they examined the recent Lipid Lowering Guidelines and exposed the lack of concrete research results underlying even the Class I recommendations. These guidelines state as fact actions that aren’t supported by RCT results. Yet, as this writer states, many take the guidelines at face value without critical appraisal. And medico-legal pressures cause blind adherence to the guidelines. What a mess. And I really worry about the LLM/AI compilations of studies that will also pick out the ones that support big pharma or device makers. What is one to do?
"What is one to do?"
Not borrow $300,000 to get a job that is being replaced by the NP + AI treatment factory?
Well said. I think one aspect of the problem is the way we select medical students. People who do well in undergrad science courses and have high MCAT scores are people who are good at memorizing things. That's a very different skill--indeed, a very different mindset--from conducting research, analyzing conflicting information, wrestling with ambiguity.
People who become medical students all are very good at figuring out what "they" want and giving it to them. When they become doctors, part of the socialization process is transferring the identity of "them" to patients and families instead of teachers and test writers.
Unfortunately, this created a vulnerability when they started in with testing after boards, and especially with the shift to employment and the EHR.
Sickness is big, scary, and filled with uncertainty. Dealing with it is the Work. How much more satisfying is it to chase your grade on MOS exams, or be the best at "accurate" coding?