You have to know what questions to ask. So I think it's not going to replace docs, in the near future anyway. But it sure saves a ton of time in looking things up.
I think critics underestimate just how many doctors are out there that don't ask follow-up questions or really listen to their patients.
Even regardless of that, AI has the benefit of being immediately available at any time of day or night and for question after question after question as they come up.
AI’s biggest limitation in medicine is not knowledge retrieval but clinical reasoning under uncertainty. The evidence is becoming clear that without structured history taking and contextual filtering, even highly capable models can misjudge acuity, which is why AI should currently function as a decision support tool rather than an independent triage system.
Thank you for raising the body language element of the in person encounter. Others have raised this point as well and it may be an AI limitation for a long time, although I suspect we will have cameras associated with patient AI input which might assess these body elements...
I’m sure you are aware that much, perhaps most of the information gleaned from a patient encounter is non-verbal. Body language. Hard to measure this, but most experienced physicians believe it is real. As you infer overall…AI is a tool only.
Being a patient now having been involved with forensic medicine for 40+ yrs what I now have seen with minimal actual exams by various specialities does little to find the nature of complex conditions….thus the deduced results like that of AI would be likely quite comparable. It takes time to get to the depth of understanding what it really means being a physician…med school from the 70s taught 3 D thinking…true forensic deduction…gleaming the unique variables..something AI lacks.
On the one hand that is called: GIGO. Or as the computer on Star Trek would say: Analysis Complete. Insufficient data to resolve the problem. If you don't give the electronic computer or the human computer (aka the Dr.) enough information they can't solve the problem. If you come in to complain of foot pain and you forget to mention that an elephant stepped on your foot it's not really the Dr.'s fault.
A friend had a minimally traumatic compression fracture followed by a bout of sepsis. Labs showed mild anemia and slight gamma globulin elevation. It was surmised that his sepsis was related to his urinary tract and he started cathing himself 3 times a day. He then had another compression fracture. Nine months after that he was again admitted for sepsis. Finally he was correctly diagnosed with multiple myeloma which AI would have concluded during his first bout of sepsis.
Since starting appropriate treatment he continues to have moderate severe back pain. Against the advice of his urologist, he has stopped the caths. AI would have saved him from the caths and chronic pain. AI will lead to significant improvement in diagnostic accuracy sooner than many professionals believe.
AI doesn't take a history or perform an exam. Neither do many humans employed in the delivery of care. The results converge.
Three years ago, retired and then 76, I encountered an 81 y.o. former patient. We'd been friends since childhood. She was in a wheelchair. Her daughter, an ICU trained RN, told me she had a paralyzed left leg mystifying her physicians.
"When did this start?"
"June 1st [2 weeks earlier] I was dancing with my partner when he gave me a 2 hand push I didn't expect. I lost my balance and fell and immediately was in excruciating pain in my knee and couldn't get up" "I was carried to the car."
"So you went to the ER where x rays were negative for fracture and they sent you home?"
"Right. Then I went to my family doctor's office where they ordered an MRI of my spine."
Using my laptop I viewed her thoraco-lumbar MRI. Other than 50% loss of L4-5 disk height it was perfect, incredible for age 81. I viewed the T2, T1, and sagitals, axials, coronals. The formal report concurred.
I examined her in the wheelchair. Her L knee was flexed 75 degrees. Active and passive L knee extension was limited at about 45 deg by anterolateral joint line pain. Tenderness was in the same location. Except for superficial varicosities the neuromuscular, vascular and joint exam of both lower extremities was otherwise normal.
"You tore a meniscus in a weight bearing twisting fall, and probably it's the anterior horn of the lateral meniscus [which it was]." "I'll get you to an orthopedic surgeon."
"You mean I should cancel the epidural injection scheduled for Thursday." [She'd also been seen by interventional radiology - pain management!]
"Richard, you're the only person who has asked me how this happened and you are the only one who examined my knee."
I could go on with other examples. What has our profession become? I don't recall such idiocy in my years of school and training followed by 36 years of private internal medicine practice. Yes, AI may make it worse, but worse than what?
I remember a doctor leaving to read an article about my condition. Returning to report back that cycling made it worse. She did not ask if it made it worse for me, it was in fact one of the few things I'd found that took the pain away. I wanted to be anywhere else in the world so I took my referral to the specialist that didn't help me with the condition I actually don't have but interestingly told me I should have googled it myself before seeing him.
It's been my experience that doctors pressed for time don't want to know more and if someone shows up with a diagnosis, right or wrong they stick to it. It hasn't felt that different than the experience with AI.
Excellent piece. What struck me most was the point about “sycophancy” and the failure to ask clarifying questions.
In neurology, many dangerous presentations begin as vague phenomenology:
“the room feels wrong,”
“my body feels strange,”
“this headache is different.”
The challenge is not just matching symptoms to diagnoses; it’s knowing which missing question changes the entire differential.
A skilled clinician knows that a normal exam is just a snapshot, and that a normal scan does not always mean a normal brain. But an AI that down-weights urgency based on a layperson’s interpretation is dangerous in the gray zones of medicine. Often, the most important clinical detail is precisely the one the patient does not realize matters enough to mention.
AI is becoming an extraordinary medical librarian. But it is still a limited detective.
I ran into this same problem when I built HealthScout, a health AI app for patients. Foundation models like ChatGPT are trained to answer immediately. So I customized HealthScout to ask clarifying questions first, which leads to much better answers.
Excellent stress test of AI. Hadn’t seen this angle before. It augurs well with the general mantra of making your questions/prompts to be as detailed and precise as possible.
But it’s interesting that you’ve demonstrated AI to be bad at taking a history, at least unless you instruct it to actually take a history. This is valuable insight.
You have to know what questions to ask. So I think it's not going to replace docs, in the near future anyway. But it sure saves a ton of time in looking things up.
Perplexity is a better AI than ChatGPT.
I think critics underestimate just how many doctors are out there that don't ask follow-up questions or really listen to their patients.
Even regardless of that, AI has the benefit of being immediately available at any time of day or night and for question after question after question as they come up.
AI’s biggest limitation in medicine is not knowledge retrieval but clinical reasoning under uncertainty. The evidence is becoming clear that without structured history taking and contextual filtering, even highly capable models can misjudge acuity, which is why AI should currently function as a decision support tool rather than an independent triage system.
Thank you for raising the body language element of the in person encounter. Others have raised this point as well and it may be an AI limitation for a long time, although I suspect we will have cameras associated with patient AI input which might assess these body elements...
I’m sure you are aware that much, perhaps most of the information gleaned from a patient encounter is non-verbal. Body language. Hard to measure this, but most experienced physicians believe it is real. As you infer overall…AI is a tool only.
Being a patient now having been involved with forensic medicine for 40+ yrs what I now have seen with minimal actual exams by various specialities does little to find the nature of complex conditions….thus the deduced results like that of AI would be likely quite comparable. It takes time to get to the depth of understanding what it really means being a physician…med school from the 70s taught 3 D thinking…true forensic deduction…gleaming the unique variables..something AI lacks.
You wrote the magic word early in the article: Judgement. It can't be programmed or computerized.
On the one hand that is called: GIGO. Or as the computer on Star Trek would say: Analysis Complete. Insufficient data to resolve the problem. If you don't give the electronic computer or the human computer (aka the Dr.) enough information they can't solve the problem. If you come in to complain of foot pain and you forget to mention that an elephant stepped on your foot it's not really the Dr.'s fault.
A friend had a minimally traumatic compression fracture followed by a bout of sepsis. Labs showed mild anemia and slight gamma globulin elevation. It was surmised that his sepsis was related to his urinary tract and he started cathing himself 3 times a day. He then had another compression fracture. Nine months after that he was again admitted for sepsis. Finally he was correctly diagnosed with multiple myeloma which AI would have concluded during his first bout of sepsis.
Since starting appropriate treatment he continues to have moderate severe back pain. Against the advice of his urologist, he has stopped the caths. AI would have saved him from the caths and chronic pain. AI will lead to significant improvement in diagnostic accuracy sooner than many professionals believe.
A very compelling story for a thoughtful history and physical exam
AI doesn't take a history or perform an exam. Neither do many humans employed in the delivery of care. The results converge.
Three years ago, retired and then 76, I encountered an 81 y.o. former patient. We'd been friends since childhood. She was in a wheelchair. Her daughter, an ICU trained RN, told me she had a paralyzed left leg mystifying her physicians.
"When did this start?"
"June 1st [2 weeks earlier] I was dancing with my partner when he gave me a 2 hand push I didn't expect. I lost my balance and fell and immediately was in excruciating pain in my knee and couldn't get up" "I was carried to the car."
"So you went to the ER where x rays were negative for fracture and they sent you home?"
"Right. Then I went to my family doctor's office where they ordered an MRI of my spine."
Using my laptop I viewed her thoraco-lumbar MRI. Other than 50% loss of L4-5 disk height it was perfect, incredible for age 81. I viewed the T2, T1, and sagitals, axials, coronals. The formal report concurred.
I examined her in the wheelchair. Her L knee was flexed 75 degrees. Active and passive L knee extension was limited at about 45 deg by anterolateral joint line pain. Tenderness was in the same location. Except for superficial varicosities the neuromuscular, vascular and joint exam of both lower extremities was otherwise normal.
"You tore a meniscus in a weight bearing twisting fall, and probably it's the anterior horn of the lateral meniscus [which it was]." "I'll get you to an orthopedic surgeon."
"You mean I should cancel the epidural injection scheduled for Thursday." [She'd also been seen by interventional radiology - pain management!]
"Richard, you're the only person who has asked me how this happened and you are the only one who examined my knee."
I could go on with other examples. What has our profession become? I don't recall such idiocy in my years of school and training followed by 36 years of private internal medicine practice. Yes, AI may make it worse, but worse than what?
I remember a doctor leaving to read an article about my condition. Returning to report back that cycling made it worse. She did not ask if it made it worse for me, it was in fact one of the few things I'd found that took the pain away. I wanted to be anywhere else in the world so I took my referral to the specialist that didn't help me with the condition I actually don't have but interestingly told me I should have googled it myself before seeing him.
It's been my experience that doctors pressed for time don't want to know more and if someone shows up with a diagnosis, right or wrong they stick to it. It hasn't felt that different than the experience with AI.
Excellent piece. What struck me most was the point about “sycophancy” and the failure to ask clarifying questions.
In neurology, many dangerous presentations begin as vague phenomenology:
“the room feels wrong,”
“my body feels strange,”
“this headache is different.”
The challenge is not just matching symptoms to diagnoses; it’s knowing which missing question changes the entire differential.
A skilled clinician knows that a normal exam is just a snapshot, and that a normal scan does not always mean a normal brain. But an AI that down-weights urgency based on a layperson’s interpretation is dangerous in the gray zones of medicine. Often, the most important clinical detail is precisely the one the patient does not realize matters enough to mention.
AI is becoming an extraordinary medical librarian. But it is still a limited detective.
Beautifully put!
Hello.
This essay about AI shows that an appropriate review of systems can be very useful.
I ran into this same problem when I built HealthScout, a health AI app for patients. Foundation models like ChatGPT are trained to answer immediately. So I customized HealthScout to ask clarifying questions first, which leads to much better answers.
I wrote an article starting with nearly the same prompt you used and compared ChatGPT to HealthScout: https://www.joesipher.com/p/generic-ai-recommended-a-drug-i-cant
Excellent stress test of AI. Hadn’t seen this angle before. It augurs well with the general mantra of making your questions/prompts to be as detailed and precise as possible.
But it’s interesting that you’ve demonstrated AI to be bad at taking a history, at least unless you instruct it to actually take a history. This is valuable insight.