I think if AI works in conjunction with doctors, we will all be better off. It's improving at an exponential rate. I'm not even sure what that means exactly but I know it's really fast. Similarly I don't profess to understand the medical terminology you use but I get the gist;) sabrinalabow.substack.com
Eric Topol is not credible. He was one of the worst, most unforgivably extreme and dishonest COVID hawks during the pandemic. He only sees evidence that confirms his worldview, and he's pathetic enough that he thinks his visibility on twitter makes him special.
The results will be skewed by the personal bias of the doctors notes. I fell from my semi-truck when I thought I was on the bottom stair. I landed on the hard February asphalt with enough force to snap the metal buckle on my hat and cause me to experience a full body stunning. For moments I could not move. I was disoriented and required help getting up. The next year I caught a bad sinus infection and had a bout of aseptic meningitis. I recovered and the MRI showed hypervascular growth in the area of my previous impact. At the time the Doctor decided that I must have epilepsy because I had a seizure during the meningitis. Two years later I had another bout of meningitis with a seizure. I was then put on epilepsy medication. I have regularly asked if the epilepsy caused the meningitis or the other way around. I cannot do my work as a truck driver because of the epilepsy medication/diagnosis. I am only on of 8 that OHSU have seen with hypervascular growth in my brain. I am now getting yearly MRI to keep track of the growth. At least until they decide it's not worth their time to follow-up.
My main objection to AI in medicine is that it is impossible to program judgement---the principal ingredient of good medical care. Once these things gain a foothold, they tend to take over and it is extremely difficult to dislodge them. I had many years experience in overreading EKGs at the hospital where I practiced. Over the years the hospital tried to remove the human element and rely on the computer generated interpretation. But it only took a few lawsuits because of obvious errors to persuade them to contract with cardiologists to review and rewrite, if necessary, the computer diagnoses. Another example is the game of bridge. There is so much judgement involved in the play of the hands that computer bridge has never been very satisfactory---especially in playing defense.
I think I share with many my concern that our lives are increasingly governed by devices and algorithms. With that, physicians are losing what individual personality and perspicacity they bring to the practice of medicine. I persist in the belief that the best practice of medicine is driven by both curiosity and compassion which can only be conveyed by persistent questioning and the human touch. I am no Luddite, but if our knowledge is in our pockets and our decisions made by machines, our practice of medicine will be reduced to functions of the Help Desk. Nobody looks forward to that.
Machine learning, artificial intelligence (AI) and data shifts are now discussed almost daily. The fact that there is a growing profusion of rhetorical questions about AI indicates how many unanswered questions remain, how uncertain the future impact of AI will be. Nevertheless, visions of AI are bifurcating; in one camp, the apocalyptic, in another, AI as oracular and the arbiter of what data means. Regardless, there is no stopping its application to medicine. Indeed, the New England Journal Group has launched a new journal, NEJM AI.
I understand the enthusiasm, but I feel a certain dread as AI slowly diffuses into the diagnosis and management of disease. These are not the concerns about AI “hallucinations”, which are treatable, or the epistemic concerns about reproducibility, the uninterpretability of the output. These are real concerns but not mine now; I’m concerned AI will continue to erode physician autonomy and lead us further into algorithmic medicine that progressively, unwittingly, ignores the human qualities clinicians bring to medicine: deep experience, a critical and questioning intellect and the desire to address the unique circumstances of a patient. In short, clinicians bring humanity and its nuance to the bedside.
One point that deserves a more expansive discussion is the creation of guidelines. The development of evidence-based treatment guidelines has been a productive endeavor over the last fifty years in an effort to derive a common understanding of the vast and complicated literature on treatments and outcomes. Experts, thoroughly familiar with a given disease, together sift through what is invariably an incomplete data set to extract basic principles and from them, practices. Many of the recommendations represent a consensus, rarely unanimous, on what the data should dictate. That discussion is rarely available; the compromises are unstated, their product drained of any hot blood. But there are references! Guidelines, when displayed as flow charts borrowed from industrial engineering, imply there are no other choices and convey an air of technical certainty. Necessarily, their synthesis must err on the side of caution balancing cost, inconvenience, toxicity and the prospect of a better outcome. Clinicians come to trust guidelines not because they understand the literature as deeply but largely for reputational reasons; they respect the organizations or academics that generate or endorse the guidelines. We acknowledge: few of us could alone sit down and critically process such a volume of literature.
There are special challenges in generating treatment guidelines for diseases with long latencies or distant relapses: ten-year outcomes may be the relevant endpoint. Guidelines generated from such long-term studies will have begun 15 or 20 years ago, using treatments available at the time. Such studies are often the product of whole careers, are expensive, often not repeated, and, by definition, do not incorporate any newer thinking. Guidelines are only as good as the data and sometimes not even that good.
The grip of guidelines was brought home to me when a family member was diagnosed with breast cancer, a new life event, sadly, 150,000+ women in the US experience a year. Very quickly her clinical state was reduced to a handful of Yes/Nos and tumor parameters, seven to be exact. The treatment regimen, governed entirely by guidelines, was implemented expertly and expeditiously. The surgeon, the radiation oncologist, the medical oncologist seemed to have no agency. There was the simulacrum of choice, but is the choice between a 80% and 90% 5-year overall survival much of a choice? The practice of personalized medicine seems in peril when a patient is one of a large population defined by 6 or 7 numbers. My genome is 6 x 109 letters and yet it is not the sum of me.
A quick (and somewhat desperate) read of the literature showed that not an insignificant number of breast cancer patients on standard estrogen suppression regimes have estrogens in the normal range. Are estrone and estradiol measured? Not mentioned in the guidelines. As total cholesterol rises on such a regime, are the lipophilic statins favored over hydrophilic statins as the literature suggests? Not in the breast cancer treatment guidelines. Is there specific mention of the importance of managing inflammation, a factor favoring the selection of tumor cells? No. Each of these and other management decisions might add a percent or two to the total of those that survive 5 years – why not reach for those few percent? In the management of patients with a potentially lethal disease, every possible advantage should be considered.
Let’s not fault the guidelines…. or their makers. Large effects are more compelling than small ones. And it may be impracticable to measure small effects sizes for breast cancer. Certainly guidelines change, sometime radically, to wit, recommendations for salt intake or screening for prostatic cancer. But guidelines clearly have the strong effect of luring clinicians into the illusion of agency and understanding. The fallacy is this: guidelines must incorporate all that is important to know and because I know the guidelines, I must know all there is to know. No need to explore or think outside the guidelines.
But AI will be much more seductive. Thousands of papers could be digested, not dozens. There is the illusion that AI has no bias as the arbiter of what is good data and bad; the “infallibility of big data” will conspire to create the illusion of understanding. I can hear the AI advocates now – “all of the data will be addressed by the Oracle of AI so the new treatment guidelines will be far superior”. Yes, AI will be better for certain applications like image analysis wherein a radiologist can look at the film. But for others where AI presents a “model” with thousands of parameters, it becomes inscrutable – we cannot interrogate for ourselves. And what is better? For certain guidelines, it might take a 10-year outcomes study to know. Will we come to trust the pronouncements of AI because their output exceeds our own cognitive abilities or even those of a committee of guideline developers? Will such surrender render us completely ovine? Will AI guidelines “fight” for attention and respect with the expert human-generated guidelines (as some guidelines do with each other now)?
Perhaps guidelines should be explicitly less proscriptive, similar to judicial sentencing guidelines that offer judges wide discretion taking into account the specific circumstances unique to the case. Guidelines could point out promising results not yet solidified into hard data but providing, maybe even provoking, clinicians to consider tailoring their management plan. Should we not address now the intellectual constraints imposed by current guidelines before AI takes over their creation? Sometimes we need a poke to make us alive to a gathering storm. Guidelines have their place unless they make us complacent; AI-generated guidelines have a much greater power to consolidate that complacency.
Exodus 17:11 - "As long as Moses held up his hands, the Israelites were winning, but whenever he lowered his hands, the Amalekites were winning."
A philosopher's aim is to "know" in order to be wise. Mostly knowing that you don't know. So of little practical use. Mostly your grandparents and parents (and your conscious) can tell you what is the right thing to do.
A scientist's aim is also to "know" but limits her focus to those things which are falsifiable and can be generalized. It is special kind of epistome.
A physician's aim is to cure the ill [1]while doing no harm. The process involves "techne" - it's active, it involves doing (sometimes not doing is doing). "Knowing" is relevant in so far as it helps the aim of doing no harm while curing the ill. But it's secondary. If placebo effect works so long as no harm, the physician takes the win.
Probably a decade or so ago I had an opportunity to take Andrew Ng's machine learning class online. It can be maddening if you are a philosopher or a scientist.(since turning 60 I will admit publicly to being a philosopher)[2]
Neural networks, once there is a sufficient number of parameters to be useful for prediction, are black boxes - no ability to "understand".
Curse of dimensionality: as number of parameters increases so does amount of spurious correlations.
Placebos - how do they work?
Stratification is essentially as I see it a clustering problem. But, there is no way to optimize (so far [3]) the number of groups. 2, 5, 7, 223? There are methods but these cannot guarantee "optimal" number of groups.
Stratification of risk is only useful if you can treat to make an actual positive difference for actual patient. Populations are not people.
For the philosopher, scientist and physician (really everybody), it is important to not be fooled by randomness.
But for the physician iff you are curing AND doing no harm - it's not weird if it works.
Written in haste
[1] Sometimes people are ill before we or they know it.
[2] The idea of "data science" might be right up there with "military intelligence". See oxymoron.
[3] Jenks natural breaks should be used more in medicine in place of arbitrary stratification and Shewhart's/Demming's ideas about method (with chance of being wrong) to discern between special causes of variation and systemic causes of variation. Each kind of cause requires different kinds of action for improvement.
I have been underwhelmed by AI's contributions in the cardiovascular field and was not impressed by this paper. My focus is on individual ASCVD risk and that assessment comes most logically from looking directly for ASCVD (atqheroscleroc plaque) in the coronary arteries.
The AI tool for this that most impresses me takes data from coronary CT angiograms on the amount of soft and calcified plaque in an individual's coronary arteries and is called Cleerly. (https://www.jacc.org/doi/10.1016/j.jcmg.2023.05.020)
As to Eric Topol, who, as you say tends towards breathless optimism. because he lacks a filter for weak AI (and observational COVID) studies I don't recommend readers follow him
Medical AI is developed exactly the same way as clinical risk scores: Decide on some relevant features based on current clinical conventional wisdom, gather a bunch of patients, fit some model to the features that coarsely predicts risk, send to JAMA. The only difference is that AI adds in complex measurement artifacts that are likely noise but it might decide are relevant.
If you're skeptical clinical risk scores, you should be more as skeptical of AI. Both are just repackaging conventional wisdom in a calculator.
Thanks for this excellent analysis, Adam, and the insights you have provided. It's futile to resist the role that AI will play in healthcare delivery in the years to come. We are only seeing the beginning of a new era. Although AI will perform better than doctors in many spheres, I am of the opinion that it will never supplant physicians for one reason. An often-quoted proverb states, "A physician often treats, sometimes cures, but always consoles." The supporting and consoling function will necessitate individuals. Future physicians will have to acknowledge this as their primary responsibility. Consequently, the entire process of selecting young individuals to become physicians will need to transform from favouring scientific intellectuals to humanists with emotional intelligence and compassion.
“I am looking forward to a time when AI can do better than we can do now. I am most looking forward to AI telling us how it is doing what it is doing so we can learn.” If AI could do what we once could but better and faster, why would we need to learn?
I’m not sure anyone is playing out the endgame here (except for those worried about some malevolent AI like you alluded to at the end of the essay). What are we hoping for? Like, what would be the best outcome of AI development? That machines do all the work we currently do?
It feels a bit like the promises made to families when household appliances were invented. “Think of how much time you’ll save with this vacuum cleaner!” Now I’m not downplaying the importance of the vacuum cleaner, but more technology has just made us more harried and anxious.
Byung-chul Han argues that this feeling has come upon us because time lacks rhythm and an orienting purpose. We whizz around from event to event, never contemplating, never lingering.
Do we really think AI will make healthcare more humane? What technical development in the last 200 years has given us an indication that’s the direction we’re going? I worry AI will degrade the quality of our work. What are we hoping for? And on what basis can we hope to retain what’s good in any of our experiences - either the practice of medicine or the experience of receiving healthcare?
I think if AI works in conjunction with doctors, we will all be better off. It's improving at an exponential rate. I'm not even sure what that means exactly but I know it's really fast. Similarly I don't profess to understand the medical terminology you use but I get the gist;) sabrinalabow.substack.com
Eric Topol is not credible. He was one of the worst, most unforgivably extreme and dishonest COVID hawks during the pandemic. He only sees evidence that confirms his worldview, and he's pathetic enough that he thinks his visibility on twitter makes him special.
The results will be skewed by the personal bias of the doctors notes. I fell from my semi-truck when I thought I was on the bottom stair. I landed on the hard February asphalt with enough force to snap the metal buckle on my hat and cause me to experience a full body stunning. For moments I could not move. I was disoriented and required help getting up. The next year I caught a bad sinus infection and had a bout of aseptic meningitis. I recovered and the MRI showed hypervascular growth in the area of my previous impact. At the time the Doctor decided that I must have epilepsy because I had a seizure during the meningitis. Two years later I had another bout of meningitis with a seizure. I was then put on epilepsy medication. I have regularly asked if the epilepsy caused the meningitis or the other way around. I cannot do my work as a truck driver because of the epilepsy medication/diagnosis. I am only on of 8 that OHSU have seen with hypervascular growth in my brain. I am now getting yearly MRI to keep track of the growth. At least until they decide it's not worth their time to follow-up.
My main objection to AI in medicine is that it is impossible to program judgement---the principal ingredient of good medical care. Once these things gain a foothold, they tend to take over and it is extremely difficult to dislodge them. I had many years experience in overreading EKGs at the hospital where I practiced. Over the years the hospital tried to remove the human element and rely on the computer generated interpretation. But it only took a few lawsuits because of obvious errors to persuade them to contract with cardiologists to review and rewrite, if necessary, the computer diagnoses. Another example is the game of bridge. There is so much judgement involved in the play of the hands that computer bridge has never been very satisfactory---especially in playing defense.
I think I share with many my concern that our lives are increasingly governed by devices and algorithms. With that, physicians are losing what individual personality and perspicacity they bring to the practice of medicine. I persist in the belief that the best practice of medicine is driven by both curiosity and compassion which can only be conveyed by persistent questioning and the human touch. I am no Luddite, but if our knowledge is in our pockets and our decisions made by machines, our practice of medicine will be reduced to functions of the Help Desk. Nobody looks forward to that.
Machine learning, artificial intelligence (AI) and data shifts are now discussed almost daily. The fact that there is a growing profusion of rhetorical questions about AI indicates how many unanswered questions remain, how uncertain the future impact of AI will be. Nevertheless, visions of AI are bifurcating; in one camp, the apocalyptic, in another, AI as oracular and the arbiter of what data means. Regardless, there is no stopping its application to medicine. Indeed, the New England Journal Group has launched a new journal, NEJM AI.
I understand the enthusiasm, but I feel a certain dread as AI slowly diffuses into the diagnosis and management of disease. These are not the concerns about AI “hallucinations”, which are treatable, or the epistemic concerns about reproducibility, the uninterpretability of the output. These are real concerns but not mine now; I’m concerned AI will continue to erode physician autonomy and lead us further into algorithmic medicine that progressively, unwittingly, ignores the human qualities clinicians bring to medicine: deep experience, a critical and questioning intellect and the desire to address the unique circumstances of a patient. In short, clinicians bring humanity and its nuance to the bedside.
One point that deserves a more expansive discussion is the creation of guidelines. The development of evidence-based treatment guidelines has been a productive endeavor over the last fifty years in an effort to derive a common understanding of the vast and complicated literature on treatments and outcomes. Experts, thoroughly familiar with a given disease, together sift through what is invariably an incomplete data set to extract basic principles and from them, practices. Many of the recommendations represent a consensus, rarely unanimous, on what the data should dictate. That discussion is rarely available; the compromises are unstated, their product drained of any hot blood. But there are references! Guidelines, when displayed as flow charts borrowed from industrial engineering, imply there are no other choices and convey an air of technical certainty. Necessarily, their synthesis must err on the side of caution balancing cost, inconvenience, toxicity and the prospect of a better outcome. Clinicians come to trust guidelines not because they understand the literature as deeply but largely for reputational reasons; they respect the organizations or academics that generate or endorse the guidelines. We acknowledge: few of us could alone sit down and critically process such a volume of literature.
There are special challenges in generating treatment guidelines for diseases with long latencies or distant relapses: ten-year outcomes may be the relevant endpoint. Guidelines generated from such long-term studies will have begun 15 or 20 years ago, using treatments available at the time. Such studies are often the product of whole careers, are expensive, often not repeated, and, by definition, do not incorporate any newer thinking. Guidelines are only as good as the data and sometimes not even that good.
The grip of guidelines was brought home to me when a family member was diagnosed with breast cancer, a new life event, sadly, 150,000+ women in the US experience a year. Very quickly her clinical state was reduced to a handful of Yes/Nos and tumor parameters, seven to be exact. The treatment regimen, governed entirely by guidelines, was implemented expertly and expeditiously. The surgeon, the radiation oncologist, the medical oncologist seemed to have no agency. There was the simulacrum of choice, but is the choice between a 80% and 90% 5-year overall survival much of a choice? The practice of personalized medicine seems in peril when a patient is one of a large population defined by 6 or 7 numbers. My genome is 6 x 109 letters and yet it is not the sum of me.
A quick (and somewhat desperate) read of the literature showed that not an insignificant number of breast cancer patients on standard estrogen suppression regimes have estrogens in the normal range. Are estrone and estradiol measured? Not mentioned in the guidelines. As total cholesterol rises on such a regime, are the lipophilic statins favored over hydrophilic statins as the literature suggests? Not in the breast cancer treatment guidelines. Is there specific mention of the importance of managing inflammation, a factor favoring the selection of tumor cells? No. Each of these and other management decisions might add a percent or two to the total of those that survive 5 years – why not reach for those few percent? In the management of patients with a potentially lethal disease, every possible advantage should be considered.
Let’s not fault the guidelines…. or their makers. Large effects are more compelling than small ones. And it may be impracticable to measure small effects sizes for breast cancer. Certainly guidelines change, sometime radically, to wit, recommendations for salt intake or screening for prostatic cancer. But guidelines clearly have the strong effect of luring clinicians into the illusion of agency and understanding. The fallacy is this: guidelines must incorporate all that is important to know and because I know the guidelines, I must know all there is to know. No need to explore or think outside the guidelines.
But AI will be much more seductive. Thousands of papers could be digested, not dozens. There is the illusion that AI has no bias as the arbiter of what is good data and bad; the “infallibility of big data” will conspire to create the illusion of understanding. I can hear the AI advocates now – “all of the data will be addressed by the Oracle of AI so the new treatment guidelines will be far superior”. Yes, AI will be better for certain applications like image analysis wherein a radiologist can look at the film. But for others where AI presents a “model” with thousands of parameters, it becomes inscrutable – we cannot interrogate for ourselves. And what is better? For certain guidelines, it might take a 10-year outcomes study to know. Will we come to trust the pronouncements of AI because their output exceeds our own cognitive abilities or even those of a committee of guideline developers? Will such surrender render us completely ovine? Will AI guidelines “fight” for attention and respect with the expert human-generated guidelines (as some guidelines do with each other now)?
Perhaps guidelines should be explicitly less proscriptive, similar to judicial sentencing guidelines that offer judges wide discretion taking into account the specific circumstances unique to the case. Guidelines could point out promising results not yet solidified into hard data but providing, maybe even provoking, clinicians to consider tailoring their management plan. Should we not address now the intellectual constraints imposed by current guidelines before AI takes over their creation? Sometimes we need a poke to make us alive to a gathering storm. Guidelines have their place unless they make us complacent; AI-generated guidelines have a much greater power to consolidate that complacency.
Thanks for writing, again!
Bud light commercial: it's not weird if it works.
Exodus 17:11 - "As long as Moses held up his hands, the Israelites were winning, but whenever he lowered his hands, the Amalekites were winning."
A philosopher's aim is to "know" in order to be wise. Mostly knowing that you don't know. So of little practical use. Mostly your grandparents and parents (and your conscious) can tell you what is the right thing to do.
A scientist's aim is also to "know" but limits her focus to those things which are falsifiable and can be generalized. It is special kind of epistome.
A physician's aim is to cure the ill [1]while doing no harm. The process involves "techne" - it's active, it involves doing (sometimes not doing is doing). "Knowing" is relevant in so far as it helps the aim of doing no harm while curing the ill. But it's secondary. If placebo effect works so long as no harm, the physician takes the win.
Probably a decade or so ago I had an opportunity to take Andrew Ng's machine learning class online. It can be maddening if you are a philosopher or a scientist.(since turning 60 I will admit publicly to being a philosopher)[2]
Neural networks, once there is a sufficient number of parameters to be useful for prediction, are black boxes - no ability to "understand".
Curse of dimensionality: as number of parameters increases so does amount of spurious correlations.
Placebos - how do they work?
Stratification is essentially as I see it a clustering problem. But, there is no way to optimize (so far [3]) the number of groups. 2, 5, 7, 223? There are methods but these cannot guarantee "optimal" number of groups.
Stratification of risk is only useful if you can treat to make an actual positive difference for actual patient. Populations are not people.
For the philosopher, scientist and physician (really everybody), it is important to not be fooled by randomness.
But for the physician iff you are curing AND doing no harm - it's not weird if it works.
Written in haste
[1] Sometimes people are ill before we or they know it.
[2] The idea of "data science" might be right up there with "military intelligence". See oxymoron.
[3] Jenks natural breaks should be used more in medicine in place of arbitrary stratification and Shewhart's/Demming's ideas about method (with chance of being wrong) to discern between special causes of variation and systemic causes of variation. Each kind of cause requires different kinds of action for improvement.
I have been underwhelmed by AI's contributions in the cardiovascular field and was not impressed by this paper. My focus is on individual ASCVD risk and that assessment comes most logically from looking directly for ASCVD (atqheroscleroc plaque) in the coronary arteries.
The AI tool for this that most impresses me takes data from coronary CT angiograms on the amount of soft and calcified plaque in an individual's coronary arteries and is called Cleerly. (https://www.jacc.org/doi/10.1016/j.jcmg.2023.05.020)
As to Eric Topol, who, as you say tends towards breathless optimism. because he lacks a filter for weak AI (and observational COVID) studies I don't recommend readers follow him
(https://theskepticalcardiologist.com/2019/08/10/are-you-taking-a-statin-drug-inappropriately-like-eric-topol-because-of-the-mygenerank-app/)
AP
Medical AI is developed exactly the same way as clinical risk scores: Decide on some relevant features based on current clinical conventional wisdom, gather a bunch of patients, fit some model to the features that coarsely predicts risk, send to JAMA. The only difference is that AI adds in complex measurement artifacts that are likely noise but it might decide are relevant.
If you're skeptical clinical risk scores, you should be more as skeptical of AI. Both are just repackaging conventional wisdom in a calculator.
Thanks for this excellent analysis, Adam, and the insights you have provided. It's futile to resist the role that AI will play in healthcare delivery in the years to come. We are only seeing the beginning of a new era. Although AI will perform better than doctors in many spheres, I am of the opinion that it will never supplant physicians for one reason. An often-quoted proverb states, "A physician often treats, sometimes cures, but always consoles." The supporting and consoling function will necessitate individuals. Future physicians will have to acknowledge this as their primary responsibility. Consequently, the entire process of selecting young individuals to become physicians will need to transform from favouring scientific intellectuals to humanists with emotional intelligence and compassion.
No way do I want A/I intruding into my life.
“I am looking forward to a time when AI can do better than we can do now. I am most looking forward to AI telling us how it is doing what it is doing so we can learn.” If AI could do what we once could but better and faster, why would we need to learn?
I’m not sure anyone is playing out the endgame here (except for those worried about some malevolent AI like you alluded to at the end of the essay). What are we hoping for? Like, what would be the best outcome of AI development? That machines do all the work we currently do?
It feels a bit like the promises made to families when household appliances were invented. “Think of how much time you’ll save with this vacuum cleaner!” Now I’m not downplaying the importance of the vacuum cleaner, but more technology has just made us more harried and anxious.
Byung-chul Han argues that this feeling has come upon us because time lacks rhythm and an orienting purpose. We whizz around from event to event, never contemplating, never lingering.
Do we really think AI will make healthcare more humane? What technical development in the last 200 years has given us an indication that’s the direction we’re going? I worry AI will degrade the quality of our work. What are we hoping for? And on what basis can we hope to retain what’s good in any of our experiences - either the practice of medicine or the experience of receiving healthcare?