I am going to pose a query that has been troubling me regarding "evidence-based Medicine" and the use of RCTs as the de facto clinical standard.
I am often confronted with a "Clinical Intuition" that the guidelines based on such massive RCTs, with intrinsic inter-subject heterogeneity are not relevant to the patient in front of me.
Can clinicians rely on opinion and intuition?
When, if ever, is it OK to deviate from RCT-evidence based guidelines that don't capture the essence of the patient in front of me?
Are RCTs universally relevant to individualized/personalized medicine, or do they represent a public health tool for the construction of algorithms that one can readily deviate from without being condemned?
In the last 10 years in particular, I have had significant misgivings about most guidelines. Where once they were (I believed) compendiums of the state of the science, they have since become mere knowledge translations tools, where nuances of the evidence are sacrificed in favour of “simplicity” and “ease of adoption”. (On my snarkier days, I wonder what role industry played in this transition).
Fortunately I work in Canada where “guidelines” are only that (as opposed to attaining the level of “commandments” as they seem to in the US) so I routinely disregard guidelines when they deviate from the foundational evidence.
The problem is that, while I am familiar with the details of the trials in my field, I have no such knowledge level in any other field of medicine. So on those occasions when I venture out of my lane (such as on CCU week when I have to manage multi-system problems) I have to rely on guidelines of other specialties…and I suspect those have become bastardized versions of the evidence as well but am in no position to pass judgement on them.
Anecdotal, but every person I've known who has been placed on beta blockers complained about the side effect of fatigue. For some it was debilitating, even after dosage adjustment, and they independently discontinued treatment. That risk/detrminent might be included in the final essay paragraph.
Yes. When I read the part about the minimal side effects of beta blockers, my first thought was that this was written by someone who either hasn't prescribed a lot of them or doesn't listen to the complaints of patients who take them.
There seems be no discussion about the fact that the REDUCE AMI trial randomized patients 1-7 days after admission (therefore these patients likely received the initial treatment with beta blockers) and also excluded patients who are high risk such as low LVEF and those who resemble the prior MI trials. I would think that early use of beta blockers in patients with MI and selective continuation in high risk patients may still be a valuable approach.
Agreed. I think it’s fine to “start” most ACS pts on BB on admission…before you know the LV function, anatomy, and degree of revasc (and hence residual burden of ischemia).
Where Reduce-AMI has changed my practice is in deciding for whom I will discharge on BB.
Standards of care may be the antithesis of good medical practice. Guidelines may be helpful but, unfortunately, are often rigidified into standards. I suspect that lawyers probably love them. A frequent and continuing problem in medical research is the incorporation of a medication or procedure into a standard when the evidence is a very small difference in incidence between the experimental and control groups. Often these marginal figures are described as "robust" evidence. The pattern in the studies presented here is repeated frequently---a very small difference in the endpoints in an early study is said to be "statistically significant" and the drug or procedure is then heavily promoted on that basis. Further studies then provide evidence that the differences were of no practical significance. One lesson should be to remain skeptical of the initial evidence until serious confirmation can be attained.
Totally agree. “Statistical significance “ and “clinical significance” are often 2 very different things. That’s why I’m a ARR/NNT guy, and very much dislike reporting of RRR.
Yes, I think relative risk reduction numbers are often used as a deliberate deception. Actual risk reduction given in percentage differences is easily understood by both professional and lay people. I am not a big fan of NNT but at least it is relatively honest compared to RRR.
This is just one example of the folly of Guideline or Protocol or just plain Cookbook Medicine that has taken over the practice of medicine since the 1970s (?) and the rise of “evidence-based” medicine.
1. You don’t know that the correct or complete scientific data is included in the cookbook recipe.
2. The recipe, even if basically correct, doesn’t apply to the 5-10% of the population falling outside the standard curve.
3. While guidelines were initially constructed as a “floor” to keep poorly educated (or just dumb) doctors from practicing substandard care, they quickly became “ceilings” that gave insurance companies – led by Medicaid and Medicare – ammunition to deny coverage due to “over treatment” AND … allowed medical boards and hospital committees to prosecute doctors who did more than the guidelines indicated.
I’ve harped on this since at least the 1970s and now it’s pretty much enshrined in the EMR systems. 🤷
Not in your field; conventional western medicine never my 1st choice, and generally don't have a super high view of its practitioners (sorry, no offence intended). But then I don't often come across people like you, who articulate the issues so well. Makes me think that people like you could restore the faith that many in the US have lost in healthcare.
I appreciate the sentiment. The problem is, there is half a century of bureaucracy and regulations (meaning money) providing an inertia that is hard to overcome. Some of us are just too old now and simply represent voices crying out in the wilderness. 🤷
Exactly. I remember being told by my favored instructors/attendings to “treat the patient, not the numbers”. Except that with the advent of the EMR along with direct pharmaceutical advertising to patients and the rise of the internet, patients (and the insurance companies and lawyers) seem to demand cookbook medicine. The art of treating the patient in front of you has been lost.
great context... thanks for the history... is the evidence for BB wrt arrhythmia up to date as well? or should that be questioned? just observational but in my small sample the evidence does not seem super compelling
I graduated from medical school in 2006. I distinctly remember being told that beta blockers provide minimal to no benefit post MI. We were told that patients still died whenever they would die, they just had better numbers at death. Yet they would continue to prescribe these patients beta blockers to take the rest of their lives.
I find it interesting that even a seminal cardiovascular trial that universally changed treatment for a common condition had an ARR of 3%. Meaning 97% of treated patients received NO benefit. And consider that most drugs we use with near universal consensus do not even reach that degree of benefit.
Giving the patient the info in an understandable way so to permit an informed decision is the point; time consuming/ often under reimbursed/ frequently countered with “ what would you do” ; nobody said being a good doc was easy.
I believe 3% ARR means that the average patient had a 3% lower chance of experiencing the aforementioned outcomes (ie subsequent MI or HF or whatever). It does NOT mean that you had a 3% chance of experiencing any benefit (which would imply 97% chance of no effect or harm). Lowering your absolute chance of those outcomes by 3% is a pretty big deal imo.
Someone please correct me if I misinterpreted the stats here!
I believe it means each individual patient in the treatment group has a 30% lower chance of experiencing the event (7 vs 10) - this is the relative risk reduction which works out to a 3% absolute risk reduction given the event rate.
An example of a 3% ARR would be if a drug reduces the adverse outcome from 10% without it to 7% with it. What you think about this level of risk reduction is a value judgement—that’s why to me the whole “safe and effective” mantra about the Covid vaccine was absurd; we humans vary greatly in the things we value and how we interpret and handle risk. If you are 1 of the 100 people in either arm of a trial aiming to reduce a particular outcome from 10%to 7% for the treatment group—I personally would say you aren’t likely to have the undesired outcome in either group, right? just numerically speaking. Not only that, I as a physician can treat all 100 patients with the drug, so 3 will be spared the undesirable outcome that otherwise would have had it, but I don’t know WHICH 3 people will be the fortunate ones, and an additional 7 will still have the undesirable outcome. And we haven’t even considered potential adverse effects of the treatment.
I see what you’re saying, basically you saved an extra 3 out of 100 patients you treated. I do think that the right way of looking at it is not from the NNT doctor perspective but from the patient perspective. You still reduced the absolute risk of all your patients by 3% on average. Like you said, some patients will find that valuable and others won’t, but idk if I would approach it from NNT.
I appreciate your response. I personally like the NNT approach, and feel it is one way to discuss risk with patients. There’s also a great website that has NNTs for all sorts of screening tests and treatments. Another way to approach risk and benefit with patients is to calculate/estimate their absolute risk with and without a treatment. I have done this with the PREVENT calculator for heart disease. Definitely there is not a single approach that will work for all people.
When I'm a patient, I want to know the NNT - it's useful information about how beneficial the proposed treatment is, versus the waffle verbiage doctors bullshit patients with. And I'm a med student.
It might well be that it’s clearer for the patient. I guess my point is that the framing makes it seem as if only 3 patients out of 100 benefitted when in reality they all did (since all of them should’ve decreased risk by 3%).
I don’t think you can say all treated patients benefited by a 3% reduction in risk, at least insofar as some likely had an increase in risk due to adverse effects of the treatment—we haven’t accounted for that here.
Yes, having thought about it extensively since I retired and started posting on Substack, I really struggle with the idea that we MUST treat a group to see a specific outcome for a small (and unidentifiable) subset of the group. It assumes that the entire group value a specific outcome equally, and that they also weight the benefits and harms in the same way that the experts do.
In the post-MI scenario, the assumption is that everybody will do whatever it takes to reduce the risk of a second MI (even if the risk is fairly small and the reduction in risk is even smaller), including taking daily medication, complete with side-effects, forever more.
There are some analogies to the vaccination scenario, although arguably vaccinating everyone has some benefits to the herd, whereas treating everyone with beta-blockers post-MI has no benefit to the herd, only to the small group that didn't have the second MI they might otherwise have had.
I agree! Having practiced as a family doctor from 1982 through 2021, it was the “standard of care” to give beta-blockers post-MI. Patients were told it was essential. Doctors who didn’t prescribe them were seen as incompetent. And yet, only a small fraction of the patients (if any) actually benefitted from the drugs.
It falls into what I refer to as the public health approach to medicine. We’ve moved away from individualized interventions for individual patients for their personal benefit, which is to exceed their personal harms. Now, we treat (and sometimes harm) large numbers of people, hoping that a few will benefit.
It’s kind of sloppy, when you think about it. It would be like giving broad spectrum antibiotics to everyone with a fever, knowing that only a small fraction of the group would have a bacterial infection responsive to the antibiotic chosen. Of course, that would be malpractice — we try to localize the source of the fever, identify the responsible agent, and choose an antibiotic (or not) accordingly. With the post-MI crowd, however, we just give everyone the same class of drug, with no effort to determine who actually benefits. Big Pharma likes that approach, because they sell lots of drugs. Good luck getting them to sponsor a drug trial showing how to prescribe more appropriately.
I am a primary care internist pretty similar in age and years practicing as you and agree 100% with your observation about medical practice—that today’s docs are operating as public health agents rather than advocates for each individual’s health. I have been thinking exactly as you on this, and if we don’t fix this, we will be entirely replaced by AI and the vast and growing community of Advanced Practice Providers—ARNPs and PAs.
As you mention, a doctor who fails to adhere to these “guidelines,” for whatever (well thought out) reason, is at increased risk from the lawyers should his/her patient have a bad outcome. I don’t have an answer, but tort reform would be a good start!
Yes. I ended up spending a lot of time documenting what I had suggested to the patient, what they chose not to do, and why. That being said, I’m sure they still could have sued me, on the basis that I misinformed them.
I thank the Heavens I take no drugs or vaccines after living 75 years. Seeing as how life expectancy is dropping steadily in the US, drugs and vaccines have not made for a longer life.
Perhaps a more important story would be when mRNA vaccines are still being advised, despite the overwhelming, independent evidence of plasmid DNA contamination (with the SV40 promoter) contained within the LNP transfection vehicle...
True, but there is now ample evidence among the “awake” that mRNA vaccines are universally bad medicine, kill people, sterilize people, cause dementia, promote cancer, etc. It’s time to move on. Constantly harping on how bad the 5-year-old mRNA vaccines are prevents discussions such as this about other aspects of medicine causing harm. What if we took all our available time posting, commenting on, talking about, and conducting trials to prove how bad the smallpox and polio vaccines are? Yes, they’re bad medicine, too, but it’s time to move onto other, more recent examples of bad medicine.
True, i agree there are 1,001 other problems with medicine.
Two pushbacks
1) Sensible Medicine, as far as I know, has not acknowledged these point. They still believe there are benefits to the jabs.
2) existing vaccines are being redesigned on the mRNA platform. The issue I raise is a fundamental flaw in the platform, so this will continue to be a problem.
Okay, valid points. I have not been following Sensible Medicine very long (< 1 month), so I’m not aware of their thoughts on the depopulation shot. And yes, the messenger RNA platform is fundamentally flawed. If anybody in the medical profession doesn’t know that by now, they must be vaccinated. (See what I did there? :-)
I am going to pose a query that has been troubling me regarding "evidence-based Medicine" and the use of RCTs as the de facto clinical standard.
I am often confronted with a "Clinical Intuition" that the guidelines based on such massive RCTs, with intrinsic inter-subject heterogeneity are not relevant to the patient in front of me.
Can clinicians rely on opinion and intuition?
When, if ever, is it OK to deviate from RCT-evidence based guidelines that don't capture the essence of the patient in front of me?
Are RCTs universally relevant to individualized/personalized medicine, or do they represent a public health tool for the construction of algorithms that one can readily deviate from without being condemned?
In the last 10 years in particular, I have had significant misgivings about most guidelines. Where once they were (I believed) compendiums of the state of the science, they have since become mere knowledge translations tools, where nuances of the evidence are sacrificed in favour of “simplicity” and “ease of adoption”. (On my snarkier days, I wonder what role industry played in this transition).
Fortunately I work in Canada where “guidelines” are only that (as opposed to attaining the level of “commandments” as they seem to in the US) so I routinely disregard guidelines when they deviate from the foundational evidence.
The problem is that, while I am familiar with the details of the trials in my field, I have no such knowledge level in any other field of medicine. So on those occasions when I venture out of my lane (such as on CCU week when I have to manage multi-system problems) I have to rely on guidelines of other specialties…and I suspect those have become bastardized versions of the evidence as well but am in no position to pass judgement on them.
Anecdotal, but every person I've known who has been placed on beta blockers complained about the side effect of fatigue. For some it was debilitating, even after dosage adjustment, and they independently discontinued treatment. That risk/detrminent might be included in the final essay paragraph.
Yes. When I read the part about the minimal side effects of beta blockers, my first thought was that this was written by someone who either hasn't prescribed a lot of them or doesn't listen to the complaints of patients who take them.
There seems be no discussion about the fact that the REDUCE AMI trial randomized patients 1-7 days after admission (therefore these patients likely received the initial treatment with beta blockers) and also excluded patients who are high risk such as low LVEF and those who resemble the prior MI trials. I would think that early use of beta blockers in patients with MI and selective continuation in high risk patients may still be a valuable approach.
Agreed. I think it’s fine to “start” most ACS pts on BB on admission…before you know the LV function, anatomy, and degree of revasc (and hence residual burden of ischemia).
Where Reduce-AMI has changed my practice is in deciding for whom I will discharge on BB.
Standards of care may be the antithesis of good medical practice. Guidelines may be helpful but, unfortunately, are often rigidified into standards. I suspect that lawyers probably love them. A frequent and continuing problem in medical research is the incorporation of a medication or procedure into a standard when the evidence is a very small difference in incidence between the experimental and control groups. Often these marginal figures are described as "robust" evidence. The pattern in the studies presented here is repeated frequently---a very small difference in the endpoints in an early study is said to be "statistically significant" and the drug or procedure is then heavily promoted on that basis. Further studies then provide evidence that the differences were of no practical significance. One lesson should be to remain skeptical of the initial evidence until serious confirmation can be attained.
Totally agree. “Statistical significance “ and “clinical significance” are often 2 very different things. That’s why I’m a ARR/NNT guy, and very much dislike reporting of RRR.
Wondering if you're the Steve who trained in Adelaide?
Yes, I think relative risk reduction numbers are often used as a deliberate deception. Actual risk reduction given in percentage differences is easily understood by both professional and lay people. I am not a big fan of NNT but at least it is relatively honest compared to RRR.
Same for ICDs in primary prevention
If you want an easy death, no to the ICD!!! Every time.
Interesting read and I agree that standards of care should be reviewed and changed more often.
However in this particular case, a monthly course of atenolol cost per patient is about $4.
It seems the basic problem here is that we spend too much and in doing so, we are getting sicker.
There must be far bigger fish to fry than a $4 a month med that doesn't help but doesn't hurt.
I understand it would take some real guts to go after the big game, we need some Alan Quatermains right now.
This is just one example of the folly of Guideline or Protocol or just plain Cookbook Medicine that has taken over the practice of medicine since the 1970s (?) and the rise of “evidence-based” medicine.
1. You don’t know that the correct or complete scientific data is included in the cookbook recipe.
2. The recipe, even if basically correct, doesn’t apply to the 5-10% of the population falling outside the standard curve.
3. While guidelines were initially constructed as a “floor” to keep poorly educated (or just dumb) doctors from practicing substandard care, they quickly became “ceilings” that gave insurance companies – led by Medicaid and Medicare – ammunition to deny coverage due to “over treatment” AND … allowed medical boards and hospital committees to prosecute doctors who did more than the guidelines indicated.
I’ve harped on this since at least the 1970s and now it’s pretty much enshrined in the EMR systems. 🤷
Not in your field; conventional western medicine never my 1st choice, and generally don't have a super high view of its practitioners (sorry, no offence intended). But then I don't often come across people like you, who articulate the issues so well. Makes me think that people like you could restore the faith that many in the US have lost in healthcare.
I appreciate the sentiment. The problem is, there is half a century of bureaucracy and regulations (meaning money) providing an inertia that is hard to overcome. Some of us are just too old now and simply represent voices crying out in the wilderness. 🤷
Exactly. I remember being told by my favored instructors/attendings to “treat the patient, not the numbers”. Except that with the advent of the EMR along with direct pharmaceutical advertising to patients and the rise of the internet, patients (and the insurance companies and lawyers) seem to demand cookbook medicine. The art of treating the patient in front of you has been lost.
Exactly!
And so the 5-10% of patients who are outliers to the “protocol” are lost to the calculus of the cookbook. 🤦
great context... thanks for the history... is the evidence for BB wrt arrhythmia up to date as well? or should that be questioned? just observational but in my small sample the evidence does not seem super compelling
I graduated from medical school in 2006. I distinctly remember being told that beta blockers provide minimal to no benefit post MI. We were told that patients still died whenever they would die, they just had better numbers at death. Yet they would continue to prescribe these patients beta blockers to take the rest of their lives.
I find it interesting that even a seminal cardiovascular trial that universally changed treatment for a common condition had an ARR of 3%. Meaning 97% of treated patients received NO benefit. And consider that most drugs we use with near universal consensus do not even reach that degree of benefit.
Giving the patient the info in an understandable way so to permit an informed decision is the point; time consuming/ often under reimbursed/ frequently countered with “ what would you do” ; nobody said being a good doc was easy.
I believe 3% ARR means that the average patient had a 3% lower chance of experiencing the aforementioned outcomes (ie subsequent MI or HF or whatever). It does NOT mean that you had a 3% chance of experiencing any benefit (which would imply 97% chance of no effect or harm). Lowering your absolute chance of those outcomes by 3% is a pretty big deal imo.
Someone please correct me if I misinterpreted the stats here!
I believe it means each individual patient in the treatment group has a 30% lower chance of experiencing the event (7 vs 10) - this is the relative risk reduction which works out to a 3% absolute risk reduction given the event rate.
An example of a 3% ARR would be if a drug reduces the adverse outcome from 10% without it to 7% with it. What you think about this level of risk reduction is a value judgement—that’s why to me the whole “safe and effective” mantra about the Covid vaccine was absurd; we humans vary greatly in the things we value and how we interpret and handle risk. If you are 1 of the 100 people in either arm of a trial aiming to reduce a particular outcome from 10%to 7% for the treatment group—I personally would say you aren’t likely to have the undesired outcome in either group, right? just numerically speaking. Not only that, I as a physician can treat all 100 patients with the drug, so 3 will be spared the undesirable outcome that otherwise would have had it, but I don’t know WHICH 3 people will be the fortunate ones, and an additional 7 will still have the undesirable outcome. And we haven’t even considered potential adverse effects of the treatment.
I see what you’re saying, basically you saved an extra 3 out of 100 patients you treated. I do think that the right way of looking at it is not from the NNT doctor perspective but from the patient perspective. You still reduced the absolute risk of all your patients by 3% on average. Like you said, some patients will find that valuable and others won’t, but idk if I would approach it from NNT.
I appreciate your response. I personally like the NNT approach, and feel it is one way to discuss risk with patients. There’s also a great website that has NNTs for all sorts of screening tests and treatments. Another way to approach risk and benefit with patients is to calculate/estimate their absolute risk with and without a treatment. I have done this with the PREVENT calculator for heart disease. Definitely there is not a single approach that will work for all people.
When I'm a patient, I want to know the NNT - it's useful information about how beneficial the proposed treatment is, versus the waffle verbiage doctors bullshit patients with. And I'm a med student.
It might well be that it’s clearer for the patient. I guess my point is that the framing makes it seem as if only 3 patients out of 100 benefitted when in reality they all did (since all of them should’ve decreased risk by 3%).
I don’t think you can say all treated patients benefited by a 3% reduction in risk, at least insofar as some likely had an increase in risk due to adverse effects of the treatment—we haven’t accounted for that here.
Yes, having thought about it extensively since I retired and started posting on Substack, I really struggle with the idea that we MUST treat a group to see a specific outcome for a small (and unidentifiable) subset of the group. It assumes that the entire group value a specific outcome equally, and that they also weight the benefits and harms in the same way that the experts do.
In the post-MI scenario, the assumption is that everybody will do whatever it takes to reduce the risk of a second MI (even if the risk is fairly small and the reduction in risk is even smaller), including taking daily medication, complete with side-effects, forever more.
There are some analogies to the vaccination scenario, although arguably vaccinating everyone has some benefits to the herd, whereas treating everyone with beta-blockers post-MI has no benefit to the herd, only to the small group that didn't have the second MI they might otherwise have had.
I agree! Having practiced as a family doctor from 1982 through 2021, it was the “standard of care” to give beta-blockers post-MI. Patients were told it was essential. Doctors who didn’t prescribe them were seen as incompetent. And yet, only a small fraction of the patients (if any) actually benefitted from the drugs.
It falls into what I refer to as the public health approach to medicine. We’ve moved away from individualized interventions for individual patients for their personal benefit, which is to exceed their personal harms. Now, we treat (and sometimes harm) large numbers of people, hoping that a few will benefit.
It’s kind of sloppy, when you think about it. It would be like giving broad spectrum antibiotics to everyone with a fever, knowing that only a small fraction of the group would have a bacterial infection responsive to the antibiotic chosen. Of course, that would be malpractice — we try to localize the source of the fever, identify the responsible agent, and choose an antibiotic (or not) accordingly. With the post-MI crowd, however, we just give everyone the same class of drug, with no effort to determine who actually benefits. Big Pharma likes that approach, because they sell lots of drugs. Good luck getting them to sponsor a drug trial showing how to prescribe more appropriately.
I am a primary care internist pretty similar in age and years practicing as you and agree 100% with your observation about medical practice—that today’s docs are operating as public health agents rather than advocates for each individual’s health. I have been thinking exactly as you on this, and if we don’t fix this, we will be entirely replaced by AI and the vast and growing community of Advanced Practice Providers—ARNPs and PAs.
As you mention, a doctor who fails to adhere to these “guidelines,” for whatever (well thought out) reason, is at increased risk from the lawyers should his/her patient have a bad outcome. I don’t have an answer, but tort reform would be a good start!
Yes. I ended up spending a lot of time documenting what I had suggested to the patient, what they chose not to do, and why. That being said, I’m sure they still could have sued me, on the basis that I misinformed them.
I thank the Heavens I take no drugs or vaccines after living 75 years. Seeing as how life expectancy is dropping steadily in the US, drugs and vaccines have not made for a longer life.
Perhaps a more important story would be when mRNA vaccines are still being advised, despite the overwhelming, independent evidence of plasmid DNA contamination (with the SV40 promoter) contained within the LNP transfection vehicle...
True, but there is now ample evidence among the “awake” that mRNA vaccines are universally bad medicine, kill people, sterilize people, cause dementia, promote cancer, etc. It’s time to move on. Constantly harping on how bad the 5-year-old mRNA vaccines are prevents discussions such as this about other aspects of medicine causing harm. What if we took all our available time posting, commenting on, talking about, and conducting trials to prove how bad the smallpox and polio vaccines are? Yes, they’re bad medicine, too, but it’s time to move onto other, more recent examples of bad medicine.
True, i agree there are 1,001 other problems with medicine.
Two pushbacks
1) Sensible Medicine, as far as I know, has not acknowledged these point. They still believe there are benefits to the jabs.
2) existing vaccines are being redesigned on the mRNA platform. The issue I raise is a fundamental flaw in the platform, so this will continue to be a problem.
Okay, valid points. I have not been following Sensible Medicine very long (< 1 month), so I’m not aware of their thoughts on the depopulation shot. And yes, the messenger RNA platform is fundamentally flawed. If anybody in the medical profession doesn’t know that by now, they must be vaccinated. (See what I did there? :-)
100%