firstly as my doctor I would hope you would look at all cause mortality , not just heart disease – that the lower your LDL , the higher the all cause mortality , risks for dementia etc , the NNT , NNH of the statins and their side effects etc . It is obvious the guidelines are a desperate attempt by the AHA and Big Pharma to support profits in the face of the collapse of the Ancel Keys theory of cholesterol driven heat disease as low carb diets become the way to treat metabolic health .- This piece really captures how I feel about the guidelines
Agree. A couple of comments. Firstly, always remember 85% of ACC guidelines are not evidence based (JAMA 2019). They're based on how a few old men, who happen to be interested parties, felt on a given day. Secondly anybody issuing a 240 page guideline is trying to confuse you. Thirdly look at the Kaiser Permanente N. California paper from 2019 showing the 2013 ACC guidelines overestimated risk by 400%! Fourthly look at the this ACC guideline shilling for a clearly disproven test-CAC, the sole purpose of which is to perform more unnecessary angiograms hoping to insert more unnecessary stents in asymptomatic people. The authors seem to have ignored COURAGE and ISCHEMIA. Oh, I guess they're an inconvenient truth.
Guidelines are for populations of patients. Clinical decision making is for individuals. It is the responsibility of the primary care physician in conjunction with the patient to decide when the guidelines do not apply to that patient. If you are always coloring inside the box you risk being replaced by a computer bot.
“ A more balanced committee would include greater representation from clinical epidemiologists, biostatisticians, and evidence-based medicine specialists — individuals whose primary expertise is in evaluating evidence rather than treating a specific disease. Their relative underrepresentation is common across guideline development.”
Excellent essay. And this was the core point which resonated most with me. A “scientific” guideline, if that is what the lipid tome purports to be, should be a rigorous appraisal of the science. In that light, anything less than level A evidence has no place receiving a “class 1” recommendation. What we need are experts at parsing science, and not subject area experts/cheerleaders who hand out “class 1” like candy.
And if you’re going to elevate recommendations to class 1 without the requisite strength of evidence, then be honest about that and don’t call it “scientific”; it’s consensus opinion, or some other squishy euphemism.
This is why I have a far dimmer view of guidelines in general, than the author. I no longer follow guidelines, precisely because they are no longer strictly evidence-based. Add this to the list for sure, but the HF guidelines also rank highly for bastardization of the evidence.
And that is a great point about what EBM actually is. In the original Sackett conception, the “science” is only one part of the 3 part Venn diagram. Show me a doc who blindly and obsequiously “follows the guidelines”, and I’ll show you one who doesn’t know how to practice EBM.
But when evidence is inadequate to reliably define a ‘best practice’ as desired with EBM, expert consensus - which can be plausibly thought of as ‘anecdotal evidence’ is worth considering. Throwing out guidelines entirely as you seem to have done seems unwise. And forgive me - I do not intend to offend - but humility is a critical virtue to be cultivated…
I totally agree. I’m not advocating for the therapeutic nihilism of “there is no good evidence here so we will do nothing”.
The number of clinical scenarios for which there is no high quality scientific evidence far outnumbers the clinical questions for which there are. So absolutely, you have to use the best evidence you’ve got, on a case by case basis.
My objection to latter day guidelines is that, if they are purportedly the repository of the best science has to say on a subject, then they should in fact BE THAT. And elevating something to class 1 on the basis of low level evidence is NOT an example of it. There is no way anecdote or expert opinion or observational data can justify a class 1 recommendation of anything. This is why I consider much of what is in most guidelines today to be garbage.
As you know, and as the Pharma/industry who pay the people on these committees most certainly knows, once something becomes widely known as “class 1 recommendation”, few will go back and worry about the basis which undergirds it. Very much as intended, I would say.
My solution is to (try my level best to) know the evidence myself. So I practice to the evidence whenever possible and available, rather than to the guideline. This is somewhat feasible as I have a specialized area…but I find I am quickly out of my depth when venturing even slightly outside my field. And when I do have to rely on guidelines outside my immediate practice domain, I often wonder if those are garbage as much as the ones in my field have become. That is not how it should be, IMO.
Thanks again for this rejoinder. I understand you better and I find nothing of substance in your latest about which I would quibble. And with regard to your last statement - we are of one heart and mind with regard to skepticism… If we were from or living in Missouri, we would be especially at home as it is known as the ‘show me state’. I live in Idaho, a place where skepticism is also commonly found.
Nicely said, and on much we do agree - especially the need to be more clear and transparent regarding the strengths and weaknesses of guideline ratings.
Some quibbles: “As you know, and as the Pharma/industry who pay the people on these committees most certainly knows, once something becomes widely known as “class 1 recommendation”, few will go back and worry about the basis which undergirds it.”
During my practice as an Emergency Physician (and PhD pharmacologist), I would not infrequently diverge from guidelines based on the details of the clinical circumstances. So then, I was one of these few. And I was comfortable doing this as I was familiar with the ways in which guidelines were/are limited. But in that I believe those who do not so inform themselves are poorly serving their patients. There are, sadly, too many such…
I was - after I left clinical practice - a senior Medical Director for a small-to-midsized PPO/TPA in the Pacific Northwest. Within that context then…
- As a Medical Director reviewing cases that were not automatically approvable due to guideline non-compliance, I encountered many providers who thoughtfully disregarded a guideline for legitimate reasons. Perhaps you were one of these (I retired just over 4 years ago.) In many of these cases I understood the reasoning behind the divergence, and if I agreed with the reasoning, I gladly approved the coverage request. If I disagreed, of course, I would deny. (Of note - whenever I denied something, it was subject to appeal. About 75% of those appeals were upheld by disinterested independent reviewers chosen for their expertise relevant to the question.) My point here is that, in my experience (during my 11 years in this roll I reviewed over 20,000 cases) many providers disregard guidelines often for the same reasons you might.
- I do not know that ‘Pharmacy/industry’ pay the persons who serve on guideline committees. Having myself spoken professionally and personally with a number of such people, my impression was they were honest normal professionals just trying their best to do the right thing. There are outliers certainly and undue influence (both self interested and merely political) that discredits the efforts of the honest, but I do not share your (is it cynical) distrust of the system.
So we agree on much, and going forward we can hope for improvements.
I’ve never had an administrative role so approvals has never been my purview. It’s a lesser occurrence as I work in Canada, although work-related extended health insurance does come into play.
I think you misunderstood my point. I’m happy you and many many others disregard guidelines when the clinical scenario requires it. But in the paragraph you quoted, what I was trying to say is that, with (-in relatively short periods of) time, people no longer remember or care if it’s class 1 LOE A or class 1 LOE C; all class 1’s become equal. And imo they shouldn’t be. My apologies for not being clear.
On the point of “payment”, I’m being less precise than I should be, based on the forum. No, I do not intend to say that Pharma A cuts a cheque to Expert B to vote in committee for their product C. However, lots of “experts” are in fact on payroll directly as “consultants”, or receive research funding, sometimes specifically for working on product C. As Dr. JMM likes to say, maybe that’s “non-nefarious”. But it beggars belief to think that mere “disclosure” obviates bias. They have a vested interest in something, they’re more likely to vote for it. It’s human nature. It’s disclosed. It’s non-nefarious. And it should never be allowed to come to that. Besides, what makes a “subject matter expert” an expert in evidence appraisal? The people who generate the evidence should not be the ones who judge it. Quite literally, different expertise is required. As I’ve used the phrase before, I’d take a platoon of Dr. JMM’s “neutral Martian” EBM experts on a guideline committee over a bunch of subject matter experts any day of the week.
Am I a cynic? Probably. At the minimum, I am an extremely robust and vigorous skeptic.
Excellently written. Thank you so much. I couldn't agree more. After retiring from practice I studied and for the first time really understood the number to treat concept. When I understood the numbers for primary prevention of cardiovascular disease using a statin, (in general NNT= 100) realized that I had not been open or honest about the benefit of such therapy to my patients, because I did not understand NNT. Most patients assume effectiveness of a treatment will succeed closer to 100% of the time, not that there is a proven 99% chance the medication will not be of benefit to them. I am certain that would significantly change patient decision making. It is actually something most physicians do not understand. Promoting the NNT concept in clinical training and experience is needed much more than the attention it is given.
Trustworthy guidelines indeed should be based on systematic reviews of the evidence, and practitioners should be a minority of those on the panel, with conflicts of interest managed in a transparent way, which should be reported. There is a great overview in a 20 min talk by Ivan Flores, who has been leading the group to develop AGREE II, which evaluates guidelines: https://youtu.be/smRzaGjQU0Y
Not all guidelines are trustworthy.They aren't if they are not based on systematic reviews of the evidence and if the strength of the recommendations is not tied to the strength of the evidence (eg low quality/certainty evidence and strong recommendations is inappropriate except in some very specific situations). Consensus is not enough. See also the paper by Brignardello-Petersen et al. on how to use clinical practice guidelines: https://pubmed.ncbi.nlm.nih.gov/34665198/
The recommendations for treatment of gender dysphoria by most US medical associations do not meet these criteria. Many current guidelines and guidelines for pediatric gender dysphoria have been rigorously assessed in https://adc.bmj.com/content/109/Suppl_2/s65 . Most were found to be poor. Some more recent guidelines (Germany, for instance) are also also consensus based rather than based on systematic reviews of the evidence. Sweden and Finland do base their recommendations for pediatric gender dysphoria on systematic reviews of the evidence and did the best (this rigorous assessment was part of the Cass Review and thus did not evaluate the Cass Review final recommendations).
EXCELLENT points, thank you. Dr. Lesser and other primary care doctors should be on the AHA (and other) guideline committees, to help iron these issues out.
This is a great approach, and applied to guidelines generally. I find guidelines helpful, not least because it makes me think twice before I depart from, and if I do, I make it explicit in any documents why I did so. All or virtually all guidelines have - correctly - a kind of disclaimer, identifying that the treatment of an individual patient can transcend the guidelines (though they don’t put it like that). The issue can arise in clinical negligence cases, where apparently sleep walking through guideline boundaries plays poorly in court.
Sage overview of this document (Tome?). Especially vital: the importance of considering NNT as well as ARR in decision making. ie: take the population based paper and make it relevant to the person on your exam table.
firstly as my doctor I would hope you would look at all cause mortality , not just heart disease – that the lower your LDL , the higher the all cause mortality , risks for dementia etc , the NNT , NNH of the statins and their side effects etc . It is obvious the guidelines are a desperate attempt by the AHA and Big Pharma to support profits in the face of the collapse of the Ancel Keys theory of cholesterol driven heat disease as low carb diets become the way to treat metabolic health .- This piece really captures how I feel about the guidelines
https://drmalcolmkendrick.org/tag/aha/
Agree. A couple of comments. Firstly, always remember 85% of ACC guidelines are not evidence based (JAMA 2019). They're based on how a few old men, who happen to be interested parties, felt on a given day. Secondly anybody issuing a 240 page guideline is trying to confuse you. Thirdly look at the Kaiser Permanente N. California paper from 2019 showing the 2013 ACC guidelines overestimated risk by 400%! Fourthly look at the this ACC guideline shilling for a clearly disproven test-CAC, the sole purpose of which is to perform more unnecessary angiograms hoping to insert more unnecessary stents in asymptomatic people. The authors seem to have ignored COURAGE and ISCHEMIA. Oh, I guess they're an inconvenient truth.
Guidelines are for populations of patients. Clinical decision making is for individuals. It is the responsibility of the primary care physician in conjunction with the patient to decide when the guidelines do not apply to that patient. If you are always coloring inside the box you risk being replaced by a computer bot.
Great article- thank you for writing it
Excellent essay. I would ask that you try to get this in front of those at HHS who are mucking around with USPSTF…!
“ A more balanced committee would include greater representation from clinical epidemiologists, biostatisticians, and evidence-based medicine specialists — individuals whose primary expertise is in evaluating evidence rather than treating a specific disease. Their relative underrepresentation is common across guideline development.”
Excellent essay. And this was the core point which resonated most with me. A “scientific” guideline, if that is what the lipid tome purports to be, should be a rigorous appraisal of the science. In that light, anything less than level A evidence has no place receiving a “class 1” recommendation. What we need are experts at parsing science, and not subject area experts/cheerleaders who hand out “class 1” like candy.
And if you’re going to elevate recommendations to class 1 without the requisite strength of evidence, then be honest about that and don’t call it “scientific”; it’s consensus opinion, or some other squishy euphemism.
This is why I have a far dimmer view of guidelines in general, than the author. I no longer follow guidelines, precisely because they are no longer strictly evidence-based. Add this to the list for sure, but the HF guidelines also rank highly for bastardization of the evidence.
And that is a great point about what EBM actually is. In the original Sackett conception, the “science” is only one part of the 3 part Venn diagram. Show me a doc who blindly and obsequiously “follows the guidelines”, and I’ll show you one who doesn’t know how to practice EBM.
But when evidence is inadequate to reliably define a ‘best practice’ as desired with EBM, expert consensus - which can be plausibly thought of as ‘anecdotal evidence’ is worth considering. Throwing out guidelines entirely as you seem to have done seems unwise. And forgive me - I do not intend to offend - but humility is a critical virtue to be cultivated…
I totally agree. I’m not advocating for the therapeutic nihilism of “there is no good evidence here so we will do nothing”.
The number of clinical scenarios for which there is no high quality scientific evidence far outnumbers the clinical questions for which there are. So absolutely, you have to use the best evidence you’ve got, on a case by case basis.
My objection to latter day guidelines is that, if they are purportedly the repository of the best science has to say on a subject, then they should in fact BE THAT. And elevating something to class 1 on the basis of low level evidence is NOT an example of it. There is no way anecdote or expert opinion or observational data can justify a class 1 recommendation of anything. This is why I consider much of what is in most guidelines today to be garbage.
As you know, and as the Pharma/industry who pay the people on these committees most certainly knows, once something becomes widely known as “class 1 recommendation”, few will go back and worry about the basis which undergirds it. Very much as intended, I would say.
My solution is to (try my level best to) know the evidence myself. So I practice to the evidence whenever possible and available, rather than to the guideline. This is somewhat feasible as I have a specialized area…but I find I am quickly out of my depth when venturing even slightly outside my field. And when I do have to rely on guidelines outside my immediate practice domain, I often wonder if those are garbage as much as the ones in my field have become. That is not how it should be, IMO.
Thanks again for this rejoinder. I understand you better and I find nothing of substance in your latest about which I would quibble. And with regard to your last statement - we are of one heart and mind with regard to skepticism… If we were from or living in Missouri, we would be especially at home as it is known as the ‘show me state’. I live in Idaho, a place where skepticism is also commonly found.
Best wishes!
Nicely said, and on much we do agree - especially the need to be more clear and transparent regarding the strengths and weaknesses of guideline ratings.
Some quibbles: “As you know, and as the Pharma/industry who pay the people on these committees most certainly knows, once something becomes widely known as “class 1 recommendation”, few will go back and worry about the basis which undergirds it.”
During my practice as an Emergency Physician (and PhD pharmacologist), I would not infrequently diverge from guidelines based on the details of the clinical circumstances. So then, I was one of these few. And I was comfortable doing this as I was familiar with the ways in which guidelines were/are limited. But in that I believe those who do not so inform themselves are poorly serving their patients. There are, sadly, too many such…
I was - after I left clinical practice - a senior Medical Director for a small-to-midsized PPO/TPA in the Pacific Northwest. Within that context then…
- As a Medical Director reviewing cases that were not automatically approvable due to guideline non-compliance, I encountered many providers who thoughtfully disregarded a guideline for legitimate reasons. Perhaps you were one of these (I retired just over 4 years ago.) In many of these cases I understood the reasoning behind the divergence, and if I agreed with the reasoning, I gladly approved the coverage request. If I disagreed, of course, I would deny. (Of note - whenever I denied something, it was subject to appeal. About 75% of those appeals were upheld by disinterested independent reviewers chosen for their expertise relevant to the question.) My point here is that, in my experience (during my 11 years in this roll I reviewed over 20,000 cases) many providers disregard guidelines often for the same reasons you might.
- I do not know that ‘Pharmacy/industry’ pay the persons who serve on guideline committees. Having myself spoken professionally and personally with a number of such people, my impression was they were honest normal professionals just trying their best to do the right thing. There are outliers certainly and undue influence (both self interested and merely political) that discredits the efforts of the honest, but I do not share your (is it cynical) distrust of the system.
So we agree on much, and going forward we can hope for improvements.
I’ve never had an administrative role so approvals has never been my purview. It’s a lesser occurrence as I work in Canada, although work-related extended health insurance does come into play.
I think you misunderstood my point. I’m happy you and many many others disregard guidelines when the clinical scenario requires it. But in the paragraph you quoted, what I was trying to say is that, with (-in relatively short periods of) time, people no longer remember or care if it’s class 1 LOE A or class 1 LOE C; all class 1’s become equal. And imo they shouldn’t be. My apologies for not being clear.
On the point of “payment”, I’m being less precise than I should be, based on the forum. No, I do not intend to say that Pharma A cuts a cheque to Expert B to vote in committee for their product C. However, lots of “experts” are in fact on payroll directly as “consultants”, or receive research funding, sometimes specifically for working on product C. As Dr. JMM likes to say, maybe that’s “non-nefarious”. But it beggars belief to think that mere “disclosure” obviates bias. They have a vested interest in something, they’re more likely to vote for it. It’s human nature. It’s disclosed. It’s non-nefarious. And it should never be allowed to come to that. Besides, what makes a “subject matter expert” an expert in evidence appraisal? The people who generate the evidence should not be the ones who judge it. Quite literally, different expertise is required. As I’ve used the phrase before, I’d take a platoon of Dr. JMM’s “neutral Martian” EBM experts on a guideline committee over a bunch of subject matter experts any day of the week.
Am I a cynic? Probably. At the minimum, I am an extremely robust and vigorous skeptic.
Excellently written. Thank you so much. I couldn't agree more. After retiring from practice I studied and for the first time really understood the number to treat concept. When I understood the numbers for primary prevention of cardiovascular disease using a statin, (in general NNT= 100) realized that I had not been open or honest about the benefit of such therapy to my patients, because I did not understand NNT. Most patients assume effectiveness of a treatment will succeed closer to 100% of the time, not that there is a proven 99% chance the medication will not be of benefit to them. I am certain that would significantly change patient decision making. It is actually something most physicians do not understand. Promoting the NNT concept in clinical training and experience is needed much more than the attention it is given.
I want to insert "clapping hands emojis" !!! I look forward to sharing this with my nursing students.
Thank you
Thank you!
Trustworthy guidelines indeed should be based on systematic reviews of the evidence, and practitioners should be a minority of those on the panel, with conflicts of interest managed in a transparent way, which should be reported. There is a great overview in a 20 min talk by Ivan Flores, who has been leading the group to develop AGREE II, which evaluates guidelines: https://youtu.be/smRzaGjQU0Y
Not all guidelines are trustworthy.They aren't if they are not based on systematic reviews of the evidence and if the strength of the recommendations is not tied to the strength of the evidence (eg low quality/certainty evidence and strong recommendations is inappropriate except in some very specific situations). Consensus is not enough. See also the paper by Brignardello-Petersen et al. on how to use clinical practice guidelines: https://pubmed.ncbi.nlm.nih.gov/34665198/
The recommendations for treatment of gender dysphoria by most US medical associations do not meet these criteria. Many current guidelines and guidelines for pediatric gender dysphoria have been rigorously assessed in https://adc.bmj.com/content/109/Suppl_2/s65 . Most were found to be poor. Some more recent guidelines (Germany, for instance) are also also consensus based rather than based on systematic reviews of the evidence. Sweden and Finland do base their recommendations for pediatric gender dysphoria on systematic reviews of the evidence and did the best (this rigorous assessment was part of the Cass Review and thus did not evaluate the Cass Review final recommendations).
The recent ASPS position statement on surgery for pediatric gender dysphoria does rely upon systematic reviews (it is not a formal guideline): https://www.plasticsurgery.org/documents/health-policy/positions/2026-gender-surgery-children-adolescents.pdf .
It would be great if these guidelines and recommendations would include NNTs, for sure.
The distinctions pointed out in this essay are extremely important! Thank you.
EXCELLENT points, thank you. Dr. Lesser and other primary care doctors should be on the AHA (and other) guideline committees, to help iron these issues out.
Also, for a look behind the new AHA guideline math, NNTs, and how they were built, see here: https://researchtranslation.substack.com/p/checking-the-math-in-new-statin-guidelines
Wise points. I say the guildelines exist to make you think twice before you vary from them.
This is a great approach, and applied to guidelines generally. I find guidelines helpful, not least because it makes me think twice before I depart from, and if I do, I make it explicit in any documents why I did so. All or virtually all guidelines have - correctly - a kind of disclaimer, identifying that the treatment of an individual patient can transcend the guidelines (though they don’t put it like that). The issue can arise in clinical negligence cases, where apparently sleep walking through guideline boundaries plays poorly in court.
Sage overview of this document (Tome?). Especially vital: the importance of considering NNT as well as ARR in decision making. ie: take the population based paper and make it relevant to the person on your exam table.