Remind me why we need guidelines? Why can’t we just use the evidence?
The most useful guidelines are those that advise clinicians to do less.
Guidelines are most helpful when there are very little data to analyze and we have to resort to expert opinion. I don’t need a guideline on statins or breast cancer screening.
For those in the UK training to be a generalist/ family medicine physician/ primary care clinician or GP, I tried to summarise just how many guidelines you actually need to know.
Prof Bauchner's article is a great reminder of how complex this is and how we need to be able to:
1 know which guidelines exist
2 apply them in practice
3 memorise them for exams
4 remember to use critical appraisal and clinical judgement when a case doesn't neatly fit into the guideline.
you probably knew I was talking about one of your comeptitors, Kaiser Permanente. I was involved in administrative roles within KP (as the Chief Of Primary Care) and was fortunate to get on some national committees, which brought me to Seattle a few times. Wish you the best.
As a practicing EM doc between 1998 & 2010, guidelines from various sources were usually helpful. As the Medical Director for a small/midsized PPO/TPA between 2010 & 2022, guidelines were crucial but challenging as I spent a large part of my work life developing coverage policies that tried to navigate the various guidelines along with my own assessment of published data. Given my personality and predilections, this work was very satisfying if at times rather frustrating… And, for the most part, guidelines were helpful if only as a mater of orientation for the coverage issues in question.
As we mentioned in the article writing about COI is always challenging. There were several comments about COI. COIs come in all shape and sizes, can be direct and indirect, and can be financial and non-financial. Sadly, in the US, where healthcare now consumes 20-25% of the GDP and is once again increasing far ahead of inflation, suffocating city and state budgets, COIs are increasingly important. We appear to have "resolved" COIs by greater transparency - I am not sure this should be acceptable. I applaud Andys group that appears to minimize financial COI in pursuit of excellence.
I do believe that USPSTF is a good model for guidelines. The guideline itself is of reasonable length, with supporting documentation in a different publication. In addition, they do comment on guidelines from other groups. Although little is known about the reliability of their grading system they have a detailed manual and I believe new members are "trained" how to use it.
I share concerns about any recommendation that is not labeled opinion when there is only weak or little evidence. It seems entirely inconsistent with the concept of EBM.
The generalist vs specialist challenge appears to have struck a nerve. Specialist certainly have a wealth of knowledge about a particular subject. Perhaps before releasing any guideline specialty societies should engage with generalists and generalists society to read it first and comment. This is particular true in areas - such as cardiology (vs oncology) - where generalists are responsible for the vast majority of care.
Again, with respect and thanks to those who commented.
Your comments about salaried physicians do not reflect my reality in my 12,900,000 member nonprofit health care system in which the physicians are salaried BUT the payment model is prepaid capitation. In that model, (which if you ask me should be the basic model of care in the US), best evidence based guidelines are essential to optimize health outcomes in a world of limited resources. The "bonuses" to which you refer are small rewards for meeting annual metric goals, and exist to improve quality. There is no conflict of interest in that because meeting your evidence based targeted goals actually costs the organization more resources. So neither the individual or the institution profit in this setting, despite your implication. Lastly, the source of guidelines in our institution are based on established guidelines from specialty groups and USPHTF. We go out of our way to be influenced by companies promoting their guidelines. Guidelines exist to avoid ordering unnecessary, unproven, and costly test.
It seems to me you are clearly talking about a fee for service world and private practice settings because each of the points you make describe the conflicts inherent in those settings. Why don't you talk about the conflicts of interest in private practice? And clarify that you are taking about employed physicians practicing in a fee for service model. Don't forget about the 12.9 million of us who work to avoid just those conflicts of interest.
“There is no conflict of interest in that because meeting your evidence based targeted goals actually costs the organization more resources.” But aren’t these organizational resource costs compensated?
The revenue is received by capitated prepayment per member, not by utilization, so those resources costs are just part of your total budget. No extra resources are provided if you do more testing. Hope that makes that understandable. Thanks for your interest. Andy
Thanks. If the extra testing (or other resource use) results in better health outcomes, your downstream costs are likely to be reduced. This sort of upstream investment reduction of down stream costs is what drives prudential designs in those businesses in the healthcare industry. Do I have this right…?
Yes. In general. That assumes that the institution is fully budgeted to do the right thing. Although the question of return on investment for preventive services is a big question. First, most preventive services, as much as it would seem otherwise, are not cost effective (vaccines and colon cancer screening the exception). Second, the cost savings may be realized in a different health care system if the patient changes insurance. Third, having sick people live longer because of excellent care may ultimately cost more. All fun things to puzzle.
Perhaps a special journal is needed for publishing guidelines, with staff with sufficient expertise and resources to assess them?
Two resources (mentioned them before in this space):
- A ~20 min talk which discusses a lot of the requirements by Ivan Florez at https://www.youtube.com/watch?v=smRzaGjQU0Y head of the AGREE collaboration. He notes that both the conflicts of interest and how they are managed should be declared.
- "How to Interpret and Use a Clinical Practice Guideline or Recommendation: Users' Guides to the Medical Literature " (by Brignardello-Petersen, Carrasco-Labra and Guyatt, https://pubmed.ncbi.nlm.nih.gov/34665198/)
How not to write a guideline-don't do what WPATH did--for their "standards of care" for those with gender dysphoria, including:
1. commission systematic reviews & then interfere with their publication (most have not been published at this point, according to PROSPERO), even saying in writing (https://www.bmj.com/content/387/bmj.q2227) that "Our concerns, echoed by the social justice lawyers we spoke with, is that evidence based review reveals little or no evidence and puts us in an untenable position in terms of affecting policy or winning lawsuits."
2. Claim what strength of recommendations means and then not follow it:
"Strong recommendations (“we recommend”) are for those interventions/therapy/strategies where:
•the evidence is of high quality
•estimates of the effect of an intervention/therapy/strategy (i.e., there is a high degree of certainty effects will be achieved in practice)
•there are few downsides of therapy/intervention/strategy
•there is a high degree of acceptance among providers and patients or those for whom the recommendation applies."
Don't then use "we recommend" after noting there are very few studies especially with longer term follow-up for adolescents (they also incorrectly claimed that a systematic review is not possible -but one can always do a systematic review--also, in this case there were already several published systematic reviews finding low/very low certainty evidence). Connecting the dots--if a systematic review is claimed to be not possible due to limitations of the evidence, the evidence is not "high quality" and "we recommend" thus does not seem appropriate.
Seems we are expecting white cloud to come out of the chimney every time we choose an editor, president or chief investigator. Nothing wrong with questioning and expecting facts to be presented but the data should never be manipulated or the conclusions swayed by interests and greed. Last one out turn the lights off….
I think it was important to note the difference in care perspective btw generalists and specialists as being a driver of some differences (or perhaps different areas of focus) in their respective guidelines. To a certain extent, we are targeting different things (eg. sens vs spec as you’ve noted).
But for me, the biggest issue is conflict of interest. There is some of the overt financial kind, both direct and indirect, as you’ve noted. But there are myriad softer and less overt opportunities for COI, such as reputational gain, endearing oneself to funders for future research and projects, and garden variety butt-kissing by junior panelists towards their more senior members (in hopes of scoring points in the first 2 categories). There is also overarching secondary gain by the issuing societies themselves….walk through the front door of the venue at any big meeting, and look at all the ads from the “partners” who are directly funding the meeting…and that society’s bottom line. It is in the direct financial and existential interest of the guideline-issuing society to endorse the shiny new thing…and it shows.
For me, the most egregious aspect of latter day (cardiology) guidelines is the disconnect btw the quality of evidence and the strength of recommendation. It boggles the mind that a level 1 recommendation can be sustained by anything less than grade A evidence. Yet this now happens as a matter of routine. This is why I no longer practice to the guideline, and practice to the evidence instead.
I would much rather guidelines be written by critical appraisal experts who are trained at parsing the evidence, rather than “subject-matter” experts whose expertise includes knowing which side their bread is buttered on.
You might review James N Kirkpatrick's review in Archives of Internal Medicine in 2004, re: conflicts of interest among cardiologists on guidelines committees.
It was an astoundingly high percentage. Have things changed? RAK
In this essay, Dr. Bauchner identifies key issues that underpin the generation and review of clinical guidelines and that determine their reliability and generalizability.
Many of these issues deserve an entire essay for themselves. Chief among these are the issues of evidence grading and the effect of conflicts of interest.
Too many guidelines make recommendations based on low grade evidence, and it is unclear how seriously clinicians should take them. and why they are included at all. Guidelines based only on high grade evidence would be easier to assimilate into practice and be far less controversial.
The possible effects of conflicts of interest of guideline writers cannot be effectively mitigated by simple disclosure. While disclosing relationships with entities that may benefit financially from a guideline may raise red flags for the reader, such a red flag casts only vague doubt over the whole document, and fails to highlight which statements in the guideline should be appraised with particular rigor. Far better to include a detailed description of which guideline or section a particular author participated in drafting, and the financial effects the guideline’s implementation would have on the conflicted entity.
I suspect that forcing authors and companies to create such detailed disclosures would result in these authors recusing themselves from that particular portion of the guideline. The guideline and all of our practices would be better for it.
About LLM. Does anybody remember using Open Evidence in the beginning? You could ask a question and get a response from the existing literature, with all its limitations - but still a fair response. Then after a while guidelines became pushed to the forefront of the response when looking for a conduct, bypassing whatever raw data you were looking for... I fear for the independence of LLM analysing data.
The argument about having a specialist and generalist's view on guideline editorials is great, preferably independent, with possibly a nuance between ressource rich and ressource poor settings (rural - urban centres).
“Guidelines are important, but they are not the definitive word on how to care for patients – that requires a discussion between patient and clinician – and what is best for that person.”
Was this your position during the pandemic??!! I think not.
I'd like to add one more point or might have a bit of a different opinion. I'm just a German generalist in my own bubble. I was part of some German guidelines but I by far do not have your expertise.
But: our guidelinesare far to long and they show a certiny in very smal questions, which nearly never really exists. Some of our guidelines are over 500 Page, have over 100 recommendations and over 1000 literature quotes. NOBODY ever read all even less critcal appraisal. And then reality is again totaly different and "in worst case" we might even have patients with their own will and having besides their body a soul and a social surrounding.
Men - even doctors - are simple. If we want to have everything perfect the result might be far to complex for a far to complex individual.
Remind me why we need guidelines? Why can’t we just use the evidence?
The most useful guidelines are those that advise clinicians to do less.
Guidelines are most helpful when there are very little data to analyze and we have to resort to expert opinion. I don’t need a guideline on statins or breast cancer screening.
One thing not mentioned in the discussion is the problematic role of guidelines in malpractice lawsuits.
https://medicurious.substack.com/p/no-4-how-many-guidelines-or-god-lines?utm_source=share&utm_medium=android&r=p3tn3
For those in the UK training to be a generalist/ family medicine physician/ primary care clinician or GP, I tried to summarise just how many guidelines you actually need to know.
Prof Bauchner's article is a great reminder of how complex this is and how we need to be able to:
1 know which guidelines exist
2 apply them in practice
3 memorise them for exams
4 remember to use critical appraisal and clinical judgement when a case doesn't neatly fit into the guideline.
you probably knew I was talking about one of your comeptitors, Kaiser Permanente. I was involved in administrative roles within KP (as the Chief Of Primary Care) and was fortunate to get on some national committees, which brought me to Seattle a few times. Wish you the best.
As a practicing EM doc between 1998 & 2010, guidelines from various sources were usually helpful. As the Medical Director for a small/midsized PPO/TPA between 2010 & 2022, guidelines were crucial but challenging as I spent a large part of my work life developing coverage policies that tried to navigate the various guidelines along with my own assessment of published data. Given my personality and predilections, this work was very satisfying if at times rather frustrating… And, for the most part, guidelines were helpful if only as a mater of orientation for the coverage issues in question.
I want to thank everyone for their comments.
As we mentioned in the article writing about COI is always challenging. There were several comments about COI. COIs come in all shape and sizes, can be direct and indirect, and can be financial and non-financial. Sadly, in the US, where healthcare now consumes 20-25% of the GDP and is once again increasing far ahead of inflation, suffocating city and state budgets, COIs are increasingly important. We appear to have "resolved" COIs by greater transparency - I am not sure this should be acceptable. I applaud Andys group that appears to minimize financial COI in pursuit of excellence.
I do believe that USPSTF is a good model for guidelines. The guideline itself is of reasonable length, with supporting documentation in a different publication. In addition, they do comment on guidelines from other groups. Although little is known about the reliability of their grading system they have a detailed manual and I believe new members are "trained" how to use it.
I share concerns about any recommendation that is not labeled opinion when there is only weak or little evidence. It seems entirely inconsistent with the concept of EBM.
The generalist vs specialist challenge appears to have struck a nerve. Specialist certainly have a wealth of knowledge about a particular subject. Perhaps before releasing any guideline specialty societies should engage with generalists and generalists society to read it first and comment. This is particular true in areas - such as cardiology (vs oncology) - where generalists are responsible for the vast majority of care.
Again, with respect and thanks to those who commented.
HCB
Your comments about salaried physicians do not reflect my reality in my 12,900,000 member nonprofit health care system in which the physicians are salaried BUT the payment model is prepaid capitation. In that model, (which if you ask me should be the basic model of care in the US), best evidence based guidelines are essential to optimize health outcomes in a world of limited resources. The "bonuses" to which you refer are small rewards for meeting annual metric goals, and exist to improve quality. There is no conflict of interest in that because meeting your evidence based targeted goals actually costs the organization more resources. So neither the individual or the institution profit in this setting, despite your implication. Lastly, the source of guidelines in our institution are based on established guidelines from specialty groups and USPHTF. We go out of our way to be influenced by companies promoting their guidelines. Guidelines exist to avoid ordering unnecessary, unproven, and costly test.
It seems to me you are clearly talking about a fee for service world and private practice settings because each of the points you make describe the conflicts inherent in those settings. Why don't you talk about the conflicts of interest in private practice? And clarify that you are taking about employed physicians practicing in a fee for service model. Don't forget about the 12.9 million of us who work to avoid just those conflicts of interest.
Andy Golden, M.D.
“There is no conflict of interest in that because meeting your evidence based targeted goals actually costs the organization more resources.” But aren’t these organizational resource costs compensated?
The revenue is received by capitated prepayment per member, not by utilization, so those resources costs are just part of your total budget. No extra resources are provided if you do more testing. Hope that makes that understandable. Thanks for your interest. Andy
Thanks. If the extra testing (or other resource use) results in better health outcomes, your downstream costs are likely to be reduced. This sort of upstream investment reduction of down stream costs is what drives prudential designs in those businesses in the healthcare industry. Do I have this right…?
Yes. In general. That assumes that the institution is fully budgeted to do the right thing. Although the question of return on investment for preventive services is a big question. First, most preventive services, as much as it would seem otherwise, are not cost effective (vaccines and colon cancer screening the exception). Second, the cost savings may be realized in a different health care system if the patient changes insurance. Third, having sick people live longer because of excellent care may ultimately cost more. All fun things to puzzle.
Yes - This is why I loved my job as Medical Director for a PPO/TPA in the Seattle area. I like puzzles…
Perhaps a special journal is needed for publishing guidelines, with staff with sufficient expertise and resources to assess them?
Two resources (mentioned them before in this space):
- A ~20 min talk which discusses a lot of the requirements by Ivan Florez at https://www.youtube.com/watch?v=smRzaGjQU0Y head of the AGREE collaboration. He notes that both the conflicts of interest and how they are managed should be declared.
- "How to Interpret and Use a Clinical Practice Guideline or Recommendation: Users' Guides to the Medical Literature " (by Brignardello-Petersen, Carrasco-Labra and Guyatt, https://pubmed.ncbi.nlm.nih.gov/34665198/)
How not to write a guideline-don't do what WPATH did--for their "standards of care" for those with gender dysphoria, including:
1. commission systematic reviews & then interfere with their publication (most have not been published at this point, according to PROSPERO), even saying in writing (https://www.bmj.com/content/387/bmj.q2227) that "Our concerns, echoed by the social justice lawyers we spoke with, is that evidence based review reveals little or no evidence and puts us in an untenable position in terms of affecting policy or winning lawsuits."
2. Claim what strength of recommendations means and then not follow it:
"Strong recommendations (“we recommend”) are for those interventions/therapy/strategies where:
•the evidence is of high quality
•estimates of the effect of an intervention/therapy/strategy (i.e., there is a high degree of certainty effects will be achieved in practice)
•there are few downsides of therapy/intervention/strategy
•there is a high degree of acceptance among providers and patients or those for whom the recommendation applies."
Don't then use "we recommend" after noting there are very few studies especially with longer term follow-up for adolescents (they also incorrectly claimed that a systematic review is not possible -but one can always do a systematic review--also, in this case there were already several published systematic reviews finding low/very low certainty evidence). Connecting the dots--if a systematic review is claimed to be not possible due to limitations of the evidence, the evidence is not "high quality" and "we recommend" thus does not seem appropriate.
These guidelines have been reviewed alongside others in the field using AGREE II (https://adc.bmj.com/content/109/Suppl_2/s65) as well as on their own (https://link.springer.com/article/10.1007/s10508-025-03399-6 ) and have not done well. They are still relied upon by, e.g., the State of Oregon (https://www.resourcepro.com/bulletin/oregon-sets-wpath-8-as-minimum-standard-for-gender-affirming-treatment-in-health-plans/) and many hospitals and insurers, as far as I know.
I agree with the points raised by the comments below.
I advise:
Guidelines should be limited to 3 pages at 12 point font, double spaced.
Writers with any tie to Pharma should not be allowed to write guidelines.
A generalist (Family Medicine, Internist or Pediatrician as appropriate) should be on every guideline.
SAGE.
Seems we are expecting white cloud to come out of the chimney every time we choose an editor, president or chief investigator. Nothing wrong with questioning and expecting facts to be presented but the data should never be manipulated or the conclusions swayed by interests and greed. Last one out turn the lights off….
I think it was important to note the difference in care perspective btw generalists and specialists as being a driver of some differences (or perhaps different areas of focus) in their respective guidelines. To a certain extent, we are targeting different things (eg. sens vs spec as you’ve noted).
But for me, the biggest issue is conflict of interest. There is some of the overt financial kind, both direct and indirect, as you’ve noted. But there are myriad softer and less overt opportunities for COI, such as reputational gain, endearing oneself to funders for future research and projects, and garden variety butt-kissing by junior panelists towards their more senior members (in hopes of scoring points in the first 2 categories). There is also overarching secondary gain by the issuing societies themselves….walk through the front door of the venue at any big meeting, and look at all the ads from the “partners” who are directly funding the meeting…and that society’s bottom line. It is in the direct financial and existential interest of the guideline-issuing society to endorse the shiny new thing…and it shows.
For me, the most egregious aspect of latter day (cardiology) guidelines is the disconnect btw the quality of evidence and the strength of recommendation. It boggles the mind that a level 1 recommendation can be sustained by anything less than grade A evidence. Yet this now happens as a matter of routine. This is why I no longer practice to the guideline, and practice to the evidence instead.
I would much rather guidelines be written by critical appraisal experts who are trained at parsing the evidence, rather than “subject-matter” experts whose expertise includes knowing which side their bread is buttered on.
You might review James N Kirkpatrick's review in Archives of Internal Medicine in 2004, re: conflicts of interest among cardiologists on guidelines committees.
It was an astoundingly high percentage. Have things changed? RAK
I gather the issue is also how they are managed and the guidelines have to note that. Just listing them isn't enough....
In this essay, Dr. Bauchner identifies key issues that underpin the generation and review of clinical guidelines and that determine their reliability and generalizability.
Many of these issues deserve an entire essay for themselves. Chief among these are the issues of evidence grading and the effect of conflicts of interest.
Too many guidelines make recommendations based on low grade evidence, and it is unclear how seriously clinicians should take them. and why they are included at all. Guidelines based only on high grade evidence would be easier to assimilate into practice and be far less controversial.
The possible effects of conflicts of interest of guideline writers cannot be effectively mitigated by simple disclosure. While disclosing relationships with entities that may benefit financially from a guideline may raise red flags for the reader, such a red flag casts only vague doubt over the whole document, and fails to highlight which statements in the guideline should be appraised with particular rigor. Far better to include a detailed description of which guideline or section a particular author participated in drafting, and the financial effects the guideline’s implementation would have on the conflicted entity.
I suspect that forcing authors and companies to create such detailed disclosures would result in these authors recusing themselves from that particular portion of the guideline. The guideline and all of our practices would be better for it.
About LLM. Does anybody remember using Open Evidence in the beginning? You could ask a question and get a response from the existing literature, with all its limitations - but still a fair response. Then after a while guidelines became pushed to the forefront of the response when looking for a conduct, bypassing whatever raw data you were looking for... I fear for the independence of LLM analysing data.
The argument about having a specialist and generalist's view on guideline editorials is great, preferably independent, with possibly a nuance between ressource rich and ressource poor settings (rural - urban centres).
“Guidelines are important, but they are not the definitive word on how to care for patients – that requires a discussion between patient and clinician – and what is best for that person.”
Was this your position during the pandemic??!! I think not.
Dear Howard Bauchner,
I'd like to add one more point or might have a bit of a different opinion. I'm just a German generalist in my own bubble. I was part of some German guidelines but I by far do not have your expertise.
But: our guidelinesare far to long and they show a certiny in very smal questions, which nearly never really exists. Some of our guidelines are over 500 Page, have over 100 recommendations and over 1000 literature quotes. NOBODY ever read all even less critcal appraisal. And then reality is again totaly different and "in worst case" we might even have patients with their own will and having besides their body a soul and a social surrounding.
Men - even doctors - are simple. If we want to have everything perfect the result might be far to complex for a far to complex individual.
Recent AHA/ACC etc guidelines on lipid management clearly fall into the “too much” category. KISS.