Empagliflozin and Semaglutide and Oseltamivir, oh my! Three studies that were well done, took time and money and expertise, were published in excellent journals and did not move medicine forward.
About your response to the semaglutide study--you seem to assume that obesity is caused by poor lifestyle choices and that if children or teens are taught more about nutrition, they wouldn't need the drug to treat their obesity, so therefore this study is some sort of pharma overreach.
However, there is *no* nutritional strategy--i.e. diet--that is proven to lead to long-term, meaningful weight loss for more than a tiny percentage of people: most regain the weight within two years, and very few dieters ever manage to achieve the amount of weight loss that these drugs enable. These are well known facts with decades of research behind them.
Teaching people of any age how to "eat better" does not enable them to eat less. You can get people to make better nutritional choices, but that will not substantially affect their weight. You can get them to start exercising and other health-promoting activities, and it will not substantially affect their weight. The new weight loss drugs, however, do enable significant weight loss for the vast majority of people who take them.
Yes, people whose obesity is effectively treated by those drugs might need to stay on them forever--and one can make the same complaint about diabetics continuing to need to take insulin. Obesity is a chronic disease that needs chronic management, we need recognize it as such and treat it as such. Even in adolescents.
Great article, once again. Shouldn't the NIH be this "review board" that determines what studies should be done to answer the most important questions?
The SGLT2 inhibitors are remarkable drugs for getting patients’ blood sugars down, but the raging UTIs and renal stones are a problem for some people. I have friends who benefitted from empaglaflozin, but had to get off it after a Candida glabrata infection ravaged her urogenital tract. It’s about trade offs and for this friend, the constant kidney stones and mycotic UTIs were too much to bear.
You’re absolutely correct that critical appraisal of clinical trials is still necessary and useful. Critical appraisal allows you to see past a lot of the common tricks employed by researchers and pharmaceutical companies (such as end point switching, early termination of trials, and more) which are often employed to generate false positives.
The bigger issue is that the far more common way of generating false positive results is sponsoring multiple properly run trials and only publishing the ones which favor your intervention. This allows sponsors to show a really rigorous clinical trial while hiding the fact that this data was only produced after multiple attempts.
Once this occurs, even diligent doctors can be deceived into accepting an intervention even if in reality it is deleterious. At this point, the hammer of “evidence based medicine” comes down on any dissenting physician who notices their patients are not doing as well as the data suggests they should, and concerns can be brushed aside as just bad luck.
Unfortunately, I don’t think that critical appraisal of individual trials can fix the mess of conflicts of interests and compromised publishers in medicine. I highly encourage everyone interested in this topic to read “Bad Science” by Ben Goldacre.
Thank you for sharing. As the co-chair of upcoming Canadian pediatric obesity management guidelines, I agree on the need for longer term trials of GLP1s, trials that also comprehensively measure health-related quality of life domains (eg, anxiety, depression, eating disorders). Trials should also consider adaptive designs and have multiple arms to test various behavioral interventions and weight regain among those who discontinue GLP1s for various reasons (which may be more than 50% after just 1 years based on observational data in adults).
As an instructor of critical appraisal at the university, strong suggestion for anyone wanting to sharpen their systematic thinking skills to use JAMA Users’ Guides to the Medical Literature. Similar guides exist for nursing, nutrition, etc.
I can see a marketer from a company that makes semaglutide FREAKING out on your analysis! “He’s suggesting we are attempting to have PERMANENT PATIENTS!” The only thing missing is a picture of a fat Joe Camel luring the children! On a less cynical note, how many young docs apply the kind of eye you apply, Dr. Cifu?
As much as I hate to say it, I think many young folks have gotten conditioned to mental detachment. The times didn’t help that. Like you, I prefer experience but then I am pretty sure we all do.
My office partner and I often joke about SGLT as “expensive diuretics” in the HFpEF population, for exactly the reasons you mentioned. And as Dr. JMM has often noted particularly in HFpEF (who tend to be older with more comorbidities), HFH is nice but needs to be considered in the context of total hospitalizations.
I would add, in the context of critical appraisal, that beyond rigorous methods and proper statistics, we should remember clinical significance (and not just statistical significance), effect size (ie ARR and NNT), and cost effectiveness.
Dr. Javed Butler et al. published a post-hoc analysis of EMPEROR-Preserved in JAMA Cardiology (2023) regarding baseline diuretic use. To quote the authors, “…diuretic status was not associated with changes in improvements in first HHF, total HHF, rate of decline in eGFR, and KCCQ-CSS with empagliflozin”.
This is a key point, but Pharma supported research often distorts the message that consumers get from studies. Empagliflozin was developed to treat type 2 diabetes but it reduces chronic kidney disease progression and heart failure hospitalizations by about a third whether the patient is diabetic or not! That is extremely important new information. Empagliflozin produces these great benefits, not because it lowers the sugar but because if increases activation of a master genetic metabolic survival switch that protects every organ and cell in the body.
What you don't get from pharma research is the balance that leads to better clinical and financial outcomes and this is a great example. Big Pharma will not tell you that metformin directly activates that same switch, or that lisinopril, losartan, statins, and spironolactone indirectly activate it. They also don't tell you that when you combine these drugs in a protocol, instead of a small reduction in cardiovascular events you get a four-fold reduction in heart attack, a five fold reduction in stroke and a six fold reduction in progression to dialysis. The protocol reduces hospitalizations for heart failure by 70%. The protocol does not reduce heart failure admissions by a third, it reduces them six fold. That make the inexpensive protocol 18 times as effective as that single drug.
Why do we always need more drugs? Because most of the previous drugs are failures. If you want to move medicine forward, get rid of big pharma as the primary driver.
Excellent journals? A/i will replace everyone of them...with who knows what?
Agree: Many principles can be learned without being a genius. I guess the challenge with appraisal is that people are either content or methodology “experts” and to some extent you need both. As someone trained in methodology and working in health technology assessment for 15 years, dissecting the clinical aspects is generally the bigger challenge for me, while the clinicians often lack training in the methods. Unfortunately the discourses between these groups are not always constructive, but sometimes too much influenced by power struggles, vested interests, preconceptions, different perspectives (say population vs individual) or not distinguishing between data and what they might mean for practice. Science and its application is tricky and there should always be humility. I think of appraisal as a skill that is best honed lifelong. That’s why I enjoy following Sensible Medicine.
I envision a giant online ecosystem that's like a mix of facebook, substack, and peer-reveiwed journals. The journal submissions have a required proposal/prereg step that is peer-reviewed, which includes by the crowd. There is also a place to list your peer-reviewed proposal and seek crowd funding which could come from two sources. Firsly, the crowd pledges their own money towards a sought goal. Secondly, NIH, philanthropists, or whomever grant a huge pot to the overall org, and this is used to match people's pledges. Thus research that matters to people and is less likely to be poorly designed gets funded. I describe this in more detailin point #3 below:
I think this is a really interesting article and thank you. I agree with the principle. As a heart failure physician I am going to comment on your points around the Emperor preserved study. I was all ready to refute you by saying that most of the patients in both groups were already on a loop diuretic, but somewhat amazingly the investigators do not report this data anywhere in the paper or supplementary appendix (unless I have missed it).
However the previous DELIVER trial of dapagliflozin in the same patient population did report this (UK authors win here!) ; 75% of both groups were on a loop diuretic. So it does appear that the benefit of dapagliflozin was in addition to the loop. https://www.nejm.org/doi/full/10.1056/NEJMoa2206286
I suspect there is a data torturing paper showing the benefit of SGLT2 inhibitors was the same with and without loop diuretics in the preserved ejection fraction population, but I have not found it yet. There is one showing that the presence or absence of diuretics did not seem to impact the efficiency of dapagliflozin in reduced ejection fraction population https://www.ahajournals.org/doi/10.1161/CIRCULATIONAHA.120.047077
So I think here it looks like SGLT2 probably do work additionally to the loop diuretics. I agree however that particularly in the preserved EF group a trial of higher dose of loop versus SGLT2 would have been interesting - if less likely to get funded.
Finally, as someone who is handing out a lot of these medications (cost is much lower here in the UK, and funded by NHS), the clinical effect appears to be pretty clear and my patients do seem to feel better even when they have already been on a loop diuretic.
Best wishes,
James Gamble, cardiologist with an interest in heart failure
About your response to the semaglutide study--you seem to assume that obesity is caused by poor lifestyle choices and that if children or teens are taught more about nutrition, they wouldn't need the drug to treat their obesity, so therefore this study is some sort of pharma overreach.
However, there is *no* nutritional strategy--i.e. diet--that is proven to lead to long-term, meaningful weight loss for more than a tiny percentage of people: most regain the weight within two years, and very few dieters ever manage to achieve the amount of weight loss that these drugs enable. These are well known facts with decades of research behind them.
Teaching people of any age how to "eat better" does not enable them to eat less. You can get people to make better nutritional choices, but that will not substantially affect their weight. You can get them to start exercising and other health-promoting activities, and it will not substantially affect their weight. The new weight loss drugs, however, do enable significant weight loss for the vast majority of people who take them.
Yes, people whose obesity is effectively treated by those drugs might need to stay on them forever--and one can make the same complaint about diabetics continuing to need to take insulin. Obesity is a chronic disease that needs chronic management, we need recognize it as such and treat it as such. Even in adolescents.
Do we know if the weight loss in this study is mainly due to fat loss or muscle mass loss?
Great article, once again. Shouldn't the NIH be this "review board" that determines what studies should be done to answer the most important questions?
The SGLT2 inhibitors are remarkable drugs for getting patients’ blood sugars down, but the raging UTIs and renal stones are a problem for some people. I have friends who benefitted from empaglaflozin, but had to get off it after a Candida glabrata infection ravaged her urogenital tract. It’s about trade offs and for this friend, the constant kidney stones and mycotic UTIs were too much to bear.
You’re absolutely correct that critical appraisal of clinical trials is still necessary and useful. Critical appraisal allows you to see past a lot of the common tricks employed by researchers and pharmaceutical companies (such as end point switching, early termination of trials, and more) which are often employed to generate false positives.
The bigger issue is that the far more common way of generating false positive results is sponsoring multiple properly run trials and only publishing the ones which favor your intervention. This allows sponsors to show a really rigorous clinical trial while hiding the fact that this data was only produced after multiple attempts.
Once this occurs, even diligent doctors can be deceived into accepting an intervention even if in reality it is deleterious. At this point, the hammer of “evidence based medicine” comes down on any dissenting physician who notices their patients are not doing as well as the data suggests they should, and concerns can be brushed aside as just bad luck.
Unfortunately, I don’t think that critical appraisal of individual trials can fix the mess of conflicts of interests and compromised publishers in medicine. I highly encourage everyone interested in this topic to read “Bad Science” by Ben Goldacre.
I read Bad Pharma by Goldacre before med school. While interesting, the severity of the problem of missing data went over my head at the time.
I appreciate this reminder of his good work. Worth revisiting 🙏
Bad Pharma is exactly the book that really made me reanalyze my life. Extremely thorough and eye opening!
Thank you for sharing. As the co-chair of upcoming Canadian pediatric obesity management guidelines, I agree on the need for longer term trials of GLP1s, trials that also comprehensively measure health-related quality of life domains (eg, anxiety, depression, eating disorders). Trials should also consider adaptive designs and have multiple arms to test various behavioral interventions and weight regain among those who discontinue GLP1s for various reasons (which may be more than 50% after just 1 years based on observational data in adults).
As an instructor of critical appraisal at the university, strong suggestion for anyone wanting to sharpen their systematic thinking skills to use JAMA Users’ Guides to the Medical Literature. Similar guides exist for nursing, nutrition, etc.
Bradley C Johnston, PhD
I can see a marketer from a company that makes semaglutide FREAKING out on your analysis! “He’s suggesting we are attempting to have PERMANENT PATIENTS!” The only thing missing is a picture of a fat Joe Camel luring the children! On a less cynical note, how many young docs apply the kind of eye you apply, Dr. Cifu?
As much as I hate to say it, I think many young folks have gotten conditioned to mental detachment. The times didn’t help that. Like you, I prefer experience but then I am pretty sure we all do.
Excellent piece.
My office partner and I often joke about SGLT as “expensive diuretics” in the HFpEF population, for exactly the reasons you mentioned. And as Dr. JMM has often noted particularly in HFpEF (who tend to be older with more comorbidities), HFH is nice but needs to be considered in the context of total hospitalizations.
I would add, in the context of critical appraisal, that beyond rigorous methods and proper statistics, we should remember clinical significance (and not just statistical significance), effect size (ie ARR and NNT), and cost effectiveness.
Dr. Javed Butler et al. published a post-hoc analysis of EMPEROR-Preserved in JAMA Cardiology (2023) regarding baseline diuretic use. To quote the authors, “…diuretic status was not associated with changes in improvements in first HHF, total HHF, rate of decline in eGFR, and KCCQ-CSS with empagliflozin”.
Emily Rosa’s study on Therapeutic Touch published in JAMA in 1998 cost $10. She is still waiting for it to be replicated.
The replication crisis.
This is a key point, but Pharma supported research often distorts the message that consumers get from studies. Empagliflozin was developed to treat type 2 diabetes but it reduces chronic kidney disease progression and heart failure hospitalizations by about a third whether the patient is diabetic or not! That is extremely important new information. Empagliflozin produces these great benefits, not because it lowers the sugar but because if increases activation of a master genetic metabolic survival switch that protects every organ and cell in the body.
What you don't get from pharma research is the balance that leads to better clinical and financial outcomes and this is a great example. Big Pharma will not tell you that metformin directly activates that same switch, or that lisinopril, losartan, statins, and spironolactone indirectly activate it. They also don't tell you that when you combine these drugs in a protocol, instead of a small reduction in cardiovascular events you get a four-fold reduction in heart attack, a five fold reduction in stroke and a six fold reduction in progression to dialysis. The protocol reduces hospitalizations for heart failure by 70%. The protocol does not reduce heart failure admissions by a third, it reduces them six fold. That make the inexpensive protocol 18 times as effective as that single drug.
https://pubmed.ncbi.nlm.nih.gov/32079684/
Why do we always need more drugs? Because most of the previous drugs are failures. If you want to move medicine forward, get rid of big pharma as the primary driver.
Excellent journals? A/i will replace everyone of them...with who knows what?
Agree: Many principles can be learned without being a genius. I guess the challenge with appraisal is that people are either content or methodology “experts” and to some extent you need both. As someone trained in methodology and working in health technology assessment for 15 years, dissecting the clinical aspects is generally the bigger challenge for me, while the clinicians often lack training in the methods. Unfortunately the discourses between these groups are not always constructive, but sometimes too much influenced by power struggles, vested interests, preconceptions, different perspectives (say population vs individual) or not distinguishing between data and what they might mean for practice. Science and its application is tricky and there should always be humility. I think of appraisal as a skill that is best honed lifelong. That’s why I enjoy following Sensible Medicine.
Really good point. Thanks.
Adam
We have even worse studies about SGLT2 blockers. This one tested it in cardiovascular respiratory or renal failure regardless the etiology.
https://thethoughtfulintensivist.substack.com/p/irrelevant-yet-significant-the-defender?r=20qrtz
Thank you. What strikes me most in this study is the absence of a sound research hypothesis. They just gave it to the patients to see what happens.
I envision a giant online ecosystem that's like a mix of facebook, substack, and peer-reveiwed journals. The journal submissions have a required proposal/prereg step that is peer-reviewed, which includes by the crowd. There is also a place to list your peer-reviewed proposal and seek crowd funding which could come from two sources. Firsly, the crowd pledges their own money towards a sought goal. Secondly, NIH, philanthropists, or whomever grant a huge pot to the overall org, and this is used to match people's pledges. Thus research that matters to people and is less likely to be poorly designed gets funded. I describe this in more detailin point #3 below:
https://substack.com/home/post/p-150468655
Open-source is the answer!
Dear Adam,
I think this is a really interesting article and thank you. I agree with the principle. As a heart failure physician I am going to comment on your points around the Emperor preserved study. I was all ready to refute you by saying that most of the patients in both groups were already on a loop diuretic, but somewhat amazingly the investigators do not report this data anywhere in the paper or supplementary appendix (unless I have missed it).
However the previous DELIVER trial of dapagliflozin in the same patient population did report this (UK authors win here!) ; 75% of both groups were on a loop diuretic. So it does appear that the benefit of dapagliflozin was in addition to the loop. https://www.nejm.org/doi/full/10.1056/NEJMoa2206286
I suspect there is a data torturing paper showing the benefit of SGLT2 inhibitors was the same with and without loop diuretics in the preserved ejection fraction population, but I have not found it yet. There is one showing that the presence or absence of diuretics did not seem to impact the efficiency of dapagliflozin in reduced ejection fraction population https://www.ahajournals.org/doi/10.1161/CIRCULATIONAHA.120.047077
So I think here it looks like SGLT2 probably do work additionally to the loop diuretics. I agree however that particularly in the preserved EF group a trial of higher dose of loop versus SGLT2 would have been interesting - if less likely to get funded.
Finally, as someone who is handing out a lot of these medications (cost is much lower here in the UK, and funded by NHS), the clinical effect appears to be pretty clear and my patients do seem to feel better even when they have already been on a loop diuretic.
Best wishes,
James Gamble, cardiologist with an interest in heart failure
Oxford
Thanks for the thoughtful analysis.
Adam