I have a severely- profoundly sibling with advanced liver disease and cachexia. Off the topic, A TTPPS procedure is recommended for quality of life. Ascites is significant and she is getting a paracentesis every 3 weeks. Cardiology and hepatic providers recommend and are on board with this. Any thoughts from medical professionals? I am struggling. I am a RN and have been a nurse for 9 years. Thank you
Excellent analysis. As someone who struggled with statistics, this author clarified some basic ideas needed to better understand studies like this. Intuition and expectations will always be present and cause us to have a bias and we must accept that. That's why Ioannides study of RC studies was so difficult to accept for so many. Worship of the RC study or the p value causes us to sometimes lose sight of common sense. Is a disabling stroke the same as a site infection complication? No. Then why is it treated the same statistically. If indeed, it decreases stroke by 10%, I want it. But in a world where US citizens do not directly pay for their healthcare, small yet very expensive medical advances will continue us on the path where money is diverted to procedures that benefit few at the expense of preventive and vital services for many. We still think medical care is only valuable when we as individuals need it. We have some difficult financial and ethical issues headed our way.
Presumably there is some stroke risk associated with maneuvering the CEP into the carotids? Ie. Knocking off carotid plaque into the cerebral circulation must be a non-negative risk in some people. So I wonder if the two risks cancel out? Could you select people for CEP on the basis of clean carotids?
Thanks to Hoarders of Magnitude for responding to my questions. If reduction in stroke risk is 0.2%, and the procedure costs $3k, we would be spending $3,000/0.002 = $1.5M to revent one stroke, for an old, often sick peson. As a society, we can't afford that. IMHO
Maybe a "Nested Case-control study" could further guide which particular cases or variables predict a higher likelihood of benefit in certain patients?
To my eyes, there are a number of things missing from your analysis. I'd be happy to write a reply saying this in more words.
1. The absolute difference in risk is small, 0.2%.
2. What is the incremental cost?
3. What is the average age of the TAVR recipients?
4. What is their expected life expectancy.
5. Details on the instrumental variable analysis you referred to, and the confidence interval from that study. If a randomized trial would need a sample of 115,000, I do not understand how the IV analysis was powered, with only 40k patients. That study almost *must* be underpowered too.
The cost is $3k of which the majority goes to the device itself. (see my other comment for laughable surgeon reimbursement which John alluded to). This is way less than the TAVR surgery itself so cost is not the driving factor on lack of uptake. It seems like docs don't think it works? Critical appraisal working as it should?
Patients are on average 82yo. Normally this would mean you have circa 7 to 10 years left, but I dont know how much a messed up aorta effects life expectancy.
Using a 2.2% stroke rate, with a 10% actual reduction of stroke as per the observational study you cited and extrapolating the serious adverse event rates published in that chart which appear to be about double overall in the study group compared to control:
In 100,000 patients I get a reduction of 220 strokes (2200 to 1980) in exchange for 230 more serious adverse events in the study group (579 vs 342). It would be nice to know what was involved in all the falling people appear to be doing in the intervention group, because there seems to be quite a bit of it. Just going by procedural complications (4/24 vs 1/13), I get 105 direct procedural complications per 100,000 versus 26 in the control group, for an excess of 79 versus 220 strokes prevented.
There's also about double the rate of non-serious adverse events listed in that chart as well and they are about as frequent in the study group as the serious events were, but less frequent in the control group, so there is a greater spread of non-serious adverse events as well, which is probably in the realm of another 200 or so excess adverse events on top of the serious ones.
Seems pretty marginal, depending on how serious these adverse events really are judged, but stroke is often a life altering complication so it would really depend on how severe these adverse events were if it were me making the decision.
The bias that CEP (should) prevent CVAs did not really pan out.
Either stick with the bias based on insufficient power, or wait till you have a better idea of the subsets who may be benefitted by CEP for another trial down the road.
I wonder if bootstrapping would be useful in this context- large observational study. That is to say if the effect was real, albeit not large, I would expect that the variance in the estimates for the 'true effect' would be mostly tightly grouped. Has anyone done research on this sort of thing ?
Always appreciate your appraisals, and I always learn something. “Cost effectiveness” of reading your work is through the roof.
For this, I would look at the ARR. The benefit, even if real, is tiny. The fact that it would require a trial enrolling in 6 figures to show a difference is testament to that. So I suspect any downstream ICER evaluation of the procedure would quickly be underwater.
I would also revisit the principle you have invoked in other areas. Drug classes like SGLT2 inhibitors and GLP 1RAs “work” in certain patient populations, even though we don’t know precisely why mechanistically. But that should not and does not stop us from using them in those appropriate populations. CEP doesn’t seem to work….for reasons we may not fully understand…but that shouldn’t obscure the main story.
As for the “why”, I’d wonder if there was any difference in “immediate” stroke. I would think if there was debris flying north now, it would cause a stroke now….and not in 3 days. So I would be curious if CEP shows a difference in a much shorter periprocedural window.
I’d also wonder if pre-procedural carotid assessments were done. Perhaps manipulation within diseased carotids themselves caused carotid embolic stroke that offsets any benefit from preventing aortic valve embolic stroke.
Our clinical diagnosis of stroke may be insufficiently sensitive to detect a difference. Brain imaging studies do show a reduction of new lesions with CEP
I don't see the problem. Miniscule differences in an event or complication are meaningless to those with common sense. With small numbers of endpoints tiny differences can be made to look significant by quoting relative risk differences. But this is usually promotional propaganda designed to fool people and suggests bias on the part of the reporter. The wide acceptance and prescription of statin drugs is evidence of the power of this sort of misleading presentation.
1. Using the relative 10% reduction in the endpoint from 1.4% to 1.2% to describe the findings is a terrible distortion of the magnitude of the difference and should not be done.
2. Seriously how much have you improved a patient’s lot by reducing his stroke risk this trivially? Is this device more or less effective than the brand of surgical soap in the cath lab, or how well the operator slept last night? Whatever incremental improvement this might represent isn’t enough to matter to a patient or his doctors today.
3. We need new ethics to guide us in the selection of new therapies. When financial cost is not considered there are no brakes on the train, and next stop is a bankrupt country.
"Doctors don’t get paid more for using the device, which involves making an extra arterial puncture and taking some time manipulating the device into the carotids."
CPT 33370: This code is specifically for reporting the transcatheter placement and removal of Sentinel embolic protection devices during procedures like TAVR.
Reimbursement Calculation:
The CMS (Centers for Medicare & Medicaid Services) adopted the recommendation for valuing code +33370 at 2.5 work RVUs. The total facility RVUs are 3.93, and the current conversion factor is $32.35. The base payment rate is calculated by multiplying the total RVUs (2.5 + 3.93 = 6.43) by the conversion factor ($32.35), resulting in $127.
1. It's an HR of .99; there is no reason to do this!
2. Would you recommend the addition of CEP for your spouse or parents or closest friends?
3. That you correctly wonder why the risk of stroke with TAVI is NOT [edit] a lot higher shows that you and your field do not actually understand the underlying mechanisms of stroke and some kind of inherent stroke prevention mechanisms in humans. Better understand the underlying mechanisms.
4. If you are looking to get rich, buying 10 lottery tickets is just as dumb as buying 1 lottery ticket.
The fundamental problem is considering a relative change as clinically meaningful. Clinical decisions should be based on absolute differences. It makes no sense to state that a ~30% relative change is clinically important.
I have a severely- profoundly sibling with advanced liver disease and cachexia. Off the topic, A TTPPS procedure is recommended for quality of life. Ascites is significant and she is getting a paracentesis every 3 weeks. Cardiology and hepatic providers recommend and are on board with this. Any thoughts from medical professionals? I am struggling. I am a RN and have been a nurse for 9 years. Thank you
Excellent analysis. As someone who struggled with statistics, this author clarified some basic ideas needed to better understand studies like this. Intuition and expectations will always be present and cause us to have a bias and we must accept that. That's why Ioannides study of RC studies was so difficult to accept for so many. Worship of the RC study or the p value causes us to sometimes lose sight of common sense. Is a disabling stroke the same as a site infection complication? No. Then why is it treated the same statistically. If indeed, it decreases stroke by 10%, I want it. But in a world where US citizens do not directly pay for their healthcare, small yet very expensive medical advances will continue us on the path where money is diverted to procedures that benefit few at the expense of preventive and vital services for many. We still think medical care is only valuable when we as individuals need it. We have some difficult financial and ethical issues headed our way.
Presumably there is some stroke risk associated with maneuvering the CEP into the carotids? Ie. Knocking off carotid plaque into the cerebral circulation must be a non-negative risk in some people. So I wonder if the two risks cancel out? Could you select people for CEP on the basis of clean carotids?
Thanks to Hoarders of Magnitude for responding to my questions. If reduction in stroke risk is 0.2%, and the procedure costs $3k, we would be spending $3,000/0.002 = $1.5M to revent one stroke, for an old, often sick peson. As a society, we can't afford that. IMHO
Bernie Black (Northwestern University)
Maybe a "Nested Case-control study" could further guide which particular cases or variables predict a higher likelihood of benefit in certain patients?
To my eyes, there are a number of things missing from your analysis. I'd be happy to write a reply saying this in more words.
1. The absolute difference in risk is small, 0.2%.
2. What is the incremental cost?
3. What is the average age of the TAVR recipients?
4. What is their expected life expectancy.
5. Details on the instrumental variable analysis you referred to, and the confidence interval from that study. If a randomized trial would need a sample of 115,000, I do not understand how the IV analysis was powered, with only 40k patients. That study almost *must* be underpowered too.
Bernie Black
Northwestern University
The cost is $3k of which the majority goes to the device itself. (see my other comment for laughable surgeon reimbursement which John alluded to). This is way less than the TAVR surgery itself so cost is not the driving factor on lack of uptake. It seems like docs don't think it works? Critical appraisal working as it should?
Patients are on average 82yo. Normally this would mean you have circa 7 to 10 years left, but I dont know how much a messed up aorta effects life expectancy.
What's the quality of life of those years left? Eg how many are spent with dementia, to start with? Frailty?
Some additional food for thought in answering this question would be to delve into this chart of adverse reactions here:
https://www.nejm.org/doi/suppl/10.1056/NEJMoa2415120/suppl_file/nejmoa2415120_appendix.pdf - Page 41
(mirror: https://drive.google.com/file/d/1-3FNVx2liThRtZm7ZFnxrra5_9yJHMaV/view?usp=sharing)
Using a 2.2% stroke rate, with a 10% actual reduction of stroke as per the observational study you cited and extrapolating the serious adverse event rates published in that chart which appear to be about double overall in the study group compared to control:
In 100,000 patients I get a reduction of 220 strokes (2200 to 1980) in exchange for 230 more serious adverse events in the study group (579 vs 342). It would be nice to know what was involved in all the falling people appear to be doing in the intervention group, because there seems to be quite a bit of it. Just going by procedural complications (4/24 vs 1/13), I get 105 direct procedural complications per 100,000 versus 26 in the control group, for an excess of 79 versus 220 strokes prevented.
There's also about double the rate of non-serious adverse events listed in that chart as well and they are about as frequent in the study group as the serious events were, but less frequent in the control group, so there is a greater spread of non-serious adverse events as well, which is probably in the realm of another 200 or so excess adverse events on top of the serious ones.
Seems pretty marginal, depending on how serious these adverse events really are judged, but stroke is often a life altering complication so it would really depend on how severe these adverse events were if it were me making the decision.
The bias that CEP (should) prevent CVAs did not really pan out.
Either stick with the bias based on insufficient power, or wait till you have a better idea of the subsets who may be benefitted by CEP for another trial down the road.
I wonder if bootstrapping would be useful in this context- large observational study. That is to say if the effect was real, albeit not large, I would expect that the variance in the estimates for the 'true effect' would be mostly tightly grouped. Has anyone done research on this sort of thing ?
Always appreciate your appraisals, and I always learn something. “Cost effectiveness” of reading your work is through the roof.
For this, I would look at the ARR. The benefit, even if real, is tiny. The fact that it would require a trial enrolling in 6 figures to show a difference is testament to that. So I suspect any downstream ICER evaluation of the procedure would quickly be underwater.
I would also revisit the principle you have invoked in other areas. Drug classes like SGLT2 inhibitors and GLP 1RAs “work” in certain patient populations, even though we don’t know precisely why mechanistically. But that should not and does not stop us from using them in those appropriate populations. CEP doesn’t seem to work….for reasons we may not fully understand…but that shouldn’t obscure the main story.
As for the “why”, I’d wonder if there was any difference in “immediate” stroke. I would think if there was debris flying north now, it would cause a stroke now….and not in 3 days. So I would be curious if CEP shows a difference in a much shorter periprocedural window.
I’d also wonder if pre-procedural carotid assessments were done. Perhaps manipulation within diseased carotids themselves caused carotid embolic stroke that offsets any benefit from preventing aortic valve embolic stroke.
Our clinical diagnosis of stroke may be insufficiently sensitive to detect a difference. Brain imaging studies do show a reduction of new lesions with CEP
https://eurointervention.pcronline.com/article/cerebral-embolic-protection-during-transcatheter-heart-interventions#:~:text=Overall%2C%20there%20were%20more%20patients,27%25;%20p=0.017).&text=Two%20(7%25)%20strokes%20at,control%20(no%20device)%20arm.
Still, plenty of new lesions either way (as many as 84% without CEP).
I'd opt for CEP with TAVR, but opt out of TAVR unless I had severe, symptomatic AS.
I don't see the problem. Miniscule differences in an event or complication are meaningless to those with common sense. With small numbers of endpoints tiny differences can be made to look significant by quoting relative risk differences. But this is usually promotional propaganda designed to fool people and suggests bias on the part of the reporter. The wide acceptance and prescription of statin drugs is evidence of the power of this sort of misleading presentation.
1. Using the relative 10% reduction in the endpoint from 1.4% to 1.2% to describe the findings is a terrible distortion of the magnitude of the difference and should not be done.
2. Seriously how much have you improved a patient’s lot by reducing his stroke risk this trivially? Is this device more or less effective than the brand of surgical soap in the cath lab, or how well the operator slept last night? Whatever incremental improvement this might represent isn’t enough to matter to a patient or his doctors today.
3. We need new ethics to guide us in the selection of new therapies. When financial cost is not considered there are no brakes on the train, and next stop is a bankrupt country.
"Doctors don’t get paid more for using the device, which involves making an extra arterial puncture and taking some time manipulating the device into the carotids."
CPT 33370: This code is specifically for reporting the transcatheter placement and removal of Sentinel embolic protection devices during procedures like TAVR.
Reimbursement Calculation:
The CMS (Centers for Medicare & Medicaid Services) adopted the recommendation for valuing code +33370 at 2.5 work RVUs. The total facility RVUs are 3.93, and the current conversion factor is $32.35. The base payment rate is calculated by multiplying the total RVUs (2.5 + 3.93 = 6.43) by the conversion factor ($32.35), resulting in $127.
1. It's an HR of .99; there is no reason to do this!
2. Would you recommend the addition of CEP for your spouse or parents or closest friends?
3. That you correctly wonder why the risk of stroke with TAVI is NOT [edit] a lot higher shows that you and your field do not actually understand the underlying mechanisms of stroke and some kind of inherent stroke prevention mechanisms in humans. Better understand the underlying mechanisms.
4. If you are looking to get rich, buying 10 lottery tickets is just as dumb as buying 1 lottery ticket.
The fundamental problem is considering a relative change as clinically meaningful. Clinical decisions should be based on absolute differences. It makes no sense to state that a ~30% relative change is clinically important.