This is a very interesting issue, and it is wonderful that Frank Harrell has contributed to this discussion. Like him, I have promulgated confidence intervals and their correct interpretation for years but appreciate confusion still occurs. I have three comments to make.
1) When discussing relative risks or hazard ratios one should always…
This is a very interesting issue, and it is wonderful that Frank Harrell has contributed to this discussion. Like him, I have promulgated confidence intervals and their correct interpretation for years but appreciate confusion still occurs. I have three comments to make.
1) When discussing relative risks or hazard ratios one should always consider the absolute risk difference as well. In the DANCAVAS the result was the intervention reduced the risk of dying by 5% relative to the control (my emphasis). If the risk of dying on the control was low, then the risk difference could be very small and not important to those at risk.
2) I think it is a false dichotomy to say a treatment ‘works’ or ‘ does not work’ . Treatments may have very small effects, but still said ‘work’ if a very large trial was conducted and a statistically significant effect found. What is needed before the trial is conducted is a robust discussion as to what clinically important differences are worth considering . (Note not a ‘significant’ difference). There has been much work on determining a meaningful difference. As I have stated many times, one of the most useful aspects of a sample size calculation is it requires interested parties to come up with some effect size that would be considered useful. In the PROTECTED-TAVR trial, what was considered a useful effect size? It should be available in the authors' reported sample size calculation. This would help interpret a confidence interval.
3) It is easy to forget that the point estimate is our best guess as to the true effect. Thus I would not promote reporting where only the confidence interval is given.
But re 3): a point estimate is NOT "the" "best" guess as to the true effect. It is a guess. It is not a random guess, (that would only occur if CI was 0 to ♾️), but to say it is "best" guess I think is not a good claim or way to think about .
There is point estimate of 1.01 with 95% CI from 0.72 to 1.42. Somehow there is way to magically know the "true" effect.
You get to bet on 1.01 as the true effect. (What you call your "best" guess.) I get to bet on NOT 1.01 as true effect. I will take my bet every day (with properly sized bet, see Kelly). I think if you thought about it you'd take my bet too.
Where human beings are involved we should minimize our enthusiasm for likelihood of benefit and maximize our caution for risk of harm. The focus should be on the tails. Point estimates distract from that focus. And to say they are our "best" guess also distracts and overstates our knowledge.
This is a very interesting issue, and it is wonderful that Frank Harrell has contributed to this discussion. Like him, I have promulgated confidence intervals and their correct interpretation for years but appreciate confusion still occurs. I have three comments to make.
1) When discussing relative risks or hazard ratios one should always consider the absolute risk difference as well. In the DANCAVAS the result was the intervention reduced the risk of dying by 5% relative to the control (my emphasis). If the risk of dying on the control was low, then the risk difference could be very small and not important to those at risk.
2) I think it is a false dichotomy to say a treatment ‘works’ or ‘ does not work’ . Treatments may have very small effects, but still said ‘work’ if a very large trial was conducted and a statistically significant effect found. What is needed before the trial is conducted is a robust discussion as to what clinically important differences are worth considering . (Note not a ‘significant’ difference). There has been much work on determining a meaningful difference. As I have stated many times, one of the most useful aspects of a sample size calculation is it requires interested parties to come up with some effect size that would be considered useful. In the PROTECTED-TAVR trial, what was considered a useful effect size? It should be available in the authors' reported sample size calculation. This would help interpret a confidence interval.
3) It is easy to forget that the point estimate is our best guess as to the true effect. Thus I would not promote reporting where only the confidence interval is given.
Mike Campbell
University of Sheffield
Points 1 and 2 are well taken.
But re 3): a point estimate is NOT "the" "best" guess as to the true effect. It is a guess. It is not a random guess, (that would only occur if CI was 0 to ♾️), but to say it is "best" guess I think is not a good claim or way to think about .
There is point estimate of 1.01 with 95% CI from 0.72 to 1.42. Somehow there is way to magically know the "true" effect.
You get to bet on 1.01 as the true effect. (What you call your "best" guess.) I get to bet on NOT 1.01 as true effect. I will take my bet every day (with properly sized bet, see Kelly). I think if you thought about it you'd take my bet too.
Where human beings are involved we should minimize our enthusiasm for likelihood of benefit and maximize our caution for risk of harm. The focus should be on the tails. Point estimates distract from that focus. And to say they are our "best" guess also distracts and overstates our knowledge.