4 Comments
User's avatar
⭠ Return to thread
JDK's avatar

Ugh. Here is difficult theoretical disagreement of approach.

"Any journal wanting to say that there is evidence that a treatment doesn't work should be forced to base that on a Bayesian probability that the treatment effect is clinically negligible."

I understand what you are saying but I think our assumption must be that treatments do NOT work UNLESS they are reasonable shown to work.

Expand full comment
Frank Harrell's avatar

I could not disagree more with that. If that were the case you would conclude from a study with n=2 that the treatment doesn't work. Also, if you took that approach there would be less incentive to force studies to have some likelihood of being informative before they are approved for launch. We need to get the idea across that futile studies are actually futile.

Expand full comment
JDK's avatar

A study with n =2, is not a study. It is a conversation in which anecdotes are shared.

A futility study is for showing that "the standard of care" is ineffective, not for showing that a hypothesis better treatment doesn't work.

We have different viewpoints. Your approach, if I understand it correctly, leads to kitchen sink medicine. Let's try this until someone shows it doesn't work. How can that be progress? Maybe you don't mean that - maybe you have assumption that bioplausibility is a prerequisite? But even that is problematic because medicine really knows a lot less than it imagines.

🤷‍♂️ It is possible I mistaken. If things were easy there would be only good studies.

Expand full comment
Frank Harrell's avatar

So right! My main philosophy is that we need to get away from methods such as p-values and null hypothesis testing that can only show evidence against an assertion and move towards methods that provide evidence in favor of any possible assertion. One such assertion is that the treatment matters little. Bayes provides direct evidence for that, e.g. Pr(0.925 < hazard ratio < 1/0.925). Anyone wanting to draw a conclusion such as a drug doesn't work needs supporting evidence, not just failure to reject the null which can happen for a variety of reasons.

Expand full comment