Reminds me of those who say language is completely arbitrary, and so we can interpret a legal statute any way we want. I ask them, as judges, do we construct the law, or do we construe it? Similarly, do we read studies, or do we read into studies? Like the law, the answer for studies is so clear that the need to even ask the question is an epistemological tragedy. It implicates our education.
Love this:”Rather than calling interpretation of trials, such as the new LAAC trials, a Rorschach test, I’d liken it more closely to a test of intelligence and neutrality.”
You know what’s in the eye of the beholder? Beauty. Different folks, ummm, different strokes (ahem).
But science should not be in the eye of the beholder. More ischemic strokes (lower end of 95% CI of 1.00) is actually just more ischemic strokes.
If a doctor offers a patient a procedure that is worse than usual care for efficacy, and no better than usual care for safety, that doctor is either very dumb, or very corrupt. That’s the image I conjure from that particular Rorschach.
Rorschach assumes the item observed has no inherent meaning. If the study has no inherent meaning, what value does it have? A study has to say something. The ability to read the meaning in the study should be assumed in a physician. Someone who assumes anyone’s interpretation is valid should not be practicing medicine
A comment that one's interpretation of a trial's outcome is a Rorschach test, he is stating that all conclusions are valid, even when the data flatly contradicts the conclusion that a procedure -- LaaC in this case -- is safe and effective. Dr Mandrola (and I) strongly disputes this kind of 'everyone's right' mentality. It's like getting a participation badge just for showing up, no matter how poorly you perform.
John, please distill your concerns about the CHAMPION AF trial into no more than 3 paragraphs — a crisp critique of endpoints and analysis without innuendo or ad hominem attacks — then send it as a letter to the NEJM. This is how to express your concerns in a forum that will be read by the journal editors. If published, your letter and the paper author’s answers to it will influence the thousands of cardiologists and internists who read this most influential journal regularly.
May I suggest that a summary of your article would be statistics don't lie, but statisticians do. Or perhaps, demings formulation- if you torture the data enough it will confess.
The data does not speak for itself. Joseph Stalin knew that it was not data that matttered ,but only how and by whom the data was selected and massaged: “ I consider it completely unimportant who in the party will vote, or how; but what is extraordinarily important is this—who will count the votes, and how.” (1923; Boris Bazhanov The Memoirs of Stalin's Former Secretary).
The average physician would never be able to do the kind of analysis you do. He/she is oriented to "follow the science", meaning peer reviewed papers and protocolos supposed to have done the analysis for him/her. In this sense, I think that your colleague's comment was correct.
I agree Maria. Dr. Mandrola’s critique of the CHAMPION AF trial results is spot on (as usual, he seems to have misinterpreted his colleagues comments.
The “priors” in question are not limited to the results of related trials, but the sum total of the potential biases / predispositions affecting one’s acceptance of the trial author’s conclusions. Physicians most interested in the outcome are interventional cardiologists, who, for many reasons, are strongly predisposed to “intervening”. Hence the Rorschach test comment and the related, and likely accurate prediction that the CHAMPION trial will lead to increased adoption of LAAC, despite the fact that the procedures hypothesized advantages remain entirely unproven, if not disproven.
John is right, evidence is evidence. That is why the Rorschach has never been a truly useful tool. When it is useful, the abnormal response is obvious and achievable through simple normal interviewing.
Reminds me of those who say language is completely arbitrary, and so we can interpret a legal statute any way we want. I ask them, as judges, do we construct the law, or do we construe it? Similarly, do we read studies, or do we read into studies? Like the law, the answer for studies is so clear that the need to even ask the question is an epistemological tragedy. It implicates our education.
Love this:”Rather than calling interpretation of trials, such as the new LAAC trials, a Rorschach test, I’d liken it more closely to a test of intelligence and neutrality.”
You know what’s in the eye of the beholder? Beauty. Different folks, ummm, different strokes (ahem).
But science should not be in the eye of the beholder. More ischemic strokes (lower end of 95% CI of 1.00) is actually just more ischemic strokes.
If a doctor offers a patient a procedure that is worse than usual care for efficacy, and no better than usual care for safety, that doctor is either very dumb, or very corrupt. That’s the image I conjure from that particular Rorschach.
Rorschach assumes the item observed has no inherent meaning. If the study has no inherent meaning, what value does it have? A study has to say something. The ability to read the meaning in the study should be assumed in a physician. Someone who assumes anyone’s interpretation is valid should not be practicing medicine
A comment that one's interpretation of a trial's outcome is a Rorschach test, he is stating that all conclusions are valid, even when the data flatly contradicts the conclusion that a procedure -- LaaC in this case -- is safe and effective. Dr Mandrola (and I) strongly disputes this kind of 'everyone's right' mentality. It's like getting a participation badge just for showing up, no matter how poorly you perform.
This is an excellent analysis.
John, please distill your concerns about the CHAMPION AF trial into no more than 3 paragraphs — a crisp critique of endpoints and analysis without innuendo or ad hominem attacks — then send it as a letter to the NEJM. This is how to express your concerns in a forum that will be read by the journal editors. If published, your letter and the paper author’s answers to it will influence the thousands of cardiologists and internists who read this most influential journal regularly.
May I suggest that a summary of your article would be statistics don't lie, but statisticians do. Or perhaps, demings formulation- if you torture the data enough it will confess.
He was very personally invested in plying this particular method, and it clearly paid dividends for him. Beware data torturers!
The data does not speak for itself. Joseph Stalin knew that it was not data that matttered ,but only how and by whom the data was selected and massaged: “ I consider it completely unimportant who in the party will vote, or how; but what is extraordinarily important is this—who will count the votes, and how.” (1923; Boris Bazhanov The Memoirs of Stalin's Former Secretary).
I think Stalin's attitude was very much in the 'torture the data' school.
The incidence of major bleeding might be 23% in the study group. When it happens to you is 100% ….
The average physician would never be able to do the kind of analysis you do. He/she is oriented to "follow the science", meaning peer reviewed papers and protocolos supposed to have done the analysis for him/her. In this sense, I think that your colleague's comment was correct.
I agree Maria. Dr. Mandrola’s critique of the CHAMPION AF trial results is spot on (as usual, he seems to have misinterpreted his colleagues comments.
The “priors” in question are not limited to the results of related trials, but the sum total of the potential biases / predispositions affecting one’s acceptance of the trial author’s conclusions. Physicians most interested in the outcome are interventional cardiologists, who, for many reasons, are strongly predisposed to “intervening”. Hence the Rorschach test comment and the related, and likely accurate prediction that the CHAMPION trial will lead to increased adoption of LAAC, despite the fact that the procedures hypothesized advantages remain entirely unproven, if not disproven.
John is right, evidence is evidence. That is why the Rorschach has never been a truly useful tool. When it is useful, the abnormal response is obvious and achievable through simple normal interviewing.