The Study of the Week considers a major trial of a pricy device that breaks a basic rule of science. Yet a major journal and regulators allowed it. What should one think?
I’m having difficulty in understanding the objection to the GUIDE HF trial’s authors’ statement: “However, a pre-COVID-19 impact analysis indicated a possible benefit of haeodynamic-guided management on the primary outcome in the pre-COVID-19 period…”
Dr. Mandrola writes:
"It turns out that three-fourths of the patients had finished follow-up before the pandemic and when they looked at this group, there were fewer primary events in the CARDIOMEMS arm, and now, the difference barely made the p-value threshold at 0.049. In the one-quarter of patients followed during the pandemic there were no differences in events. This is weird because a wireless device ought to have over-performed during a time when in-person follow-up was curtailed."
Suppose the authors had decided to end the trial at the beginning of the pandemic, which would have been a reasonable decision in order to avoid incurring a significant confounder induced by curtailment of in-person follow-up (and one which would have been expected to enhance the perceived effectiveness of the device). Would they then not have been justified in making the statement: “Haeodynamic-guided management of heart failure resulted in a lower composite endpoint rate of mortality and heart failure events compared with the control group in the overall study analysis.”?
Similarly, I don’t fully understand Sean’s and Dr. Mann’s objections to changing the way one looks at data after the fact. Suppose that Dr. Mandrola had instead originally decided to study the association of that variable with that procedural outcome among older patients in his hospital, and that that association did indeed meet the threshold for significance; would this result be any more significant than the identical result that would be obtained by Sean doing what Dr. Mandrola requested? Since the only difference is Dr. Mandrola’s state of mind when he designd the study, to confer more value on one result than the other seems to me to partake of “magical thinking.”
Forgive me if I'm misunderstanding your question, but I believe part of the answer is this. If you go back into your data and "fish" for groups of data that come up with the required p-value, you will likely find something. In other words, if you are looking for something that only has a 5% chance of happening, you will find it eventually (because 5% is not zero.)
I could understand, however, if you slice up your data and find a significant result that seems it could be a real effect. I believe in this case, it could be wise to perform another study limited to that group. For example, suppose one were looking to see if a certain treatment improved outcomes in a population, and overall the study's results were not significant. But, if the experimenter removed all the participant data, except for, say, black males under the age of 20, and found that yielded a significant result, that COULD indicate a real effect. A curious researcher might go back and do another study on just that population alone. I guess their decision to do so would be based on many factors, like if the effect seemed plausible, and what the level of significance was.
I'm not a professional researcher, so take my answer with a grain of salt! :)
"Suppose the authors had decided to end the trial at the beginning of the pandemic,..." They could have done this in blinded fashion and taken the hit to their statistical power due to lower patient numbers. In this case it would have paid off for them as the study would have had a putatively positive outcome. However, they would not have known this a priori and would have been quite nervous about doing so given that the trial size was specified based on the expected minimum effect size and they would have risked not meeting significance due to reduced statistical power. They actually probably thought that the pandemic would have made it more likely to see improvement so there was no incentive to stop the trial early.
"Similarly, I don’t fully understand Sean’s and Dr. Mann’s objections to changing the way one looks at data after the fact..."
When you do not control for a variable on the front end, then the risk of introducing bias / confounding factors is quite high. In your hypothesized example of the pre- and post-hoc studies, it is not Dr. Mandrola's state of mind that changed, but the quality of the data used in the comparison. In Dr. Mandrola's example, if age was not specifically controlled for in the study, it is unlikely that the treatment and control groups reflect a randomized sample with respect to this variable, increasing the risk that any difference between the groups is due to a factor un-related to the treatment.
It would be fine to do the analysis of the elderly population that Dr. Mandrola requested (along with an analysis of how the subgroup analysis effects the robustness of the statistics) for the purposes of planning future studies to confirm, but it should never be used as the basis for a product approval absent extraordinary circumstances.
You’ve completely missed my point. In each case I was asking you to consider a Gedankenexperiment which would yield exactly the same data set, but one for which the same statistical analysis would be considered entirely valid.
Perhaps we are talking past each other, but the point I was trying to make is that predefining a new variable of interest will inherently change the execution of the study and resulting data set because you will factor the new variable of interest into your randomization strategy / group assignment and sample size / power calculations. Under the artificial presumption that the pre-defined and retrospective studies generated identical data output, the confidence in the data from the pre-defined study would be higher because we would know that the pre-defined study decreased the potential for confounders whereas we would not the same confidence in the context of the retrospective analysis study.
Indeed, I think we have been talking past each other, but thanks to your explanation I now do fully understand the objections made to Dr. Mandrola’s request. I had incorrectly assumed that the hypothetical alternative scenario would have yielded, if not exactly the same data set, at least one that was statistically equivalent.
Yet, I still don’t understand the objection to the authors adding the additional sentence to the conclusion. Here, I think you’ve missed my point because in the hypothetical scenario (terminating the trial at the beginning of the pandemic) there would have been no retrospective study, the data sets would indeed have been identical, and there would have been no predefined new variable of interest.
What if they had a done post-hoc power analysis for this pre-covid subgroup which showed a positive benefit and they find that the power was adequate enough? I know post-hoc power analysis is frowned upon by many, but in dire circumstances like these I think it is justifiable to do it. Not sure whether the study authors reported this to FDA or not.
Power analysis is performed in the study planning and design phase in order to determine the appropriate sample size to evaluate the hypothesis under test. Once you have identified an effect between treatment group and control group (or subgroups as in the CARDIOMEMS study) it is no longer informative. The one place I have seen post hoc power analysis sometimes applied is when you have a subgroup difference that does not meet significance and you want to show that the study was not adequately powered for that analysis (i.e., we cannot reject a treatment effect because the study was not adequately powered for that variable).
I did some further looking into the study and the pre-CoVID analysis was actually pre-specified in an amendment to the statistical analysis plan of the protocol while the study was blinded, so this is not an example of a true post hoc analysis. This was justified on the basis that the CoVID-19 pandemic was a confounding factor for the study, but the statistical criteria that they used to substantiate that the pre-CoVID and post-CoVID were different (p value 0.15) was not as conclusive as one would like. The investigators cite that hospital visits were significantly reduced post pandemic compared to pre-pandemic. In my opinion, the lingering issue is that the study investigators have only loosely demonstrated that the pre- and post pandemic cohorts were different in accessing medical care but have not really demonstrated a plausible mechanism for why that would essentially eliminate the benefit of the CARDIOMEMS device. However, given that the FDA signed off on the revised pre-specified statistical analysis plan while the study was still blinded, it was probably difficult to reject the application.
The power analysis during study protocol and SAP development is different. I am only speaking about post-hoc analysis. Regardless of whether it is 'typically' only done to assess 'absence of evidence is not evidence of absence' for a subgroup effect, it can still be done to justify say 'look, we know that the primary endpoint didn't show any benefit, but this subgroup had a significant effect and our post-hoc power analysis shows that the sample size for this subgroup analysis was also having adequate power'. Not a concrete evidence, but I would take this as a positive hypothesis generation.
Anyway, for this GUIDE-HF trial, it seems like this discussion is perhaps not relevant now as I see you mentioned this analysis was 'preplanned' and was approved before unblinding. In that regard, I think this main article of 'how not to look at the data' is a misleading one because the example (cardiomems guide-hf trial) it shows is not the right one.
Great piece once again. I had read your Medscape column last spring. It’s not hard to understand how one goes from a skeptic to a cynic, with examples like this.
Given that Cardiomems was LESS effective at preventing HHF during the pandemic when we all went to telehealth, it is also possible the investigators were simply woeful and sorry clinicians when it came to physical examination, and could not discern a high JVP if it struck them btw the eyes.
Changing the data analysis after the fact (called post hoc analysis) carries a high risk of biasing the results due to the introduction of confounders or compromising the validity of the comparator groups. The form of post hoc analysis that is cited in this post is what is called sub-group analysis (i.e., essentially looking across a range of variables to see if any subgroups within the study exhibited a statistically significant treatment effect). In the authors case he wanted a subgroup analysis on the older population, in the cardiac monitoring case it was the pre- vs. post- pandemic groups. One of the issues with this is that if you did not specify these variables before the study, it can compromise your randomization because the baseline participant characteristics are no longer matched between treatment and control with respect to the new variable. However, the main issue is that subgroup analysis is often performed across multiple variables / comparisons which significantly increase the likelihood of a type I error (concluding that results are significant when they were actually due to random chance or confounding factors). The more subgroups you analyze further increases the risk of a type I error. If I perform subgroup analysis across 10 different variables, then the chances of at least one subgroup exhibiting a p value < 0.05 is actually quite high. For subgroup analysis to be at all valid it is critical that the investigators can establish a mechanism by which that new variable was critical to the outcome of the study and even then, it is only useful in the generation of your hypothesis for a subsequent controlled study designed to test that variable. There have been innumerable examples where the results of a subgroup identified by a seemingly logical post hoc analysis failed to replicate in a subsequent study specifically designed to test that subgroup against a matched control.
It will depend on the design of the study as to whether or not it is valid to do so. Parsing the data might give some insight for further studies in the subgroups though.
Thank you for sharing! “What should we think..?”. I am scratching my head that more people don’t “get it”. Perhaps sharing more examples like this will open more people’s eyes. The system is biased, deeply, perhaps irreparably. @Robert Malone did a nice piece about Integrity Lost, which comes to mind here again.
Money talks. Science, not so much. Welcome to US capitalism. Medicine is our biggest industry in America and sadly, our entire medical industrial complex is complicit.
John, just look at the “evidence” behind “maintaining” our board certification as I studied some time ago here (http://drwes.blogspot.com/2014/10/reviewing-regulators.html). No way this work would be published anywhere in today’s journals. So they put it behind a paywall and imply that it’s better not to look at the evidence and just pay those fees to our US medical board “chosen ones,” or else.
Very puzzling, especially after I read the Dr. Mandrola's Medscape 3/18/22 piece describing what might (or should likely) have happened during the COVID period in the trial (when hospitalizations were actually MORE frequent in the CardioMEMS arm).
Before approving a $10+ billion expansion of device use, why didn't the FDA demand a redo of the trial? Unplanned subgroup analysis can be helpful for generating new hypotheses, but they need to be tested! And who doesn't want replication (other than the device makers and those who profit from it).
The pandemic opened my eyes to two things. People's resolve is so, so, so much lower than I could possibly have imagined. And two, FDA isn't there to protect the populous. It's a marketing arm and a LinkedIn for a juicy consulting job after one's allegiance as a political bureaucract measures up to Bourla's greed.
Thank God the forefathers had the wisdom to put in the Bill of Rights. An made it so hard to amend it by feckless politicians. These morons can't seem to remember the litmus test of "do I want my opponent to have this power?"
As Cocaine Mitch once said "You’ll regret this, and you may regret this a lot sooner than you think,”
This reminds me of an assignment I was given as an undergrad psychology student. We had to find examples of bad research. A member of my team found the Journal of Parapsychology in the library. What a goldmine! We found a study that tested whether mice could received "thought messages" from humans that could help them through a maze. The researchers timed how long it took mice to get through a maze, both with a human "thought" guide, and without. There was no significant difference between the two groups. However, the researchers sliced apart their data until they found some that would get them the p-value they needed. Their conclusion? Mice are able to receive thought messages from humans, but only before 10am!
This is an excellent and convincing illustration of the pervasive contamination of medical "science" by commercial interests. The dynamic is decades old, but likely progressing with every passing journal publication. When reading a medical journal article, my first view is the authors' conflict of interest statements. When industry ties are revealed, as is nearly always the case, the study deserves the highest degree of skepticism.
There is further suppression of honest scientifically sound research due to publication bias and belief in the dominant paradigm even if the researchers are apparently free of conflicts of interest.
Current medical journal publications feel a lot like infomercials for a vacuum cleaner or set of steak knives.
Wow, thank you for your continued efforts to enlighten the masses, or at least those of us who aren't taking the time to read and critique the publications. We in primary care depend on our specialist colleagues to root out the rot; so many of us simply follow their lead. If the specialists think it's a good idea, we should support it, no?
Still waiting to see an exegetical commentary on Sensible Medicine from you experts regarding the use of a composite categorical "outcome" in randomized experiments. You know what I mean: Where some binary "outcome" is tallied as "yes" or "no" if any one of three dependent variables (as in the saga presented here) rings the buzzer or not, respectively. What the hell, why not kill many birds with one stone --- viz. execute studies where the tallied "outcome" comprises some mix of 7 putative dependent variables, or 14, or 124, etc.?
And I declare that I have not received any moolah, or free trips, or jewelry, or angora sweaters, or Mont Blanc 149 fountain pens from any pharmaceutical firms (at least since 1999).
Insightful as usual. Thank you.
Thank you for this. Please keep writing.
I’m having difficulty in understanding the objection to the GUIDE HF trial’s authors’ statement: “However, a pre-COVID-19 impact analysis indicated a possible benefit of haeodynamic-guided management on the primary outcome in the pre-COVID-19 period…”
Dr. Mandrola writes:
"It turns out that three-fourths of the patients had finished follow-up before the pandemic and when they looked at this group, there were fewer primary events in the CARDIOMEMS arm, and now, the difference barely made the p-value threshold at 0.049. In the one-quarter of patients followed during the pandemic there were no differences in events. This is weird because a wireless device ought to have over-performed during a time when in-person follow-up was curtailed."
Suppose the authors had decided to end the trial at the beginning of the pandemic, which would have been a reasonable decision in order to avoid incurring a significant confounder induced by curtailment of in-person follow-up (and one which would have been expected to enhance the perceived effectiveness of the device). Would they then not have been justified in making the statement: “Haeodynamic-guided management of heart failure resulted in a lower composite endpoint rate of mortality and heart failure events compared with the control group in the overall study analysis.”?
Similarly, I don’t fully understand Sean’s and Dr. Mann’s objections to changing the way one looks at data after the fact. Suppose that Dr. Mandrola had instead originally decided to study the association of that variable with that procedural outcome among older patients in his hospital, and that that association did indeed meet the threshold for significance; would this result be any more significant than the identical result that would be obtained by Sean doing what Dr. Mandrola requested? Since the only difference is Dr. Mandrola’s state of mind when he designd the study, to confer more value on one result than the other seems to me to partake of “magical thinking.”
Forgive me if I'm misunderstanding your question, but I believe part of the answer is this. If you go back into your data and "fish" for groups of data that come up with the required p-value, you will likely find something. In other words, if you are looking for something that only has a 5% chance of happening, you will find it eventually (because 5% is not zero.)
I could understand, however, if you slice up your data and find a significant result that seems it could be a real effect. I believe in this case, it could be wise to perform another study limited to that group. For example, suppose one were looking to see if a certain treatment improved outcomes in a population, and overall the study's results were not significant. But, if the experimenter removed all the participant data, except for, say, black males under the age of 20, and found that yielded a significant result, that COULD indicate a real effect. A curious researcher might go back and do another study on just that population alone. I guess their decision to do so would be based on many factors, like if the effect seemed plausible, and what the level of significance was.
I'm not a professional researcher, so take my answer with a grain of salt! :)
"Suppose the authors had decided to end the trial at the beginning of the pandemic,..." They could have done this in blinded fashion and taken the hit to their statistical power due to lower patient numbers. In this case it would have paid off for them as the study would have had a putatively positive outcome. However, they would not have known this a priori and would have been quite nervous about doing so given that the trial size was specified based on the expected minimum effect size and they would have risked not meeting significance due to reduced statistical power. They actually probably thought that the pandemic would have made it more likely to see improvement so there was no incentive to stop the trial early.
"Similarly, I don’t fully understand Sean’s and Dr. Mann’s objections to changing the way one looks at data after the fact..."
When you do not control for a variable on the front end, then the risk of introducing bias / confounding factors is quite high. In your hypothesized example of the pre- and post-hoc studies, it is not Dr. Mandrola's state of mind that changed, but the quality of the data used in the comparison. In Dr. Mandrola's example, if age was not specifically controlled for in the study, it is unlikely that the treatment and control groups reflect a randomized sample with respect to this variable, increasing the risk that any difference between the groups is due to a factor un-related to the treatment.
It would be fine to do the analysis of the elderly population that Dr. Mandrola requested (along with an analysis of how the subgroup analysis effects the robustness of the statistics) for the purposes of planning future studies to confirm, but it should never be used as the basis for a product approval absent extraordinary circumstances.
You’ve completely missed my point. In each case I was asking you to consider a Gedankenexperiment which would yield exactly the same data set, but one for which the same statistical analysis would be considered entirely valid.
Perhaps we are talking past each other, but the point I was trying to make is that predefining a new variable of interest will inherently change the execution of the study and resulting data set because you will factor the new variable of interest into your randomization strategy / group assignment and sample size / power calculations. Under the artificial presumption that the pre-defined and retrospective studies generated identical data output, the confidence in the data from the pre-defined study would be higher because we would know that the pre-defined study decreased the potential for confounders whereas we would not the same confidence in the context of the retrospective analysis study.
Indeed, I think we have been talking past each other, but thanks to your explanation I now do fully understand the objections made to Dr. Mandrola’s request. I had incorrectly assumed that the hypothetical alternative scenario would have yielded, if not exactly the same data set, at least one that was statistically equivalent.
Yet, I still don’t understand the objection to the authors adding the additional sentence to the conclusion. Here, I think you’ve missed my point because in the hypothetical scenario (terminating the trial at the beginning of the pandemic) there would have been no retrospective study, the data sets would indeed have been identical, and there would have been no predefined new variable of interest.
What if they had a done post-hoc power analysis for this pre-covid subgroup which showed a positive benefit and they find that the power was adequate enough? I know post-hoc power analysis is frowned upon by many, but in dire circumstances like these I think it is justifiable to do it. Not sure whether the study authors reported this to FDA or not.
Power analysis is performed in the study planning and design phase in order to determine the appropriate sample size to evaluate the hypothesis under test. Once you have identified an effect between treatment group and control group (or subgroups as in the CARDIOMEMS study) it is no longer informative. The one place I have seen post hoc power analysis sometimes applied is when you have a subgroup difference that does not meet significance and you want to show that the study was not adequately powered for that analysis (i.e., we cannot reject a treatment effect because the study was not adequately powered for that variable).
I did some further looking into the study and the pre-CoVID analysis was actually pre-specified in an amendment to the statistical analysis plan of the protocol while the study was blinded, so this is not an example of a true post hoc analysis. This was justified on the basis that the CoVID-19 pandemic was a confounding factor for the study, but the statistical criteria that they used to substantiate that the pre-CoVID and post-CoVID were different (p value 0.15) was not as conclusive as one would like. The investigators cite that hospital visits were significantly reduced post pandemic compared to pre-pandemic. In my opinion, the lingering issue is that the study investigators have only loosely demonstrated that the pre- and post pandemic cohorts were different in accessing medical care but have not really demonstrated a plausible mechanism for why that would essentially eliminate the benefit of the CARDIOMEMS device. However, given that the FDA signed off on the revised pre-specified statistical analysis plan while the study was still blinded, it was probably difficult to reject the application.
The power analysis during study protocol and SAP development is different. I am only speaking about post-hoc analysis. Regardless of whether it is 'typically' only done to assess 'absence of evidence is not evidence of absence' for a subgroup effect, it can still be done to justify say 'look, we know that the primary endpoint didn't show any benefit, but this subgroup had a significant effect and our post-hoc power analysis shows that the sample size for this subgroup analysis was also having adequate power'. Not a concrete evidence, but I would take this as a positive hypothesis generation.
Anyway, for this GUIDE-HF trial, it seems like this discussion is perhaps not relevant now as I see you mentioned this analysis was 'preplanned' and was approved before unblinding. In that regard, I think this main article of 'how not to look at the data' is a misleading one because the example (cardiomems guide-hf trial) it shows is not the right one.
Great piece once again. I had read your Medscape column last spring. It’s not hard to understand how one goes from a skeptic to a cynic, with examples like this.
Given that Cardiomems was LESS effective at preventing HHF during the pandemic when we all went to telehealth, it is also possible the investigators were simply woeful and sorry clinicians when it came to physical examination, and could not discern a high JVP if it struck them btw the eyes.
Non-scientist here. Can someone explain to me WHY it's bad to change the way you look at data after the fact?
Changing the data analysis after the fact (called post hoc analysis) carries a high risk of biasing the results due to the introduction of confounders or compromising the validity of the comparator groups. The form of post hoc analysis that is cited in this post is what is called sub-group analysis (i.e., essentially looking across a range of variables to see if any subgroups within the study exhibited a statistically significant treatment effect). In the authors case he wanted a subgroup analysis on the older population, in the cardiac monitoring case it was the pre- vs. post- pandemic groups. One of the issues with this is that if you did not specify these variables before the study, it can compromise your randomization because the baseline participant characteristics are no longer matched between treatment and control with respect to the new variable. However, the main issue is that subgroup analysis is often performed across multiple variables / comparisons which significantly increase the likelihood of a type I error (concluding that results are significant when they were actually due to random chance or confounding factors). The more subgroups you analyze further increases the risk of a type I error. If I perform subgroup analysis across 10 different variables, then the chances of at least one subgroup exhibiting a p value < 0.05 is actually quite high. For subgroup analysis to be at all valid it is critical that the investigators can establish a mechanism by which that new variable was critical to the outcome of the study and even then, it is only useful in the generation of your hypothesis for a subsequent controlled study designed to test that variable. There have been innumerable examples where the results of a subgroup identified by a seemingly logical post hoc analysis failed to replicate in a subsequent study specifically designed to test that subgroup against a matched control.
Thanks for your explanation!
Well said.
It will depend on the design of the study as to whether or not it is valid to do so. Parsing the data might give some insight for further studies in the subgroups though.
Thank you for sharing! “What should we think..?”. I am scratching my head that more people don’t “get it”. Perhaps sharing more examples like this will open more people’s eyes. The system is biased, deeply, perhaps irreparably. @Robert Malone did a nice piece about Integrity Lost, which comes to mind here again.
Money talks. Science, not so much. Welcome to US capitalism. Medicine is our biggest industry in America and sadly, our entire medical industrial complex is complicit.
John, just look at the “evidence” behind “maintaining” our board certification as I studied some time ago here (http://drwes.blogspot.com/2014/10/reviewing-regulators.html). No way this work would be published anywhere in today’s journals. So they put it behind a paywall and imply that it’s better not to look at the evidence and just pay those fees to our US medical board “chosen ones,” or else.
And they wonder why we’re burnt out…
Very puzzling, especially after I read the Dr. Mandrola's Medscape 3/18/22 piece describing what might (or should likely) have happened during the COVID period in the trial (when hospitalizations were actually MORE frequent in the CardioMEMS arm).
Before approving a $10+ billion expansion of device use, why didn't the FDA demand a redo of the trial? Unplanned subgroup analysis can be helpful for generating new hypotheses, but they need to be tested! And who doesn't want replication (other than the device makers and those who profit from it).
The pandemic opened my eyes to two things. People's resolve is so, so, so much lower than I could possibly have imagined. And two, FDA isn't there to protect the populous. It's a marketing arm and a LinkedIn for a juicy consulting job after one's allegiance as a political bureaucract measures up to Bourla's greed.
Thank God the forefathers had the wisdom to put in the Bill of Rights. An made it so hard to amend it by feckless politicians. These morons can't seem to remember the litmus test of "do I want my opponent to have this power?"
As Cocaine Mitch once said "You’ll regret this, and you may regret this a lot sooner than you think,”
This reminds me of an assignment I was given as an undergrad psychology student. We had to find examples of bad research. A member of my team found the Journal of Parapsychology in the library. What a goldmine! We found a study that tested whether mice could received "thought messages" from humans that could help them through a maze. The researchers timed how long it took mice to get through a maze, both with a human "thought" guide, and without. There was no significant difference between the two groups. However, the researchers sliced apart their data until they found some that would get them the p-value they needed. Their conclusion? Mice are able to receive thought messages from humans, but only before 10am!
The Ignobel awaits those morning mouse-whisperer researchers.
This is an excellent and convincing illustration of the pervasive contamination of medical "science" by commercial interests. The dynamic is decades old, but likely progressing with every passing journal publication. When reading a medical journal article, my first view is the authors' conflict of interest statements. When industry ties are revealed, as is nearly always the case, the study deserves the highest degree of skepticism.
There is further suppression of honest scientifically sound research due to publication bias and belief in the dominant paradigm even if the researchers are apparently free of conflicts of interest.
Current medical journal publications feel a lot like infomercials for a vacuum cleaner or set of steak knives.
Sadly, so do Fauci and the POTUS!
Wow, thank you for your continued efforts to enlighten the masses, or at least those of us who aren't taking the time to read and critique the publications. We in primary care depend on our specialist colleagues to root out the rot; so many of us simply follow their lead. If the specialists think it's a good idea, we should support it, no?
Again, thank you.
Another very informative post. Thank you.
As Lily Tomlin said, “I try to be cynical, but I just can’t keep up.” The FDA just approved a drug that will cost $3.5 million per dose.
Thank you for taking the time to write this article.
Still waiting to see an exegetical commentary on Sensible Medicine from you experts regarding the use of a composite categorical "outcome" in randomized experiments. You know what I mean: Where some binary "outcome" is tallied as "yes" or "no" if any one of three dependent variables (as in the saga presented here) rings the buzzer or not, respectively. What the hell, why not kill many birds with one stone --- viz. execute studies where the tallied "outcome" comprises some mix of 7 putative dependent variables, or 14, or 124, etc.?
And I declare that I have not received any moolah, or free trips, or jewelry, or angora sweaters, or Mont Blanc 149 fountain pens from any pharmaceutical firms (at least since 1999).