A stroke doc, I served as an investigator in the NIH t-PA stroke study wherein what initially was chosen as a secondary endpoint revolutionized our approach to acute ischemic stroke.
At the outset of the study, the primary outcome variable was neurological status at 24 hours, with neurologic/functional status at 3 months a secondary end point. An interim analysis indicated no significant difference between the active and placebo groups at 24 hours but a clear separation developing at 3 months. The NIH committee overseeing the study consequently permitted a change in the primary endpoint.
While routinely changing endpoints in midstream is not appropriate for all clinical research, in this case the change reflected our coming to understand that ischemic injury to the brain is not an all or nothing phenomenon, where the light switch turns off at the onset of neurologic deficit and will not turn on again unless rapid reperfusion is accomplished. Some neurons die rapidly and irreversibly with the onset of ischemia; others stop functioning but are still recoverable.
This became clear via the interim analysis and led to not only to an appropriate change in the protocol but also to a new understanding of how to regard and manage acute ischemic brain injury.
Especially in studies that are relatively long-term, our understanding of the disease state we are attempting to treat– and the treatment we are evaluating – may change radically during the course of the study. In such cases, clinging to a protocol conceived in ignorance may be a waste of time and a detriment to patients.
Good post. The initial of the study seems problematic as I would not expect use of high flow nasal oxygen to do very much to reduce mortality, whereas it would be expected (and in this study seems to have shown) to prevent intubation - very much a patient centered outcome. I would have preferred an RCT where intubation was the primary outcome but it seems unlikely that such a study will be conducted.
I am always suspicious of documented respiratory rate. I have never seen someone outside of nursing students actually stand there and measure a rate for an entire minute. Most of the time CNAs and RNs do little more than glance at the patient and decide if they “look like an 18” vs “look like a 20.”
Great piece, important nuances, and love the call for discussion. This trial is crucial, and I believe the endpoint was correct. Needed to know if the initial findings could be replicated, that's step one. They could not—hugely important knowledge. Next, with this knowledge in hand, seems to me it's right to focus trials on the important secondary endpoints you highlight. Thanks for an excellent article.
My thoughts - not new but never well-received! There should just be a set of (coherent) endpoints from studies (including trials) not primary, secondary, etc. The results on each should be published in a way consistent with their entry into a Multi Criteria Decision Analysis, where the conclusion following from a variety of endpoint weights, reflecting the user’s preferences, can be established. Researchers and content experts may publish the results from entering their preferences, but their's have no more epistemic legitimacy than anyone else’s. Empirical research can produce a relevant ‘set of population weights, but ultimately it is the individual patient’s preferences that are required for informed consent.
I agree mortality seems to be a harsh metric here when HFNC is the “starting” point or initial modality, and not the last word in hypoxemic RESP failure. Any patient who worsened on HFNC would have progressed to the next step (ie. intubation). So I think reduction in need for intubation (without any apparent cost in excess deaths) would be sufficient to vouch for this method.
OTOH, need for intubation was indeed reduced…but the reduction in ventilator-free days was small. Roughly 35 fewer pts on HFNC needed to go on a vent….but that resulted in only 2 fewer vent days on average….does that mean the HFNC pts who ultimately needed a ventilator in turn required a ventilator for longer periods?
Fascinating point about therapeutic fashion also. Amazing that a 100-patient trial was sufficient to establish that “this is what the cool cats do”…enough so that clinicians would not randomize their pts. That’s a bit of an indictment of our profession.
The dictum that Hypoxemic respiratory failure requires ventilator supportive care remains valid…
The widespread use of HFNC occurred mostly because of demand for temporizing or bridging patients to available rooms and ventilators during the SARs-CoV2 pandemic. It’s no panacea, and perhaps, leads to delayed treatment of hypoxemic resp failure if higher levels of PEEP are required for adequate systemic oxygenation.
Nothing wrong with secondary endpoints as long as they are well-defined so that we can judge the degree of subjectivity involved. Much of what we do in medicine is directed toward outcomes other than plain mortality. Composite endpoints, however, often seem to be employed in order to sell a drug or procedure. A worthwhile study should include the raw numbers and percentages for each of the primary and secondary endpoints. Practically any reader can then use common sense to assess the value of the drug or procedure under investigation.
In diseases causing profound hypoxemia (eg COVID 19), supplemental oxygen can be life saving. However, for most pulmonary disease, supplemental oxygen tides the patient over until the underlying cause of the hypoxemia can be treated and lung function improved. It’s unlikely that the primary outcome in this study was the one most likely to provide useful information. .
High flow oxygen therapy can improve ventilation, decrease the work of breathing and increase FRC, none of which regular flow oxygen can achieve, likely leading to the improvements in the secondary outcomes of this study.
Understanding what a therapy is likely to achieve, and how, leads to better choices of study outcomes.
Slavish reliance on death as the only worthwhile primary outcome may make both trialist and reviewers feel that a study is more important. But often whether a treatment helps patients live longer is the wrong question to ask.
Perhaps so. But surrogate endpoints exactly serve the Big Pharma cartel in every example I can think of, in cancer treatment, hypertension, CVD, on and on.
I am not treating patients. But I assume and believe that treating patients involves making them more comfortable.
But in the world of health research, all cause mortality is by far the most important factor and the one that Big Pharma inevitably prefers to squirm out of.
For instance, cancer treatments are sold with the promise that they postpone recurrence and lower death from cancer. That they don’t lengthen lifespan, or that they increase morbidity from side effects, is under reported.
CVD: statins are sold with the idea that they lower death from heart attacks. Their numerous bad side effects and the fact that they don’t extend lifespan is under reported.
Etc. This is just business as usual for Big Pharma.
So you bring up a narrow example that involves patient comfort and show a small but perhaps significant difference in all-cause mortality but I think this example is perhaps not particularly important. If it makes patients more comfortable to have oxygen delivered a different way, then wouldn’t it just be good medicine to deliver oxygen that way?
They used to deliver oxygen with a small amount of CO2, for many good reasons. So-called carbogen fell out of favor but perhaps it should be brought back. Patients are often over oxygenated, lowering their CO2 to dangerous levels.
We treat patients for two reasons : to feel better and/or live longer ( note the two do not always go in the same direction).
In very ill patients , living longer is not necessarily the most important outcome . John Spertus and his team illustrated this in an elderly cohort, who asserted ( summarizing) that freedom from disability was MUCH more important than prolongation of life ( my emphasis) .
Our primary purpose as caregivers is to relieve suffering .
Any chance of changing the title to "When Secondary Endpoints Best the Primary..."? The reason you are interested in those secondary endpoints is that they aren't surrogates -- they are clearly patient-important outcomes -- and you label them as secondary in the text.
A stroke doc, I served as an investigator in the NIH t-PA stroke study wherein what initially was chosen as a secondary endpoint revolutionized our approach to acute ischemic stroke.
At the outset of the study, the primary outcome variable was neurological status at 24 hours, with neurologic/functional status at 3 months a secondary end point. An interim analysis indicated no significant difference between the active and placebo groups at 24 hours but a clear separation developing at 3 months. The NIH committee overseeing the study consequently permitted a change in the primary endpoint.
While routinely changing endpoints in midstream is not appropriate for all clinical research, in this case the change reflected our coming to understand that ischemic injury to the brain is not an all or nothing phenomenon, where the light switch turns off at the onset of neurologic deficit and will not turn on again unless rapid reperfusion is accomplished. Some neurons die rapidly and irreversibly with the onset of ischemia; others stop functioning but are still recoverable.
This became clear via the interim analysis and led to not only to an appropriate change in the protocol but also to a new understanding of how to regard and manage acute ischemic brain injury.
Especially in studies that are relatively long-term, our understanding of the disease state we are attempting to treat– and the treatment we are evaluating – may change radically during the course of the study. In such cases, clinging to a protocol conceived in ignorance may be a waste of time and a detriment to patients.
Good post. The initial of the study seems problematic as I would not expect use of high flow nasal oxygen to do very much to reduce mortality, whereas it would be expected (and in this study seems to have shown) to prevent intubation - very much a patient centered outcome. I would have preferred an RCT where intubation was the primary outcome but it seems unlikely that such a study will be conducted.
Seems to me another explanation would be that there is no way the providers were blinded to the treatment.
Perhaps the HFNC were less likely to get intubated because of bias? Similar for perceived RR?
I am always suspicious of documented respiratory rate. I have never seen someone outside of nursing students actually stand there and measure a rate for an entire minute. Most of the time CNAs and RNs do little more than glance at the patient and decide if they “look like an 18” vs “look like a 20.”
In the ICU we don't sit and count respiratory rate. The monitor does it for us best on ECG changes or end tidal CO2
These "secondary" end-points are NOT "surrogate" end-points.
Great piece, important nuances, and love the call for discussion. This trial is crucial, and I believe the endpoint was correct. Needed to know if the initial findings could be replicated, that's step one. They could not—hugely important knowledge. Next, with this knowledge in hand, seems to me it's right to focus trials on the important secondary endpoints you highlight. Thanks for an excellent article.
My thoughts - not new but never well-received! There should just be a set of (coherent) endpoints from studies (including trials) not primary, secondary, etc. The results on each should be published in a way consistent with their entry into a Multi Criteria Decision Analysis, where the conclusion following from a variety of endpoint weights, reflecting the user’s preferences, can be established. Researchers and content experts may publish the results from entering their preferences, but their's have no more epistemic legitimacy than anyone else’s. Empirical research can produce a relevant ‘set of population weights, but ultimately it is the individual patient’s preferences that are required for informed consent.
I agree mortality seems to be a harsh metric here when HFNC is the “starting” point or initial modality, and not the last word in hypoxemic RESP failure. Any patient who worsened on HFNC would have progressed to the next step (ie. intubation). So I think reduction in need for intubation (without any apparent cost in excess deaths) would be sufficient to vouch for this method.
OTOH, need for intubation was indeed reduced…but the reduction in ventilator-free days was small. Roughly 35 fewer pts on HFNC needed to go on a vent….but that resulted in only 2 fewer vent days on average….does that mean the HFNC pts who ultimately needed a ventilator in turn required a ventilator for longer periods?
Fascinating point about therapeutic fashion also. Amazing that a 100-patient trial was sufficient to establish that “this is what the cool cats do”…enough so that clinicians would not randomize their pts. That’s a bit of an indictment of our profession.
The dictum that Hypoxemic respiratory failure requires ventilator supportive care remains valid…
The widespread use of HFNC occurred mostly because of demand for temporizing or bridging patients to available rooms and ventilators during the SARs-CoV2 pandemic. It’s no panacea, and perhaps, leads to delayed treatment of hypoxemic resp failure if higher levels of PEEP are required for adequate systemic oxygenation.
There was some concern as I recall the high flow oxygen increased the risk of spread of COVID amongst caregivers....was this pur to rest??
Nothing wrong with secondary endpoints as long as they are well-defined so that we can judge the degree of subjectivity involved. Much of what we do in medicine is directed toward outcomes other than plain mortality. Composite endpoints, however, often seem to be employed in order to sell a drug or procedure. A worthwhile study should include the raw numbers and percentages for each of the primary and secondary endpoints. Practically any reader can then use common sense to assess the value of the drug or procedure under investigation.
Great article review. I completely agree with you that these composite endpoints are at least as relevant as mortality.
In diseases causing profound hypoxemia (eg COVID 19), supplemental oxygen can be life saving. However, for most pulmonary disease, supplemental oxygen tides the patient over until the underlying cause of the hypoxemia can be treated and lung function improved. It’s unlikely that the primary outcome in this study was the one most likely to provide useful information. .
High flow oxygen therapy can improve ventilation, decrease the work of breathing and increase FRC, none of which regular flow oxygen can achieve, likely leading to the improvements in the secondary outcomes of this study.
Understanding what a therapy is likely to achieve, and how, leads to better choices of study outcomes.
Slavish reliance on death as the only worthwhile primary outcome may make both trialist and reviewers feel that a study is more important. But often whether a treatment helps patients live longer is the wrong question to ask.
Perhaps so. But surrogate endpoints exactly serve the Big Pharma cartel in every example I can think of, in cancer treatment, hypertension, CVD, on and on.
I am not treating patients. But I assume and believe that treating patients involves making them more comfortable.
But in the world of health research, all cause mortality is by far the most important factor and the one that Big Pharma inevitably prefers to squirm out of.
For instance, cancer treatments are sold with the promise that they postpone recurrence and lower death from cancer. That they don’t lengthen lifespan, or that they increase morbidity from side effects, is under reported.
CVD: statins are sold with the idea that they lower death from heart attacks. Their numerous bad side effects and the fact that they don’t extend lifespan is under reported.
Etc. This is just business as usual for Big Pharma.
So you bring up a narrow example that involves patient comfort and show a small but perhaps significant difference in all-cause mortality but I think this example is perhaps not particularly important. If it makes patients more comfortable to have oxygen delivered a different way, then wouldn’t it just be good medicine to deliver oxygen that way?
They used to deliver oxygen with a small amount of CO2, for many good reasons. So-called carbogen fell out of favor but perhaps it should be brought back. Patients are often over oxygenated, lowering their CO2 to dangerous levels.
Thank you John , perceptive as usual
We treat patients for two reasons : to feel better and/or live longer ( note the two do not always go in the same direction).
In very ill patients , living longer is not necessarily the most important outcome . John Spertus and his team illustrated this in an elderly cohort, who asserted ( summarizing) that freedom from disability was MUCH more important than prolongation of life ( my emphasis) .
Our primary purpose as caregivers is to relieve suffering .
Any chance of changing the title to "When Secondary Endpoints Best the Primary..."? The reason you are interested in those secondary endpoints is that they aren't surrogates -- they are clearly patient-important outcomes -- and you label them as secondary in the text.