Sensationalism Does Not Help Us Understand the Consequences of Race-Based Algorithms

Distinguishing between diseased and healthy states in medicine has immense consequences because it influences treatment, insurance coverage, employment, and disability. But this distinction is arbitrary since health exists on a continuous spectrum. The people whose diagnostic test result is a few smidgeons on the ‘healthy’ side of the line are meaningfully no different than those a few smidgeons on the ‘diseased’ side. But nonetheless a line needs to be drawn somewhere for practical implementation reasons.  But what if the arbitrarily chosen line systematically discriminates against certain groups?

That is the question that Diao et al sought to study in a recent study published in the New England Journal of Medicine. In an ambitious study compiling several datasets, Diao et al. projected the national impact of adopting a spirometry test that used a race-neutral instead of race-based equation.

Spirometry is a widely used test to see how a patient’s lungs are working to diagnose, stage the severity of, and monitor a variety of lung diseases such as emphysema (also known as chronic obstructive pulmonary disease or COPD). A patient blows into a specialized machine as hard and fast as they can. The amount and speed you can blow out air are compared against the values expected among people with healthy lungs. For more than a century, these spirometry results have been benchmarked against a healthy population of similar age, sex, and height—biologic phenomena that influence lung function—but also of the same race. Since race is more often a social construct and not a biologic one, in recent years, lung experts redeveloped the reference standards to be race-neutral by removing race from the benchmarking.

Enter the study by Diao et al. Funded by the National Institute of Health, they used five different sources of data to quantify the magnitude of clinical, occupational, and financial changes that would be expected if the US universally adopted the newer race-neutral versus the traditional race-based references for spirometry. Because reference lines are arbitrary, understanding the implications is crucial to ensure whether adopting this newer reference equation does not unduly harm one racial group over another.

The authors findings are alarming. If universal adoption occurred overnight, hundreds of thousands, if not millions, of Black Americans would be classified under the newer race-neutral equation as having more severe emphysema, greater medical impairment, more occupational disqualifications, and higher disability payments because of their newly classified lung disease. In other words, they conclude that much is at stake by where we draw the line of ‘normal’ and ‘abnormal’ spirometry.

But there is a critical error with the author’s approach. The authors relied nearly exclusively on spirometry to define eligibility criteria for disease, disease staging, occupational eligibility, and disability benefits, which grossly inflates these projections and obscures the broader public health implications.

For instance, for occupational eligibility the authors examined the impact on people’s ability to be employed as a firefighter. Only 0.1% of the US population are career firefighters. Even assuming a tenfold occupational interest, eligibility changes would affect 23,000 individuals, not their projected 2.3 million Black Americans.

To qualify for disability payments, veterans need a service-connected diagnosis beyond meeting abnormal spirometry criteria alone. For example, veterans with abnormal spirometry results would also need documented or presumed exposure to toxic inhalants during their service, such as burn pits or agent orange. By applying this more accurate eligibility determination, adopting the race-neutral equation would likely affect annual payments by thousands of dollars annually, not the billions touted in the study.

To estimate the impact on changes in emphysema (or COPD) severity classification requires far more than spirometry results alone, including the constellation of symptoms, smoking, and other inhalation exposure. However, despite the availability of these criteria in the dataset that they used, the authors did not include these other critical pieces of information in determining COPD status.  

From the data presented in the study, only roughly one-quarter of Black Americans with an abnormal spirometry result per the newer race-neutral reference had any compatible symptoms of emphysema such as breathlessness with walking or wheezing. Thus, the new equation would amount to drastically far fewer Black Americans being labeled as having moderate-to-severe emphysema. To more accurately project these impacts, the authors should have focused on only adults with a smoking history and respiratory symptoms, stratified by respiratory disease history, to quantify clinically recognized & unrecognized reclassifications.  

Refining eligibility criteria is essential to clarify these tradeoffs to ensure accurate public health planning. While I can’t speak to intentions, I can speculate as to why the authors chose these more simplistic though sensational choices. Some of the criteria to define the public health impact are simply unavailable in these datasets, such as occupational interest and veteran service-connected status. For the criteria that are available, such as symptoms and smoking history, the number of people in the dataset whom these would apply to would likely be far too small to be enable national projections. In other words, among the 30,000 individuals in the NHANES IV dataset that the authors used to estimate the change in emphysema severity, too few individuals of each racial/ethnic group would have the combination of abnormal spirometry, respiratory symptoms, and a smoking history.

What is far more concerning in my view is the decision to fund (National Institute of Health) and publish a study (New England Journal of Medicine) with these inherent limitations that preclude a sober analysis by two of our most eminent scientific institutions. While a version of this letter was rejected by the Journal, peer review will live another day through outlets like this one and the public space of social media.

---

Photo by Philippe Bachelier