I’ve been avoiding writing about the latest data and news about the Galleri test. I’ve already posted too many articles about screening in general and Galleri in particular. More importantly, I feel like I know how the story will end, and it makes me sad: Medicare will approve and pay for an expensive test that promises little to no benefit. The leadership of Grail, the company that makes the Galleri test, and investors will become wealthier, while those who pay into Medicare (i.e., all of us) will be poorer, spending more on healthcare without getting healthier.
After a few weeks, though, maybe there is something to add: a discussion of the data we have, where it falls short, and the data that we need.
The study that would prove that the Galleri test is useful is the same study that we should demand for every screening test. People who are at risk for the disease (or diseases) in question are randomized to screening or no screening. The patients are then followed for a prespecified number of years, and then overall mortality for the two groups is compared. If the screened patients die less — enough less that the lives saved are worth the cost — then we adopt the screening test. If the mortality rates are the same in the two groups, we don’t. For Galleri, a “multicancer screening test”, Mandrola and I would not even need to argue about overall vs. cancer-specific mortality.
Although the study we need is simple, there are at least five reasons that it is hard to run.
1. When we are looking at a population-based cancer screening test. This means that the incidence of cancer will be low, so the study will need to be somewhere between large and enormous.
2. We will need to design this study as an “intention to screen” analysis because not everyone to whom we offer the test will follow through with screening. This means the study will need to be closer enormous than to large.
3. Because the lag time between a test finding an early cancer (or precancerous lesion) and a person dying from that disease is long, the large study must run for years.
4. We will be enrolling healthy people in our study so the risk that we harm more people than we help will be high. We will need to delineate the sensitivity and specificity of the test, and quantify the harms of overtesting, overdiagnosis, and overtreatment.
5. Lastly, because our treatment for cancer is always improving, and screening is less important for treatable cancers, our study will have to be occasionally re-run because evidence should come with an expiration date.
This study design is key because we often forget that the goal of screening is not to find more disease or to find it earlier; it is to decrease morbidity and/or mortality from the screened disease. The design described above avoids the biases associated with finding disease early without improving outcomes: lead time bias, length time bias, and overdiagnosis.
Lead time bias occurs because people with disease detected by screening will have longer survival “with the disease” than those whose disease came to attention when they developed symptoms. This is true even when screening does not decrease mortality. Finding a disease earlier but not changing the outcome is harmful.
Length time bias occurs because screening tests detect more indolent subtypes of cancer, cancer that is present in a population for long periods of time. Thus, people whose cancer is detected by screening do better/live longer, on average, than people whose cancer is detected when they become symptomatic. People don’t benefit from this detecting the “less bad” subsets of disease.
Lastly, screening tests risk overdiagnosis. Screening seems effective because we detect tumors that would never have become clinically apparent. This Bleyer and Welch NEJM article from 2012 does an excellent job explaining overdiagnosis as it pertains to breast cancer screening.
Let’s get back to Galleri.
Grail’s Galleri test is a single blood test that screens for multiple cancers by assaying for cell-free DNA, or other circulating substances, shed by tumors. Early studies focused on the test's characteristics and were disappointing. When specificity was set near 100% (to minimize false positives), sensitivity only exceeded 50% for later stage (stage 3) cancers. (See figure 3 here). This means the test is mostly good at detecting cancer when it is clinically detectable and/or when we are not very good at treating it.
The latest study looks at the Galleri test's ability to find cancer and shift the stage of diagnosed cancer, over a short period of time, from high to low. In this 3-year, 140,000-person study, Galleri failed. The primary endpoint was a shift in the incidence of stage III or IV cancers between the screened and non-screened groups. There was no difference: incidence rate ratio, 1.03; 0.92 to 1.14, P=0.63.
So as not to come off as a total buzz-kill, there are some itzy-bitty reasons to be optimistic. If you look at the curves, you could almost imagine that, years into the future, the incidence of stage III and stage IV cancers will fall in the screened group compared to the not-screened group.
BUT! Remember that this has not yet happened in a trial of 140,000 people over 3 years. Any benefit we eventually see, if we see one, is likely to be small and take a long time to accrue. And even if it does, that still does not mean this test saves lives. There are plenty of reasons a screening test that slightly decreases the number of stage 3 and 4 cancers in a trial population does not save lives at a population level, let alone does so at a remotely cost-effective rate.
There will be anecdotes, true anecdotes, of people whose lives might have been saved by Galleri; people whose pancreatic cancer was caught early by the test and survived.
The real question is, does the test save lives and, if it does, what are the costs, financial and otherwise?1 Eventually, we will need to ask these questions unless we want to keep turning over public dollars to private companies and get nothing in return, much like we did with Hoffmann-La Roche/Genentech/Gilead for Tamiflu or with Pfizer for Paxlovid.
After I finished this article, the authors of this interesting JAMA article — Cancer Diagnostic Delay Rates Associated With a Population-Based Screening Trial Evaluating a Cell-Free DNA Multicancer Early Detection Test — reached out to John. It speaks to the other kinds of costs of tests like this and what it does to the rest of our healthcare system.
Well said, thank you. Smart, helpful. (And that footnote is amaaazing).