Should evidence come with an expiration date?
An empirical analysis
Sensible Medicine readers are probably familiar with the idea that medical evidence can expire. Back in 2019, Dr. Cifu and Dr. Prasad, together with Dr. Greene, published an opinion paper titled “Should evidence come with an expiration date?”. On Sensible Medicine, the same concept keeps turning up.
E.g 1 - Paxlovid
Example 2 -
Here I would like to present the findings of an empirical analysis Vinay Prasad and I conducted on the idea of evidence expiration.
In our paper – titled Quantifying the Age of Evidence: Lessons From Cardiovascular Drugs – we sought to develop a method to measure the age of evidence for medical interventions. In order to do it, we used 2 widely used drugs as examplars – statins and sodium-glucose co-transporter protein 2 inhibitors (iSGLT-2).
Some background
Patient characteristics and clinical contexts change over time. Thus, even strong evidence favouring a certain medical intervention can become outdated. Therapies that were once supported by robust evidence may later prove less effective or not effective at all.
For example, aspirin was once recommended for cardiovascular disease (CVD) primary prevention. However, when new studies were conducted, the recommendation was revoked because aspiring was more likely harmful than beneficial. The decline in efficacy might be explained partly by healthier lifestyles and improved risk-factor control among patients enrolled in more recent trials compared with those in older trials.
So, when one aims at practising evidence-based medicine, it is important to consider the age of evidence. In principle, the older the evidence the less likely it is to reflect current patient populations, and therefore less likely to remain applicable to present-day patients. While time itself should not be judged alone – for instance, RCT evidence supporting the use of steroids in severe COVID is only 5 years old, yet it is arguably outdated –, the accuracy of the “age of evidence” as a proxy for outdatedness seems to improve as time increases.
Our study
We performed a cross‐sectional analysis of all Cochrane reviews evaluating the effects of statins and SGLT-2 inhibitors on CVD management up to August 2024. We summarised the evidence and calculated the lag in time between patient enrolment in trials and August 2024.
Reviews were included if they focused on statin/ iSGLT-2 therapy in adults, or were broader in scope but included meta-analyses for statins/iSGLT-2 alone. The primary outcomes had to include CV mortality, CV events or death from a major CV event.
First, we calculated the age of evidence for each trial as the time between the midpoint of the enrolment period and August 1, 2024. Then, the age of evidence was summarised for statins and iSGLT-2 as drug classes using the median age across all trials.
Results
Nine reviews of statins and two reviews of iSGLT-2 were included. Table 1 shows a list of all included reviews, their topics and the number of trials included in each review.
The 9 reviews of statins include a total of 153 different trials with enrolment periods spanning 29 years (between 1988 and 2017). The figure below illustrates the enrolment periods for individual trials, along with the overall enrolment period for each review. For context, the declining annual CVD death rates in the US during the same time period are also displayed. Study periods are unspecified or unavailable for 56% of trials.
In turn, the 2 reviews of iSGLT-2 include a total of 67 different trials spanning 13 years (patients enrolled between 2008 and 2021). Study periods are unspecified or unavailable for 19% of trials.
In our paper, we also provide a table summarising both drugs’ effect estimates on all-cause mortality and CV outcomes. Among the statins review topics, evidence was most robust for statins in the context of primary CVD revention and chronic kidney disease (CKD) not requiring dialysis. iSGLT-2 drugs have also demonstrated significant benefits in CVD and patients with CKD and diabetes.
Finally, the overall age of evidence for statins is 24.1 years and 8.6 years for iSGLT-2.
Implications of our findings
First, we developed a method to quantify the age of medical evidence and a graphical way to convey it. This way, one can more objectively and effectively evaluate the potential outdatedness of medical evidence.
Second, the fact that the age of evidence supporting statin use is 24 years old suggests that this evidence may be outdated. Over the last few decades (after most statins trials were conducted), the patient population has changed considerably. Risk factor management has improved significantly and newer drugs have become part of the CV therapeutic armamentarium (e.g. SGLT-2 inhibitors, GLP-1 agonists). High BMI and obesity have become the leading behavioural risk factors for disability in the western world, whereas a few decades ago other risk factors, like smoking, were preponderant. In contrast, the evidence supporting SGLT-2 inhibitors is far more likely to apply to present-day patients.
Third, our findings are relevant because statins are among the most widely prescribed drugs. Between 2019 and 2021, atorvastatin was the most prescribed drug in the UK and the second most in the US between 2017 and 2019.
Given the simplicity of our method, we propose that all future Cochrane reviews report the age of evidence not only numerically but also visually. We believe this can be readily implemented, as it does not require any additional data collection.
Ultimately, we believe these tools can be highly valuable not only for clinicians, but also for researchers and policymakers committed to evidence-based medicine.
 | A guest post by
|
I would just settle for evidence. Just provide the raw data of every study. Such a low bar but here we are.
As a biased observer (retired radiologist), I have been saying the physical exam to be outdated in many contexts. When the famous diagnosticians of the late 19th and early to mid 20th centuries wrote their textbooks, people didn't come to hospitals or doctors until really sick, so classic signs and symptoms were often present. But, in the office now, the pre-test probabilities are so much lower that hoofbeats are very faint. Thus, one has to decide between doing nothing, versus really specific tests, as wishy-washy ones like the PE don't change that pre-test probability enough to be useful. OTOH, I am sure there is value in laying on of the hands and getting to know a patient with whom one hopes to have a long term relationship, less directly related to Bayesian calculations.