Why We Question the Safety Profile of mRNA COVID-19 Vaccines

Robert M Kaplan and Sander Greenland

By Robert Kaplan and Sander Greenland

A new bivalent COVID vaccine will become available this week.  The FDA used results from mouse experiments and the original vaccine trials to reassure the public that the new boosters are safe.  Is this evidence sufficient?

Last week, along with an international group of physicians and scientists, we published a study suggesting that the risks of COVID-19 vaccines may be greater than previously reported.  Using publicly available data from Pfizer and Moderna studies, we found one serious adverse event for each 800 vaccinees.   That translates to about 1,250 serious events for each million vaccine recipients. DHHS reports the rate for other vaccines is only 1 or 2 per million.

Many physicians and scientists believe that vaccination programs are the key to ending the coronavirus pandemic.  Some warn that our analysis might harm public health by stimulating more vaccine hesitancy.   Yet, if some concerns are valid, remaining quiet could also result in harm and further erode public trust in science.

We believe that scientists have a responsibility to report suspected hazards to authorities.  Consider a 1 in 800 risk of a serious adverse reaction in the context of other vaccines.  The 1976 swine flu vaccine was withdrawn after it was associated with Guillain-Barre Syndrome at a rate of approximately 1 in 100,000. In 1999, the rotavirus vaccine Rotashield was withdrawn following reports of intussusception in about 1 or 2 in 10,000.  As widely acknowledged, COVID vaccines prevent hospitalizations, and the clinical trials estimated that between 225 and 625 hospitalizations were prevented per million vaccinated persons. But these benefits are likely to be concentrated among vaccinees who are elderly or have chronic illnesses. It is less clear which groups are at risk for serious adverse vaccine reactions. Those at low risk for hospitalization may still be at risk of serious vaccine reactions. We only considered mRNA vaccines and it is not clear that other COVID-19 vaccines confer the same risk. 

We recognized that our results might be incorrect. So, we reported our preliminary findings to both the US FDA and the European Medicines Agency. To their credit, leaders in these agencies took our study seriously.  They met with us and provided feedback that resulted in revisions in our analysis.  Next, we encouraged both regulatory agencies to replicate our work. The published article and a previous preprint include links to our data and information that allow others to repeat the steps that lead to our conclusions. The preprint has been downloaded over 110,000 times, which places it among the most highly accessed papers in 2022.   

For sure, many colleagues pushed back against our report.   Some asked why we concentrated on the randomized clinical trials when hundreds of millions of people have now received vaccines.  For many physicians, randomized trials remain the gold standard for evaluating pharmaceutical products.   Vioxx and postmenopausal hormone therapy are two examples of widely used drugs that were only accepted as harmful after large clinical trials examining harms were performed.    It is difficult to interpret observational studies of vaccines because those who elect to receive vaccines likely differ from refusers on many dimensions, including obesity, diabetes, medication compliance, and propensity to use healthcare.  We thus concentrated on randomized trials because they remain the only widely accepted method of controlling for these extraneous influences.

The published article incorporates responses to those that could be addressed with the available data.  In developing the analysis, we attempted to follow the template used for teaching research methods to advanced students. One critic incorrectly argued that we looked at numerous adverse events and selectively reported only those that produced statistically significant differences. This process is called “P-hacking” or “cherry-picking”.  We did neither.    To avoid these problems, we focused only on potential side effects that had been prospectively identified by an international collaboration associated with the World Health Organization. Each serious adverse event was examined independently by two clinicians, and a third adjudicated differences of opinion. Results were then presented with due caution about the limitations of the data.   Full Fact, a fact checking organization, collected a series of challenges to our analysis and we have made public our responses to each.   

Regrettably, our analysis was hindered by an addressable problem:  The individual level data that could confirm or refute our analysis have not been made public.    For example, we would have greater confidence in our conclusions if we knew how often individuals experienced multiple serious adverse events. Pfizer, Moderna and the FDA have these data, but have kept them hidden from public view. This information is essential to the understanding of the balance between vaccine benefits and harms. We are calling upon Pfizer, Moderna and the FDA to release all information needed for a comprehensive assessment of these products.

COVID-19 vaccines are now among the most widely disseminated medicines in the history of the world. They have cost taxpayers tens of billions of dollars, rivaling the annual US federal expenditure on biomedical research.  There is no legitimate reason why scientists and the public should not have access to the evidence that justified that purchase. Yet evidence is being withheld, which adds uncertainty to our conclusions and leaves lingering questions about the scientific foundation for COVID-19 vaccine promotion.  Public posting of raw data is a reasonable response: “Open data” is becoming the norm in science and is now required by many leading journals.   The time has come for the FDA and EMA to reopen their investigations, and for Pfizer, Moderna and all vaccine manufacturers to provide the data that will allow scientists and physicians to address outstanding concerns.

Robert M Kaplan is an Emeritus Distinguished Professor at the UCLA Fielding School of Public Health and an Adjunct Professor of Medicine at Stanford University's Clinical Excellence Research Center,  He is an elected member of the National Academy of Medicine.

Sander Greenland is Emeritus Professor of Epidemiology and Statistics at UCLA, and Fellow of the American Statistical Association and the Royal Statistical Society.