Errors in Science: Self-Correcting, or Self-Propagating?

A Sensible Medicine Guest Post

History and folklore are filled with “cumulative tales” that involve minor events escalating via assumptions, eventually leading to a final dramatic outcome. One thinks of the story of Chicken Little, in which an acorn hits her head, leading her to assume that “the sky is falling,” and ultimately resulting in a fox eating her entire family.

As scientists, we might imagine that we are not the target audience of such tales. But it sometimes seems like an entire area of scientific inquiry can go “off the rails” and stay off the rails for a long time, sometimes even arriving at an incorrect “consensus.”

We recently published an article proposing that there are different categories of errors in science, and that some categories are more likely to lead a field of study down the wrong path. We are grateful to Sensible Medicine for providing us with the opportunity to discuss our idea.

Science as “Self-Correcting”

People often describe science as being self-correcting, meaning that, over time, a preponderance of studies will make increasingly true claims about a given area of research. This phenomenon is similar to the law of large numbers. We expect that this convergence will mitigate or nullify the effects of errors made in individual studies.¹ However, it is essential to recognize that self-correction is not automatic and often requires effort – people must actively work to correct the record.

Some Errors are Just “Random Errors”

Some errors – basic errors – are fundamentally random. These include honest mistakes, like transposing two digits when filling out a table or making a programming error.

Early work in probability shows that random errors over the same dimension (the “random walk”) mostly, but not completely, cancel each other – eventually. We illustrate this elsewhere. Yet, scientific inquiry is more complex than walking, which reduces the chances that basic errors will cancel each other out. So, even if all scientific errors were basic errors, it would be important to pursue scientific self-correction actively.

Further, the discourse around scientific self-correction seems based on the assumption that most errors are basic errors: effectively random and equally likely to deviate from the truth in any direction. We propose that this is not always true.

Other Errors are More Likely to Lead to Additional Errors

There are at least two types of errors that we argue are more likely to lead to subsequent errors.

1. In some cases, an error made in one study directly increases the chances of errors occurring in subsequent studies in the same field. We call these “Non-Markovian Errors.” We borrow the term from physics, where it is used to describe how memory of previous states can influence subsequent states. This can happen in multiple ways. For example, a research team may successfully publish a paper using an incorrect method, and other teams may then adopt that same approach simply because the first team “got away with it.”

2. In other cases, fields of inquiry can be affected by a “prevailing wind” that constantly blows from the same direction. We call these “Directional Errors.” One example (of many) is “domain-specific social-emotional bias”: Sometimes, a field of inquiry can proceed in a specific direction because of assumptions that the topic of study is either generally good (e.g., white hat bias) or generally bad. In such cases, the inability to consider the nuances of a topic (e.g., it could be good for some purposes, and bad for others) can influence the very question(s) being studied, or their framing, such as through a halo effect.

The Improbable Case of Gardening and Obesity

In our paper, we illustrate these concepts by examining the idea that gardening can treat or prevent obesity. Although space limits in this post prohibit full exposition, we and others have shown that expecting large effects of gardening on obesity was never reasonable. We further contend that there is not a single study showing a causal effect of gardening on either prevention or treatment of obesity, and at least one study shows no statistically significant effect on any obesity measure. Nevertheless, there is a body of literature that treats the notion of a causal relationship between gardening and obesity as having a high degree of credibility.

One recent evidence synthesis from 2023 concluded, “…we investigated… school gardening… Types of positive outcomes included …improved BMI…” However, the literature cited to support that claim variously did not measure BMI, did not find changes in BMI score, did not analyze gardening as a primary intervention, or contained other errors. Then, that synthesis was cited in a 2025 review of gardening (with a focus on cancer prevention) as showing “improvements in anthropomorphic outcomes after gardening initiatives.” Evidence provided to support that claim also included a paper (which was already part of the 2023 synthesis) that found that “changes in cooking and gardening behaviors were not associated with changes in BMI z-score or waist circumference.”

While this science may (and likely will) be corrected over time, the specific types of errors made in the gardening and obesity literature appear, in the short term, to be cultivating additional errors.

How Can We (Scientists) Do Better?

• Avoid assumptions and errors, particularly those that lead to “cumulative tales” about a field of study. We need to make it easier for scientists to both admit to and correct honest mistakes without fear of reprisal or major adverse consequences to their careers.

• Hold ourselves to the highest standards possible, which includes acknowledging that we can fall prey to our own preconceptions (“…you must not fool yourself, and you are the easiest person to fool”).

• Further increase public access to raw data, code, and analytic plans to enable scholars to check each other’s work.

• Take statistical expertise (particularly professional statisticians) more seriously.

• Reconceptualize established systems for producing science. For instance, journal editing requires expertise, bravery, and resources, but many editors are part-time volunteers with no formal training and encounter scientifically and even ethically complex questions on a regular basis.

Where Do We Go Next?

Our concern that studies in a specific field may have gone astray does not diminish our enthusiasm for science. Science is a systematic process that attempts to determine what is true. One of the best things about science is its self-critical nature and its ongoing struggle for greater rigor. Such work continues a centuries-long tradition.

In the United States, the separate but related topics of trust and trustworthiness in science are as–or more–important to discuss now than ever. For science to be taken seriously by the public and to be afforded credibility, it is critical that we continually take steps to improve scientific rigor. To be trusted, we must show that we are trustworthy.

In fact, we believe, but cannot prove, that science has gotten progressively more rigorous and trustworthy over time. Yet, as argued elsewhere, comparatively small numbers of scientists (including individuals, groups, or even specific fields of study) can drive negative public narratives about science either innocently (e.g., resisting error correction) or not (e.g., deliberate misconduct).

While some changes to science are assuredly needed, what appears lost in the current discourse is that most scientists continue to be dedicated to the transparent, objective pursuit of truth.

Jon Agley is an associate professor in the Department of Applied Health Science at Indiana University Bloomington in Bloomington, Indiana.

Sarah Deemer is an assistant professor in the Department of Kinesiology, Health Promotion, and Recreation at the University of North Texas in Denton, Texas.

David Allison is a professor of pediatrics, endowed chair, and director of the USDA Children’s Nutrition Research Center at Baylor College of Medicine in Houston, Texas

1. To be clear about terminology: When we write the word “consensus,” we do not mean an agreement among scientists, as the term is sometimes used, but rather a situation where multiple rigorous studies independently indicate that the same thing is likely to be true. When we refer to “scientific claims” we specifically mean statements about “what is likely true of reality” and not to other statements sometimes made by scientists about “what people should do” (i.e., normative statements; see discussion of this distinction here and here). Finally, when we refer to “errors” we are referring to “clearly incorrect procedures, demonstrably incorrect reporting of findings, or probably illogical conclusions or invalid reasoning,” and not to other uses of the word that may occur in scientific discussion.

Photo Credit: Ben Moreland