In Defense of the Randomized Trial

It took millennia to develop, yet we still can't embrace it

Most histories of the randomized controlled trial include James Lind and the MRC studies of the 1940s, and many debate the precise instance when a study was both randomized and controlled. I won’t rehash that discussion but will broadly say: nearly all scholars agree that randomized trials were devised in 20th century with some early attempts in the 19th, and most importantly, all agree, we didn’t run RCTs in 200 BC.

But, why not?

In order to make airplanes, you need light, strong and durable materials, like aluminum and other metal alloys. We had to extract, refine, mold and shape metal before we could have the Boeing 767. For technologies that build upon others, it is logical they did not occur sooner, but this doesn’t apply to RCTs.

You need nothing to randomize. A group of people 10,000 years ago could have divided land into 100 quadrants, and randomized it to two strains of a crop and measured which one produced a greater yield. 4,000 years ago, we could have randomized 1000 people suffering from leprocy to 4 different medicinal tinctures. Europeans could have randomized people into drinking one of two beers for a week, and found out which caused worse hangovers. And even the Romans could have randomized people to lead vs other pipes and studied their mental health 10 years later.

Yet, none of them did.

Fast forward to the present day. Despite nearly a century of randomization constantly upending conventional wisdom, continually giving us insights, there remains a large group of people (some call themselves scientists) who remain opposed.

“You can’t study whether masks slow COVID19 or RSV’s spread in kids because we can’t randomize kids.”

“We can’t randomize people to bivalent boosters and measure rates of severe disease— to see if they truly have benefit”

“We can’t do a randomized trial of teclistimab in relapsed disease or any of smoldering myeloma”

“We can’t randomized people to free COVID tests or not”

“We can’t randomized health care workers to mask indefinitely or not”

Of course, all of these claims are lies— they are false, pure and simple. Worse they are harmful. They prevent us from knowledge.

You could have run many cluster RCTs of different masking strategies. You could easily demand RCTs measuring mortality for new cancer drugs (we proved that they would even result faster than the current uncontrolled trials), and you could run human trials of bivalent boosters measuring severe disease. In fact, Peter Marks of FDA just wrote in JAMA last week that he may ask this in the future. Of course, you could randomize a pre-cancer (like smoldering myeloma) to treatment or observation and measure survival.

More than could— we should do these trials, we ought to do them, and we must do them. It is scientifically and morally wrong to push policies year after year with no credible data they help. Only randomization solves all the problems at once. Time zero, confounding, selection bias and more. It is the single most powerful tool to separate wishful thinking from a real signal.

Some people use smoking as an example for why RCTs are not needed. Others use the parachute as an analogy. To keep this commentary short: we have debunked those arguments in the peer-reviewed literature here.

Yet, on all these issues- we are a just like the people of 200BC. They didn’t do trials of masking kids in years of the plague, and neither did we.

It is impossible to know why a simple idea like randomization took so long to develop, and is still stuck on the ground, but my best guess is that it fundamentally does not appeal to our storytelling brain.

In order to embrace randomization, you have to understand that a lot of things in life are random, and have random variation. Many patterns you observe, or believe in, is just telling a story out of random variation. That’s hard for brains that work based on story telling. Of course it “just makes sense” that masks work, and it once “just made sense” that flecainide post MI must save lives. The trouble with “makes sense” medicine is often, it doesn’t work. Maybe someone should write a book about that.

Thousands of years have passed from the pyramids of Egypt to the randomized trial, and yet, the ancients so easily could have devised this method. They had all the tools for it at their disposal. The fact that they didn’t, and that experts today still don’t get it is only proof that we still have miles to go. I look forward to the future society that fully embraces the power of randomization. I suspect it will be long after my lifetime.