5 Times You Didn’t Know You Had Participated in a Randomised Trial – Andrew Leigh

Andrew Leigh offers five times you (maybe) didn’t know that you had participated in a randomised trial. Experiments have consistently been used in the hard sciences, but in recent decades social scientists have adopted the practice. Randomised trials have been used to design policies to increase educational attainment, lower crime rates, elevate employment rates, and improve living standards among the poor. Below, Leigh shares common randomised trials that you may have taken part in without realising.

randomised trial

Andrew Leigh –

In 1747, a young naval surgeon by the name of James Lind carried out an experiment to find a cure for scurvy. He took a dozen sailors who had been afflicted by the disease, and randomly assigned them to one of the proposed cures. Sailors given vinegar, sea water or sulphuric acid languished. Men who got oranges and lemons were back on duty within a week. Eventually, the British navy began giving lemon juice to all its sailors. One reason Admiral Lord Nelson won the Battle of Trafalgar is that he was fighting French and Spanish ships that were stricken with scurvy.

Today, most new drugs have been tested in randomised trials, in which patients are assigned the treatment or a placebo based on the toss of a coin. By uncovering what works best, randomised trials have saved hundreds of thousands of lives. But it isn’t just medicine. In business, A/B tests are ubiquitous, while smart governments are constantly tweaking policies through randomised experiments.

Do you imagine you’ve never been touched by randomised trials? Think again. In fact, you’ve probably been part of a randomised trial if:

  1. You’ve used the internet. The shade of blue on the Google toolbar is the result of a randomised trial, which tested 40 different shades of blue. Quora, a question-and-answer web­site, devotes a tenth of its staff to running randomised trials, and is conducting about thirty experiments at any given time. Netflix uses experiments to improve the algorithms that decide which movie to show you next. The company estimates that such experiments save it over US$1 billion annually.


  1. You’ve shopped in a store. Wondering why half of all consumer prices end in nine? Blame the use of randomised marketing trials, which find that customer demand is higher for prices like £9 or £9.99. Retailers are constantly using randomised experiments to test the effect of shelf placement, promotions and even store opening hours.


  1. You’ve gotten a fine. Britain’s Behavioural Insights Team, led by David Halpern, has been a pioneer globally in the use of randomised trials. One experiment showed that people were more likely to pay their car tax if the letter included a photograph of the offending vehicle, along with the caption ‘Pay Your Tax or Lose Your Car’. Income taxpayers who were informed that most people pay their debts on time were 5 percentage points more likely to cough up – an impact that represents millions of pounds of additional revenue.


  1. You’ve donated to charity. Randomised experiments run by charities have shown that they can increase their takings by offering a prize or announcing that donations will be matched. When it comes to door-to-door canvassing, men are more likely to donate to an attractive woman (perhaps the least surprising finding to emerge from such studies). An international aid organisation found that request letters which included a modest gift of postcards drawn by Bangladeshi street children nearly doubled their donations.


  1. You’ve gone to school. Britain’s Education Endowment Foundation has tested hundreds of interventions, assigning them to pupils based only on the toss of a coin. Some sound promising – such as the Saturday reading program Chatterbooks – but produce no measurable results. Even among interventions that work, there are massive differences in cost-effectiveness. For example, a personal academic coaching program cost thirty times as much as a philosophy-based intervention, just to get the same gains.

Beyond this, randomised trials have improved farm productivity and road safety. They have given us new insights about managing diverse workplaces and sticking to our exercise plans. Once you know where to look, you’ll find the randomistas all around us.

Randomised TrialAndrew Leigh is the author of Randomistas: How Radical Researchers Are Changing Our World.

Share this

You must be logged in to post a comment