I acknowledge the Wurundjeri people of the Kulin nation, the traditional owners of these lands, and pay respects to all First Nations people present.
I am chuffed to speak to such a distinguished group of practitioners and scholars in policing, from Australia and around the world. Special thanks to those who have travelled internationally to be here. You are integral to creating, developing, and implementing evidence‑based approaches to policing
My thanks to David Cowan for the invitation to speak, and for the work he has been doing here in Australia to spearhead evidence‑based policing, as Superintendent in charge of the Organised Crime Division by day, and President of the Australia and New Zealand Society of Evidence Based Policing by night.
David Cowan – like so many of you in this room – is a randomista. Not only is he seeking to run experiments, but to build support for long‑term evidence‑based policing in Australia, and around the world.
In preparing this speech, I set out to find the first example of a randomised trial of policing practices. I had hoped that Hammurabi or Confucius, Robert Peel or Jeremy Bentham might have carried out such a randomised trial. But if they did, I was unable to locate it. Within the broader discipline of criminology, randomised trials date back at least to Richard Cabot’s 1935 delinquency prevention experiment, known as the Cambridge‑Somerville Youth Study. But if we focus specifically on policing, it is not until the post‑war era that randomised methods appear to have been used.
The year was 1968. The pioneer was Howard Earle. Aged 39, Earle was the Assistant Sheriff of Los Angeles County, responsible for the largest police training program in the United States. And his goal was to improve it.
For centuries, police training had followed rigid, authoritarian methods. The thinking was that police officers represent authority, so many police training programs took a similar approach to army training. During the 1960s, this was how police recruits for Los Angeles County were trained. The rationale was that a high‑stress training program meant that when the officers encountered stress on the street, they would not panic.
Yet social attitudes were changing in 1960s America, and Assistant Sheriff Howard Earle wanted to try something different. So he persuaded his senior officer, Sheriff Peter Pitchess to let him conduct a randomised trial.
At the start of 16 week recruit training, the academy took 74 candidates and divided them into pairs, matched by age, marital status, race, education, and whether they had had prior police or military experience. One member of each pair was randomly chosen to undergo stressful training, and the other to undergo non‑stressful training.
Classwork for both groups was the same, but as soon as the lecture ended, they were treated very differently. The stress half were subject to what Earle described as ‘a strict military atmosphere, with double‑time marching between classes, the philosophy that no matter how hard they tried they could never do it right, and extra punishments’. For the non‑stress group, he said, ‘we tried to create a college‑campus atmosphere’.
As they rolled out the program, supervisors had little doubt what it would produce. As Earle noted, ‘we had expected the stress group to do better’. Yet as they watched the 2 groups on the job, it became clear that they were wrong.
Police officers trained in a non‑stressful atmosphere outperformed the stressed in just about every way. Their job knowledge was better. They were more accurate with their weapons. More responsible. More adaptable. Better liked by their peers and superiors. They even wore their uniforms better.
Just to be sure, Earle ran the experiment a second time, with another 100 recruits, and got the same results. He published his results in a 1972 book, Police Recruit Training: Stress vs. Non‑Stress; A Revolution in Law Enforcement Career Programs. The results changed the way that Los Angeles County conducted police training. The result was not perfection – this is after all the police force involved in the Rodney King incident 2 decades later – but the experiment showed that police recruits who went through non‑stressful training were one‑third as likely to be subject to disciplinary actions as those who went through stressful training.
Randomised policing experiments became increasingly common in the 1980s. In 1981, Minneapolis ran the Minneapolis Domestic Violence Experiment, which tested different police responses to domestic violence incidents. Offenders were randomly assigned to either arrest, separation from the victim for 8 hours, or being given a warning. The study found that arresting the offender led to lower rates of reoffending compared to the other 2 approaches. The Minneapolis Domestic Violence Experiment not only changed policing practices towards favouring arrest; it also created publicity that boosted public support for randomised experiments.
In the coming years, the US National Institute of Justice funded a series of replication experiments in other US cities, including Milwaukee, Charlotte, Colorado Springs, Miami and Omaha. Many of these follow‑up studies had less powerful findings than the original Minneapolis experiment. It led researchers to a more nuanced conclusion: offenders who have a lot to lose (such as those who are married and employed) tended to get better after arrest, while those with a lower stake in conforming to society (such as socially disadvantaged men) sometimes got worse after arrest.
The finding demonstrates a common result in empirical analysis – an initially large finding is worth testing to be sure it holds up in other contexts. Follow‑ups provide empirical rigour, and can help us better understand the causal pathways.
Certain kinds of policing practices depend on being unpredictable. If offenders know for sure when a police patrol will pass by, they will be more likely to commit crimes without being caught. The need for unpredictability creates an opportunity for learning: if a certain amount of randomness helps catch criminals, then why not also use that randomness to refine policing practices?
This helps explain the willingness of police to work with researchers on a series of experiments to study ‘hot spots’ policing. Working in Oakland, Kansas City and Minneapolis, police randomised their approach to high‑crime areas. One randomised trial found that increased police patrolling in hot spots caused an 11 per cent drop in crime, while another found that police raids on crack houses led to a 9 per cent drop in crime.
Another study looked at what happened when police worked with city authorities to help control civil disorder, through measures such as getting landlords to clean up properties being used for crime, enforcing municipal regulations and posting ‘no trespassing’ signs. The result was a massive drop in drug sales.
In recent years, randomised trials have been used to test the impact on police behaviour of body worn cameras. In one study, 2224 police officers in Washington DC were randomised into one group that wore cameras, and another that did not. The study found that the cameras produced only small and statistically insignificant effects on police use of force and civilian complaints, as well as other policing activities and judicial outcomes. Other studies tend to reach similar conclusions.
There are certainly benefits of body worn cameras, such as providing better evidence in court. But the findings are a useful counterpoint to those who have argued that body worn cameras will transform the behaviour of those who wear them.
In Australia, a growing number of researchers are working with police to use randomised trials to improve public safety. The Queensland Community Engagement Trial tested the effect of procedural justice training on citizen views of the police. Traffic police were randomly assigned to a status quo control group or a treatment group in which they were taught to use a script shaped by the principles of procedural justice and community engagement when speaking with drivers at roadside random breath testing stops. The study, led by the University of Queensland’s Lorraine Mazerolle, found that the treatment improved drivers’ views of the police. Those stopped by officers with the procedural justice training had more trust and confidence in police (though it should also be noted that they were no more likely to obey officers’ directions). Former Deputy Commissioner Peter Martin (now Professor Peter Martin), is with us tonight and I congratulate him on his work with Lorraine Mazerolle on designing, implementing and evaluating this important randomised trial.
Here in Victoria, David Cowan is conducting a spate of randomised trials. Since he will speak shortly, I won’t attempt to summarise his work, but I’d like to mention one study that fascinates me. This is a randomised trial that he is running which seeks to reduce the number of people who fail to appear in court. When someone doesn’t show for a hearing, a warrant must be issued, which is a costly and time‑consuming process. The randomised experiment aims to see whether providing simpler information to accused people raise the odds that they show up in court.
As David Cowan notes, this approach is well founded in behavioural science, and trials in New York and New Zealand have shown promising results. The Victorian study replaces a 2200 word, 7 page document with a 60 word, 1 page document, accompanied by QR codes that link to support service from Victoria Legal Aid and the Victorian Aboriginal Legal Service.
The experiment, with a sample size of 60,000 accused people, tests the shorter document against its predecessor and includes 3 variants: a positive message, a negative message, and one without QR codes. David Cowan tells me that initial results are showing that the shorter summons document leads to better court attendance, with a commensurate reduction in arrests and incarceration, particularly for First Nations offenders. If evidence‑based policing can help reduce Indigenous incarceration by increasing court attendance, it will have had a valuable social benefit to the community. But the real point to make is that if this wasn’t tested, we would have no idea whether it worked or not.
To date, many of the randomised trials in policing have depended on the passion, curiosity and enthusiasm of key researchers. Howard Earle, Larry Sherman, Lorraine Mazerolle and David Cowan and many of you in the room have been instrumental in advancing evidence‑based policing. But the biggest gains will come from institutionalising this approach. Just as medicine has established structures that require randomised evaluations of new pharmaceuticals and encourage randomised evaluations of new treatments, so too policing can benefit from similar evidence‑based structures.
One such approach can be found in the United Kingdom, where a network of What Works Centres operate across policy areas. One of these is the What Works Centre for Crime Reduction, established in 2013, within the College of Policing.
The Centre supports police practitioners to interpret and adopt evidence in policing. They do this by overseeing the development of evidence maps, creating ‘a standard system to rate and rank interventions in terms of their effectiveness and cost savings’.
The What Works Centre for Crime Reduction publishes easily accessible evidence reviews, on issues such as knife crime. They have also created an online Crime Reduction Toolkit.
The UK What Works Centre for Crime Reduction works closely with police on evidence‑based standards, national policies, and learning. This has included introducing an evidence‑based policing module as a requirement for people to achieve the rank of Chief Officer.
Some have argued that a similar What Works Centre for Crime Reduction might be valuable in the Australian context. We certainly have the expertise. Established in Brisbane in April 2013, the Australia and New Zealand Society of Evidence Based Policing has built a powerful movement to improve the use of science and evidence in policing.
As David Cowan has noted to me, the application of science to policing is critically important because randomised trials tell us what works. As he notes, randomised trials are not an abstract academic concept, they are practical, do‑able and real – as the examples I have outlined clearly demonstrate. Police in their specialist fields have an incredible capacity to innovate and lead trials to test what works, and ask important questions that need to be answered.
Operating on the frontlines of policing, randomised trials expand our understanding of what works, and what does not. Good evidence sticks and has the power to change policy. Better policing can save money, and save lives. Senior policing leaders need not be experts in randomised trials. They simply need to ask the question ‘Does it work? Have we tested it? How do we know it works?’ That alone helps create an authorising environment for evidence to prosper.
From a Commonwealth level, our government is committed to evidence, and creating institutions which support the development of evidence in policymaking.
Last year, the Australian Government created the Australian Centre for Evaluation in the Treasury as a central source of cross‑sector expertise on evidence and evaluation.
The team now comprises a dozen full time staff who support government to scope and conduct successful evaluations and support capability building across the Australian Public Service.
They provide support to government departments undertaking robust, effective and ethical evaluations, providing guidance and expertise on best practice, and helping practitioners to navigate risks and uncertainties. And they work across the public service to support and build evaluation capability across public service Departments.
Since the Australian Centre for Evaluation’s Managing Director, Eleanor Williams, will speak next, I will restrain myself from waxing lyrical about ACE’s exciting work.
The shift towards using randomised trials is a global one. A couple of years ago, I served as one of 2 Australian commissioners on the Global Commission on Evidence to Address Societal Challenges. Led by a secretariat at McMaster University in Canada, the Global Commission’s report recommended formalising and strengthening domestic evidence‑support systems, enhancing and leveraging the global evidence architecture, and putting evidence at the centre of everyday life. It showed that many countries are facing similar challenges when it comes to building the evidence base on what works. And it demonstrated how much countries can learn from one another through global collaboration.
In this talk, I have drawn on a range of overseas examples, alongside Australian ones. It is exciting to see the momentum for sharing knowledge globally on what works in policing being celebrated here tonight. Reducing crime is a shared goal across our countries, and randomised trials of policing programs are one way we can achieve that objective.
I am looking forward to seeing how your formal global evidence‑based policing collaboration, across Australia, New Zealand, the United States, Canada, the United Kingdom and the Netherlands – take shape over coming years.
The Australian Government, too, takes a global perspective. We recognise that good policy ideas often emerge overseas, and that Australia can learn a great deal from similar countries. We also recognise that context matters, and that it can be valuable to run a randomised trial of a promising overseas idea before rolling it out at scale.
In policing, as in medicine, employment, education and a slew of other fields, randomised trials are a valuable tool for understanding what works. Their simplicity – allocating treatment and control groups with the toss of a coin – belies their power to uncover causal effects. Natural experiments and process evaluations have their place, but it’s a rare policy problem where you wouldn’t want a randomised trial to help inform better decision making.
Sometimes, randomised trials can be expensive and time‑consuming. But simple randomised trials can also be ethical and insightful. If you are rolling out a new program across a population, why not consider incorporating randomisation into the rollout? If you have a waitlist, why not use randomisation to ensure fair allocation and improve our understanding of what works? If you are trialling a new way of communicating with people, why not randomly split the population in half, and test the new communication against the old? Many areas of policy, including policing, offer plenty of opportunities to conduct randomised trials. Rather than implementing ad hoc policy changes, why not use a switch in approach as a chance to provide scientifically rigorous evidence?
Thank you for all you do to bring rigour and insight into the vital task of maintaining public safety. The Commonwealth government appreciates your energy, your rigour and your commitment to evidence in improving the quality of policing in Australia and across the world.
Note
This speech draws on 2 excellent literature reviews:
- Farrington D (1983) ‘Randomized experiments on crime and justice’, Crime and Justice: A Review of Research, 4:257–308; and
- Farrington D and Welsh B (2005) ‘Randomized experiments in criminology: What have we learned in the last 2 decades?’, Journal of Experimental Criminology, 1:9–38.
My thanks to Frances Kitt for assistance in preparing these remarks, and to Eleanor Williams and David Cowan for feedback on earlier drafts.