AI-Assisted Police Reports: Preliminary Musings about Axon’s Draft One, Measurement, and AI Hype

I’m experimenting with writing out some early-stage research ideas in blog post form. Happy to hear your thoughts!

In April 2024, Axon — a technology company supplying over 90% of U.S. police departments — released Draft One, their latest AI tool. Draft One transcribes the audio from officer body-worn cameras to generate a narrative of what happened in a “police encounter” using ChatGPT-4.

I think this product is more AI hype than something that will see widespread deployment. The conditions in which Axon recommends using the product (no arrest made, environments with limited background noise or crosstalk) limited its use and impact. But let’s take the product seriously for a minute. Similar AI tools keep popping up in the market, and there’s a lot of interest in this space. Both police accountability advocates and “AI for social good” technologists see the intersection of artificial intelligence and policing as fertile ground for intervention.

The justification for AI assisted police report writing always starts with time-savings. Estimates are that officers can spend up to half of their days writing reports. That’s very boring! Many police officers also believe that this is a bad use of their time (perhaps a more complicated question). Axon advertises that Draft One has led to an 82% decrease in time spent writing reports. However, independent research by Ian T. Adams challenges this claim — his randomized controlled trial with a mid-sized police department found no statistically significant reduction in report writing time.

The second justification for such tools is about report quality. Most everyone agrees that police reports are often “inaccurate” or “incomplete.” As someone who spent only 10 weeks doing criminal public defense work, I can tell you that’s putting it mildly. And these reports matter — they’re important artifacts when prosecutors use to decide whether to bring criminal cases, and they are what police officers use to “remember” an encounter when testifying in court.

Figuring out how to measure police report quality is difficult and important. I want to side-step that conversation and talk about how Axon imagines that quality issues in police report writing come about.

The main suggestion that Axon provides for making Draft One be effective (beyond the quiet environment you need to be having a police encounter in) is to have the police narrate for their body-worn camera what is happening. This makes it easier for ChatGPT to match what’s happening to pre-written templates. First, from a sociotechnical perspective, how fascinating! Changing the practices of policing and how people hear the police talk to them to account for technology!

But this “narrate what you’re doing” approach misses why police reports are “low-quality”. The underlying logic seems to be that reports are bad because officers forget to write down what was in their head during the encounter. Therefore, by talking out loud to create a better audio transcript the problem is solved. Put simply, the primary problematic quality gaps between body camera footage and police reports aren’t because officers are forgetful about minor details.

As AI-assisted police reporting continues to be piloted across the country, multiple things can be true at the same time. Draft One is unlikely to be a revolutionary tool. AI interventions in criminal justice— including Draft One — often fail to remedy the problems they seek to address. The first two points do not preclude that the introduction of these often-limited tools from shaping the way that police function on the ground.

Both Andrew Guthrie Ferguson and Ian T. Adams have written really interesting papers about Draft One and AI-assisted police report writing. Sarah Brayne’s scholarship has also greatly influenced how I think about how police interact with technology.