Amazingly, the NBA has managed to succeed where many police forces have failed.

Michael Lewis, famed author of the books (and subsequent hit movies) Moneyball and The Big Short, has a fascinating new podcast titled Against the Rules which explores the special role of referees and other adjudicators in society. They keep systems fair, make markets and institutions function properly, and protect vulnerable individuals from exploitation. The premise of the podcast is that this venerable duty is under attack because of rising inequality and polarization in society.

One episode will be especially interesting for those in the criminal justice and policing worlds. It explores a massive experiment to reduce bias in judgement.

Recently, huge databases of millions of police stops and enforcement actions have allowed social scientists to precisely measure bias in policing practices. Bias has a large impact on communities but can be hard to rectify. Many police forces have tried to minimize bias through training on implicit bias mechanisms and offer ways to counteract their influence. It is extremely difficult, however, to overcome habits that have formed over a lifetime and work against organizational cultures and social networks that reinforce bad behavior.

In the episode “Ref, You Suck!” Lewis introduces us to the NBA replay center in Secaucus, NJ, a state-of-the art facility with a hundred screens playing real-time feeds from dozens of camera angles from every live NBA game. Each call by a referee is reviewed and evaluated by experts using playback from several camera angles. They have incorporated this feed into ongoing training for referees, and each ref is assigned a score representing the accuracy of their game calls.

As a result, the overall accuracy of calls has improved dramatically, and bias has virtually disappeared.

After listening to the podcast one can’t help but wonder, can these same sort of processes be applied to performance review in policing? With body cameras and massive databases of policing stops to serve as reference points for performance, can random samples be used to assess individual officer performance and address potential bias or improve policing tactics?

It may not be a perfect analogy. NBA referees make between $150,000 and $500,000 a year, whereas patrol officers get paid an average of $65,000 a year. It cost $15 million to build the replay center, and it employs 15 professional referees at any time to review plays. On the other hand, cities are paying out millions of dollars each year in civil lawsuits brought against departments for discrimination and excess force cases. Perhaps they would actually save money by investing in expensive training practices if they can avoid future lawsuits?

Another important question is who watches the watchman? In many cases reports of abuse by police are adjudicated internally within police forces, and citizen oversight is primarily advisory or educational. Can fratenal societies be unbiased referees of their own actions? Would citizens or elected officials bring their own biases to the process? If a police replay center was created to review police actions, who would the experts be behind the screen? Technology often doesn’t resolve the fundamental challenges of good governance!