Yes, this is something we researched and thought about. The idea is "garbage in, garbage out". If the data input into the algorithm reflects a systemic bias, ob

Yes, this is something we researched and thought about. The idea is "garbage in, garbage out". If the data input into the algorithm reflects a systemic bias, obviously any predictions are going to be a product of the input to some extent. Our predictions were based on the city's crime report data, which is clearly not an accurate picture of crime in the city, but it's what we've got. Sure, it's entirely possible that if police are dramatically over-policing certain poor or minority neighborhoods and generating a lot of crime report data based on those arrests that the predictions could potentially be affected. But poor and minority neighborhoods tend to under-report crimes compared to wealthier areas, so there is a balancing counter-effect to some extent. Another thing we've seen happen sometimes is systematic under-reporting or non-reporting of crimes to make the crime stats for the city look better, as the police department is often judged on these crime stats. Ultimately we have to work with what we have and the police are responsible for maintaining high quality data and ethical standards, but I don't lose any sleep worrying about if my code systematically oppressed minorities for two reasons: 1.First of all, most police departments had separate predictions run per "district" and typically had a fixed number of officers allocated to each district. The idea that we were directing cops away from rich neighborhoods and having them go after the poor or minorities assumes that we had far more power than we actually did. The number of cops assigned to each city district / beat is a political or bureaucratic decision and not something our software is involved in. 2. Furthermore, our app was not directing the cops every move, nor was it trying to. We just told them "when you're not answering a call or doing anything else, when you have a spare minute, check if you're near a prediction box and go figure out why the computer flagged it as a high risk area. Maybe it's a parking lot and the lights have gone out encouraging car burglaries". And even as just a small slice of their time, we really had to push to get them to use the app because the way they saw it was "What does this stupid app know about my city that I, with 20 years of experience, don't know? One of these prediction boxes is in the middle of a lake". Based on what I saw from the analytics and usage logs, I think it would be tough to make a case that anyone was oppressed because the cops barely even used the thing! Not that I take these issues lightly, but I always laugh when people criticize PredPol or someone calls me a murderer on Twitter because it vastly overstates our impact. Maybe I should take it as a compliment that people really think I am powerful enough to direct police to oppress minorities using only a shitty rails app I wrote in my college dorm room. But whatever, you can't do or try anything new in this country today without getting dramatically criticized for it to fuel the internet-rage clickbait economy, so I guess it's just something all of us have to get used to. I haven't had a day to day role at the company for a few years, which makes it a lot easier not to worry about these things as well. But ultimately police using technology and data effectively can REDUCE bias and injustice AND make people safer. I think the questions the Verge presents are certainly worth asking, but based on my experience I do believe that the app I implemented helped more people than it harmed.