7 ms·

Predictive policing algorithms are racist. They need to be dismantled

tha0xb 6y ago
The narrative on these things change so fast it leaves my head spinning. I thought the algorithms were touted as being less racist than the system that was in place. Isn't ripping these algorithmic systems out just reverting back to the old system?
- Yetanfou 6y ago
  The best way to handle this is to ignore any headline which claims something is some form of -ist or -ism. Like Betteridge's Law of Headlines [1] states that any headline which ends in a question mark can be answered with 'no' there could be a corollary of -ism and -ist which states that any headline which claims something is -ist or promotes -ism is most likely false. While this will cause a few false positives these are dwindled by the sheer mass of false claims which are published daily. False positives can be handled by changing the headline to exclude the -ism/-ist, in this case it could be "An evaluation of predictive policing" or "Predictive policing not the correct approach" or something along those lines. [1] https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
  - deleted 6y ago
    [deleted]
ExcavateGrandMa 6y ago
Done by racists for the racists...
kepler1 6y ago
You know, a problem with all this discourse about police bias and racial inequity is that it ignores, or chooses to wave away, some fundamental truths about pattern recognition, and the dual (sometimes conflicting) charge being given to law enforcement agencies. Law enforcement officers are charged with preventing crime. They are also human beings who learn quickly and use pattern recognition. They would not be effective if they didn't do this. Pattern recognition after years of serving in law enforcement tells you that certain cars and driving behaviors are suspicious. That people who are behaving out of place in a certain neighborhood at a certain time of day are suspicious. That there are reliable indicators that someone is deserving of higher scrutiny if their behavior warrants. You would be a fool not to listen to these accumulated and learned indicators of crime prevention. But now they are being told (or strongly disincentivized) that they must be unbiased. That appearance alone is not sufficient to raise an alert. Well, unfortunately, that is not true, and no law enforcement officer would tell you that every car driving in a neighborhood is equally likely to be a candidate if a robbery were reported. Or that if a violent drug deal went down, every person on the street would be equally likely to be involved. Sure, law enforcement personnel should be trained to not allow uninformed or personal biases to come into play. But the job of policing most definitely is a job that involves identifying patterns and learning biases (in the most technical sense of the word) in order to be effective. You want totally unbiased policing, yet you want police to be effective? Those two things are in conflict, and we need to make some grown-up choices about what is tolerable. An "unbiased" world is a fantasy.
- londons_explore 6y ago
  Couldn't agree more. We need to throw out the idea of removing bias, and instead decide which biases we are happy to use to do policing, and which we are not. Personally, I think this line is impossible to draw, and therefore we should pay people who are inconvenienced by the police. Police stopped your car? Receive $15. Police busted into your house with a warrant? Receive $1000. Now we can let the police use any source of bias in their policing decisions, with the knowledge that those being inconvenienced are being paid well to compensate.
  - raducu 6y ago
    I totally agree (no, I actually don't)... IF the money comes from the cops own pocket.
- raducu 6y ago
  Of course you cannot say or do anything of value without generalization. However: there's a big difference between stopping a certain type of car after a robbery and stopping certain kinds of cars every day, without any specific reason, until you can charge a black youth with cannabis possession. A white person hasn't ever been stopped in a "routine" traffic stop and a black youth has been stopped 4 times in the last month and then searched because a cop claimed he "smelled marijuana" -- that is a bias problem.
- trabant00 6y ago
  Paraphrasing something I heard a while back: "Without preconceived ideas I wouldn't be able to find my way in the bedroom at night". Reconsidering everything you think you know in every situation is clearly impossible and can only lead to inaction and indecision. I've worked a few years on spam filtering. There is no way you can achieve zero false positives and negatives. You can only make it easy to appeal mistakes and processes to minimize their impact. The same is true for the legal system.
- drewcoo 6y ago
  > Law enforcement officers are charged with preventing crime. Some people think the current system it's not preventing crime well. What with the bias and all that. This is underscored by the police brutality in the media. > You want totally unbiased policing I don't think anyone has suggested that's even possible. People have claimed there are gradations somewhere between "totally unbiased" and the current state. And some of us would to try out "somewhat less biased" for a while. Just to see if that's better.
adammunich 6y ago
These just seem like redlining with extra steps. I'm not sure how you could ensure they /don't/ amplify existing bias.
- romwell 6y ago
  This is just redlining, but with "Computer Says NO"[1], which allows the user to pass the responsibility buck onto the mysterious "algorithms". The best part is that they don't even have to run the algorithm, they can just claim they did so[2] to justify their biases. [1]https://www.youtube.com/watch?v=AJQ3TM-p2QI https://www.youtube.com/watch?v=AJQ3TM-p2QI [2]https://www.politico.com/news/magazine/2020/08/30/black-us-diplomat-customs-border-protection-cbp-detained-harassed-325676 https://www.politico.com/news/magazine/2020/08/30/black-us-d...
dayaz36 6y ago
They need to be dismantled regardless of if they're racist or not.
- romwell 6y ago
  Ding ding, this is the right answer! What people don't seem to mention in these discussions is that these algorithms give cops plausible deniability to harass anyone they wish. "Sorry, ma'am, the computer told me to strip-search you. Yes, again." This is the good old "the dog barked, I'm searching your car for weed" trick, except nobody can even see the dog. You think this is far fetched? It's already happening[1]: You have been selected for a random computer-generated inspection. Yes, surely the computer randomly generates an inspection request for a black diplomat every other time she crossed the border. What are the chances, huh. What logs? There isn't a problem. In a lengthy statement, CBP spokesman Matthew Dyman disputed Spears‘ account of her border crossings, saying that she was referred to secondary inspection during 12 of 43 crossings, and that “all referrals were system generated.“ The language of the statement did not make clear whether CBP keeps records of referrals to secondary inspection made for other reasons, such as an officer’s discretion. [1]https://www.politico.com/news/magazine/2020/08/30/black-us-diplomat-customs-border-protection-cbp-detained-harassed-325676 https://www.politico.com/news/magazine/2020/08/30/black-us-d...
maxharris 6y ago
What is the actual definition of racism is being used here? Is it the one advanced by Critical Race Theory, which is along the lines of: "The marginalization and/or oppression of people of color based on a socially constructed racial hierarchy that privileges white people." Or is it the one that most people use: "Racism is the belief that groups of humans possess different behavioral traits corresponding to physical appearance and can be divided based on the superiority of one race over another." These two definitions are mutually exclusive. You can't be a fence-sitter here! Furthermore, they so different that I think that the Critical Race Theorists need to create a new word to describe the concept they are trying to slip into the language. Why is this important? Well, for starters, racism is a violation of the Civil Rights Act of 1964, and institutions such as Princeton University run into legal trouble when they make use of the new idiosyncratic concept masquerading as actual racism. https://www.scribd.com/document/476441457/Princeton-Letter-09-15-2020 https://www.scribd.com/document/476441457/Princeton-Letter-0...
- Kednicma 6y ago
  Predictive policing is racist by either of your definitions, isn't it? What are you so animated about? There's a third, simpler definition, for what it's worth: Racism is the (erroneous, falsified) belief that there exist multiple human races/species today. This third definition leads to your first definition, because the false belief in races leads to separating people according to those false beliefs, which leads to groups of people harboring prejudice towards each other, which leads to social constructions which embed those prejudices. It also leads to your second definition, because the false belief in races leads to false emphasis on skin tone, facial features, and other innocuous differences between people. The point isn't to make "racism" or "racist" bad words. The point is to end racist effects in our society. It's inhumane, wasteful, and at odds with science.
  - ceceron 6y ago
    1) predictive policing is racist by either of your definitions, isn't it? Predictive policing isn't racist by any definition. Given an imaginary perfect predictor, one could detect all the potential criminals. Now, it may happen than a part of society (e.g. the pink people) could be over-represented in this group, and algorithm would target them more often. Would it mean that the predictor is racist? In my opinion, it would only mean, that the society itself has some kind of racial bias and the problem is not the predictor's accuracy, but the environment infected with the racial bias towards the pink people. One could argue that predictive policing would help to sustain this bias and it's probably true, but it doesn't entail that the policing itself is racist. There are many honest and reasonable actions that may inadvertently keep the bias alive. Unless we create a fourth circular definition, I've seen recently in the internet: "Any action/stance that doesn't actively counteract racism is racist". If that's the case, then sure, racism is everywhere. Anyhow, predictive policing is still a terrible idea, a perfect dystopian future. It should be illegal even if it wasn't sustaining the racial bias (which it probably does in the current form). 2) your third definition doesn't entail any of the two presented by @maxharris. One can easily believe there are many human races (it's a common error) and still not penalize/marginalize based on it and not believe there is any relation of superiority between those (imaginary) races. And I do not believe, one should call such people racists, as they do not harm anyone, they are just mistaken.
xupybd 6y ago
Isn't intent required for something to be racist? Not to say that something is good and acceptable if it disadvantages one race over another without intent. It just seems the word racist is being diluted to mean something less.
jpxw 6y ago
> According to US Department of Justice figures, you are more than twice as likely to be arrested if you are Black than if you are white. Why is black being capitalised and not white? Reads very strangely
jpxw 6y ago
> Lack of transparency and biased training data mean these tools are not fit for purpose. If we can’t fix them, we should ditch them. It seems to be taken as a given that arrest data is “biased”. I don’t think there’s solid evidence for this. What if this data isn’t biased?
templain 6y ago
it's 13/56 now not 13/50 anymore, the fbi just released their 2019 numbers
rendall 6y ago
If I understand correctly, the argument against seems to be that bad and unfair arrests will be used as the basis for future arrests, convictions and sentencing. I think that's a valid point. The article goes on to make the case that the entire process is inherently unsalvageable, which I find less than convincing, but I also am not sure that I trust US police to get it right This article is also in the context of recent rioting and calls to defund the police, versus data that indicate the basis for these calls is perhaps not so clear-cut It is a conundrum

The narrative on these things change so fast it leaves my head spinning. I thought the algorithms were touted as being less racist than the system that was in place. Isn't ripping these algorithmic systems out just reverting back to the old system?

Yetanfou 6y ago
The best way to handle this is to ignore any headline which claims something is some form of -ist or -ism. Like Betteridge's Law of Headlines [1] states that any headline which ends in a question mark can be answered with 'no' there could be a corollary of -ism and -ist which states that any headline which claims something is -ist or promotes -ism is most likely false. While this will cause a few false positives these are dwindled by the sheer mass of false claims which are published daily. False positives can be handled by changing the headline to exclude the -ism/-ist, in this case it could be "An evaluation of predictive policing" or "Predictive policing not the correct approach" or something along those lines. [1] https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
- deleted 6y ago
  [deleted]

ExcavateGrandMa 6y ago

Done by racists for the racists...

kepler1 6y ago

You know, a problem with all this discourse about police bias and racial inequity is that it ignores, or chooses to wave away, some fundamental truths about pattern recognition, and the dual (sometimes conflicting) charge being given to law enforcement agencies. Law enforcement officers are charged with preventing crime. They are also human beings who learn quickly and use pattern recognition. They would not be effective if they didn't do this. Pattern recognition after years of serving in law enforcement tells you that certain cars and driving behaviors are suspicious. That people who are behaving out of place in a certain neighborhood at a certain time of day are suspicious. That there are reliable indicators that someone is deserving of higher scrutiny if their behavior warrants. You would be a fool not to listen to these accumulated and learned indicators of crime prevention. But now they are being told (or strongly disincentivized) that they must be unbiased. That appearance alone is not sufficient to raise an alert. Well, unfortunately, that is not true, and no law enforcement officer would tell you that every car driving in a neighborhood is equally likely to be a candidate if a robbery were reported. Or that if a violent drug deal went down, every person on the street would be equally likely to be involved. Sure, law enforcement personnel should be trained to not allow uninformed or personal biases to come into play. But the job of policing most definitely is a job that involves identifying patterns and learning biases (in the most technical sense of the word) in order to be effective. You want totally unbiased policing, yet you want police to be effective? Those two things are in conflict, and we need to make some grown-up choices about what is tolerable. An "unbiased" world is a fantasy.

londons_explore 6y ago
Couldn't agree more. We need to throw out the idea of removing bias, and instead decide which biases we are happy to use to do policing, and which we are not. Personally, I think this line is impossible to draw, and therefore we should pay people who are inconvenienced by the police. Police stopped your car? Receive $15. Police busted into your house with a warrant? Receive $1000. Now we can let the police use any source of bias in their policing decisions, with the knowledge that those being inconvenienced are being paid well to compensate.
- raducu 6y ago
  I totally agree (no, I actually don't)... IF the money comes from the cops own pocket.
raducu 6y ago
Of course you cannot say or do anything of value without generalization. However: there's a big difference between stopping a certain type of car after a robbery and stopping certain kinds of cars every day, without any specific reason, until you can charge a black youth with cannabis possession. A white person hasn't ever been stopped in a "routine" traffic stop and a black youth has been stopped 4 times in the last month and then searched because a cop claimed he "smelled marijuana" -- that is a bias problem.
trabant00 6y ago
Paraphrasing something I heard a while back: "Without preconceived ideas I wouldn't be able to find my way in the bedroom at night". Reconsidering everything you think you know in every situation is clearly impossible and can only lead to inaction and indecision. I've worked a few years on spam filtering. There is no way you can achieve zero false positives and negatives. You can only make it easy to appeal mistakes and processes to minimize their impact. The same is true for the legal system.
drewcoo 6y ago
> Law enforcement officers are charged with preventing crime. Some people think the current system it's not preventing crime well. What with the bias and all that. This is underscored by the police brutality in the media. > You want totally unbiased policing I don't think anyone has suggested that's even possible. People have claimed there are gradations somewhere between "totally unbiased" and the current state. And some of us would to try out "somewhat less biased" for a while. Just to see if that's better.

adammunich 6y ago

These just seem like redlining with extra steps. I'm not sure how you could ensure they /don't/ amplify existing bias.

romwell 6y ago
This is just redlining, but with "Computer Says NO"[1], which allows the user to pass the responsibility buck onto the mysterious "algorithms". The best part is that they don't even have to run the algorithm, they can just claim they did so[2] to justify their biases. [1]https://www.youtube.com/watch?v=AJQ3TM-p2QI https://www.youtube.com/watch?v=AJQ3TM-p2QI [2]https://www.politico.com/news/magazine/2020/08/30/black-us-diplomat-customs-border-protection-cbp-detained-harassed-325676 https://www.politico.com/news/magazine/2020/08/30/black-us-d...

dayaz36 6y ago

They need to be dismantled regardless of if they're racist or not.

romwell 6y ago
Ding ding, this is the right answer! What people don't seem to mention in these discussions is that these algorithms give cops plausible deniability to harass anyone they wish. "Sorry, ma'am, the computer told me to strip-search you. Yes, again." This is the good old "the dog barked, I'm searching your car for weed" trick, except nobody can even see the dog. You think this is far fetched? It's already happening[1]: You have been selected for a random computer-generated inspection. Yes, surely the computer randomly generates an inspection request for a black diplomat every other time she crossed the border. What are the chances, huh. What logs? There isn't a problem. In a lengthy statement, CBP spokesman Matthew Dyman disputed Spears‘ account of her border crossings, saying that she was referred to secondary inspection during 12 of 43 crossings, and that “all referrals were system generated.“ The language of the statement did not make clear whether CBP keeps records of referrals to secondary inspection made for other reasons, such as an officer’s discretion. [1]https://www.politico.com/news/magazine/2020/08/30/black-us-diplomat-customs-border-protection-cbp-detained-harassed-325676 https://www.politico.com/news/magazine/2020/08/30/black-us-d...

maxharris 6y ago

What is the actual definition of racism is being used here? Is it the one advanced by Critical Race Theory, which is along the lines of: "The marginalization and/or oppression of people of color based on a socially constructed racial hierarchy that privileges white people." Or is it the one that most people use: "Racism is the belief that groups of humans possess different behavioral traits corresponding to physical appearance and can be divided based on the superiority of one race over another." These two definitions are mutually exclusive. You can't be a fence-sitter here! Furthermore, they so different that I think that the Critical Race Theorists need to create a new word to describe the concept they are trying to slip into the language. Why is this important? Well, for starters, racism is a violation of the Civil Rights Act of 1964, and institutions such as Princeton University run into legal trouble when they make use of the new idiosyncratic concept masquerading as actual racism. https://www.scribd.com/document/476441457/Princeton-Letter-09-15-2020 https://www.scribd.com/document/476441457/Princeton-Letter-0...

Kednicma 6y ago
Predictive policing is racist by either of your definitions, isn't it? What are you so animated about? There's a third, simpler definition, for what it's worth: Racism is the (erroneous, falsified) belief that there exist multiple human races/species today. This third definition leads to your first definition, because the false belief in races leads to separating people according to those false beliefs, which leads to groups of people harboring prejudice towards each other, which leads to social constructions which embed those prejudices. It also leads to your second definition, because the false belief in races leads to false emphasis on skin tone, facial features, and other innocuous differences between people. The point isn't to make "racism" or "racist" bad words. The point is to end racist effects in our society. It's inhumane, wasteful, and at odds with science.
- ceceron 6y ago
  1) predictive policing is racist by either of your definitions, isn't it? Predictive policing isn't racist by any definition. Given an imaginary perfect predictor, one could detect all the potential criminals. Now, it may happen than a part of society (e.g. the pink people) could be over-represented in this group, and algorithm would target them more often. Would it mean that the predictor is racist? In my opinion, it would only mean, that the society itself has some kind of racial bias and the problem is not the predictor's accuracy, but the environment infected with the racial bias towards the pink people. One could argue that predictive policing would help to sustain this bias and it's probably true, but it doesn't entail that the policing itself is racist. There are many honest and reasonable actions that may inadvertently keep the bias alive. Unless we create a fourth circular definition, I've seen recently in the internet: "Any action/stance that doesn't actively counteract racism is racist". If that's the case, then sure, racism is everywhere. Anyhow, predictive policing is still a terrible idea, a perfect dystopian future. It should be illegal even if it wasn't sustaining the racial bias (which it probably does in the current form). 2) your third definition doesn't entail any of the two presented by @maxharris. One can easily believe there are many human races (it's a common error) and still not penalize/marginalize based on it and not believe there is any relation of superiority between those (imaginary) races. And I do not believe, one should call such people racists, as they do not harm anyone, they are just mistaken.

xupybd 6y ago

Isn't intent required for something to be racist? Not to say that something is good and acceptable if it disadvantages one race over another without intent. It just seems the word racist is being diluted to mean something less.

jpxw 6y ago

> According to US Department of Justice figures, you are more than twice as likely to be arrested if you are Black than if you are white. Why is black being capitalised and not white? Reads very strangely

jpxw 6y ago

> Lack of transparency and biased training data mean these tools are not fit for purpose. If we can’t fix them, we should ditch them. It seems to be taken as a given that arrest data is “biased”. I don’t think there’s solid evidence for this. What if this data isn’t biased?

templain 6y ago

it's 13/56 now not 13/50 anymore, the fbi just released their 2019 numbers

rendall 6y ago

If I understand correctly, the argument against seems to be that bad and unfair arrests will be used as the basis for future arrests, convictions and sentencing. I think that's a valid point. The article goes on to make the case that the entire process is inherently unsalvageable, which I find less than convincing, but I also am not sure that I trust US police to get it right This article is also in the context of recent rioting and calls to defund the police, versus data that indicate the basis for these calls is perhaps not so clear-cut It is a conundrum