Did you read the article on the study you're criticizing? > The formula was particularly likely to falsely flag black defendants as future criminals, wrongly l

Did you read the article on the study you're criticizing? > The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants. The algorithm did not just say black people are more likely to arrested, it said black people were more likely to be arrested than the future data revealed. The algorithm was wrong, and it was wrong even if you take the demonstrably false position that being arrested is the same thing as committing a crime, ignoring the blatant and harmful biases in police behaviors already. Given that this algorithm is being used to inform sentences by judges today, this is absolutely appalling. The broader problem is that algorithms have imperfect signals to work with, and given those signals, it's easier to figure out someone's race than it is to figure out your attitude toward crime, so the algorithm overfits to the data it can see. This exacerbates the problem in a positive feedback loop, and we absolutely need to be cognizant of the damaging implications of this.