3 ms·

Thanks for posting this. I encourage this crowd to to take a look at the methodology too: https://www.propublica.org/article/how-we-analyzed-the-compas-recidivi

• gleb 10y ago
  Thanks for posting a link to the methodology. Does this sentence "Northpointe does offer a custom test for women, but it is not in use in Broward County. " imply that the base COMPAS model does not take gender into account?
• gleb 10y ago
  Are you sure what you found is not just Simpson's paradox? When I look at the 2 KM plots for white/blacks, they are mostly the same. It's pretty clear that the model is not prejudiced against blacks, in fact it's somewhat prejudiced against whites. [1] Your main editorial claim is that whites tend to be misclassified as "good" and blacks as "bad." But I think what's actually happening is that algorithm is more likely to misclassify low_risk as "good", and high_risk as "bad".[2] Combine that with vastly more whites than blacks being low_risk (as you show earlier) and you get the observed "injustice". I'll also note that the KM for whites flatten out at 2 years, unlike for blacks. This is actually a big deal if statistically significant. But that's a separate conversation. Footnotes: 1 - this is acknowledged in methodology page "black defendants who scored higher did recidivate slightly more often than white defendants (63 percent vs. 59 percent)." 2 - why that is I don't yet fully understand (and I'd like to) but it looks's to be simple math that follows from low risk mostly not recidivating, and high risk mostly yes recidivating