5 ms·

Haha, I just looked up the product sheet for Compas CORE, the software at issue: http://www.equivant.com/assets/img/content/Risk-Needs-Assessment.pdf http://ww

by jawns 10y ago

Haha, I just looked up the product sheet for Compas CORE, the software at issue: http://www.equivant.com/assets/img/content/Risk-Needs-Assessment.pdf http://www.equivant.com/assets/img/content/Risk-Needs-Assess... It has a section titled "Make Defensible Decisions": > Fully web-based and Windows compliant, COMPAS is applicable to offenders at all levels from non-violent misdemeanors to repeat violent felons. COMPAS offers separate norms for males, females, community and incarcerated populations. That has ... nothing to do with defensibility. Defensibility is being able to say, "Here's the evidence and the reasoning I used to arrive at this conclusion." What I think is especially frustrating is that in all three risk scales the product offers -- risk of new violent crime, risk of general recidivism, and pretrial risk -- they're dealing in probabilities that don't work on an individual level. That's why they're called "norms." So, if you have 1,000 offenders who match a certain profile, you might be able to fairly accurately predict that 40% of that population are going to reoffend. But being able to predict which offenders constitute the 40% and which constitute the 60% is a totally different thing. There is simply no way to take these probabilities that apply to groups of people and apply them in any fair way to individuals.

CobrastanJorji 10y ago
Wait, does that mean they use entirely separate models for males and females? Does that in turn imply that they explicitly factor in your gender when deciding whether to send you to prison or not? Isn't that on its face not okay?
- deleted 10y ago
  [deleted]
- cryptarch 10y ago
  Would it be ok if we swapped gender with race, religion or any other protected class? There's your answer.
  - dragonwriter 10y ago
    Note that gender is not, in Constitutional case law, in the same tier of protected classes as race and religion (the last two are subject to strict scrutiny, the first to intermediate scrutiny.) Even where the same justification applies, a distinction that survives intermediate scrutiny may not survive strict scrutiny.
    - cryptarch 10y ago
      My bad. Is sex included though?
sideshowb 10y ago
Not saying it's right but isn't that exactly what we do with insurance for driving, health etc?
- slinkyavenger 10y ago
  Yes, it is what we do with insurance. The crux of the matter is that driving is a privilege, incarceration is a restriction on/revocation of a person's rights. Healthcare is a good demonstration of where lines are blurred (at least in the United States). Some view it as a right, others don't.
  - ouid 10y ago
    driving is a privilege? Is that really the core of your argument? How do you distinguish between rights and privileges?
- timoth3y 10y ago
  Not really. It's true that insurance companies charge more to certain people because they are statistically more likely to, for example, be in a car accident than another person or because a home in a particular location is more at risk to wildfires. But that's where the similarities end. Insurance companies have a much more and higher quality data than these companies do. More important, the ways the insurance companies price their product is public and subject to competition. If an insurance company realizes that a specific segment of the market is less risky than previously thought they can make a lot of money by dropping the price and underwriting policies for that group. These courtroom applications use secret algorithms, and frankly, have little incentive to improve them.
  - sideshowb 10y ago
    Good point. I think competition is the crux of the matter here. Alas I don't think the shop-around model would work for sentencing.
- pdkl95 10y ago
  Many types of data are prohibited for various types of insurance decisions. This varies greatly, from general rules such as banning price discrimination due to race or sex, to specific topics like the ACA[1]'s prohibition against denying coverage due to "pre-existing conditions". With modern data analysis methods, it is (sometimes) possible to infer prohibited data from other collections of data. For example, various machine learning techniques used on economic values such as income, home value, and proximity to good schools might reveal something about recidivism rates, but it's probably finding a confounding variable such as redlining[2] or other types of institutional bias[3]. This is a problem in both legal decisions and insurance, and transparency in how decisions are made is vital in both. That said, the impact on insurance is probably[4] a relatively minor change in price, which is very different from a Boolean decision about someone's freedom. Even in the unlikely case where the statistical biases are known and accounted for, it still isn't appropriate to over-interpret results about populations as if they apply to any particular individual. [1] Affordable Care Act ("Obamacare") [2] https://en.wikipedia.org/wiki/Redlining https://en.wikipedia.org/wiki/Redlining [3] https://www.youtube.com/watch?v=qXQA6D4JC0A https://www.youtube.com/watch?v=qXQA6D4JC0A [4] for definitions of "probable" somewhere between "an educated guess" and "meh; whatever"

by jawns 10y ago

CobrastanJorji 10y ago

Wait, does that mean they use entirely separate models for males and females? Does that in turn imply that they explicitly factor in your gender when deciding whether to send you to prison or not? Isn't that on its face not okay?

deleted 10y ago
[deleted]
cryptarch 10y ago
Would it be ok if we swapped gender with race, religion or any other protected class? There's your answer.
- dragonwriter 10y ago
  Note that gender is not, in Constitutional case law, in the same tier of protected classes as race and religion (the last two are subject to strict scrutiny, the first to intermediate scrutiny.) Even where the same justification applies, a distinction that survives intermediate scrutiny may not survive strict scrutiny.
  - cryptarch 10y ago
    My bad. Is sex included though?

sideshowb 10y ago

Not saying it's right but isn't that exactly what we do with insurance for driving, health etc?

slinkyavenger 10y ago
Yes, it is what we do with insurance. The crux of the matter is that driving is a privilege, incarceration is a restriction on/revocation of a person's rights. Healthcare is a good demonstration of where lines are blurred (at least in the United States). Some view it as a right, others don't.
- ouid 10y ago
  driving is a privilege? Is that really the core of your argument? How do you distinguish between rights and privileges?
timoth3y 10y ago
Not really. It's true that insurance companies charge more to certain people because they are statistically more likely to, for example, be in a car accident than another person or because a home in a particular location is more at risk to wildfires. But that's where the similarities end. Insurance companies have a much more and higher quality data than these companies do. More important, the ways the insurance companies price their product is public and subject to competition. If an insurance company realizes that a specific segment of the market is less risky than previously thought they can make a lot of money by dropping the price and underwriting policies for that group. These courtroom applications use secret algorithms, and frankly, have little incentive to improve them.
- sideshowb 10y ago
  Good point. I think competition is the crux of the matter here. Alas I don't think the shop-around model would work for sentencing.
pdkl95 10y ago
Many types of data are prohibited for various types of insurance decisions. This varies greatly, from general rules such as banning price discrimination due to race or sex, to specific topics like the ACA[1]'s prohibition against denying coverage due to "pre-existing conditions". With modern data analysis methods, it is (sometimes) possible to infer prohibited data from other collections of data. For example, various machine learning techniques used on economic values such as income, home value, and proximity to good schools might reveal something about recidivism rates, but it's probably finding a confounding variable such as redlining[2] or other types of institutional bias[3]. This is a problem in both legal decisions and insurance, and transparency in how decisions are made is vital in both. That said, the impact on insurance is probably[4] a relatively minor change in price, which is very different from a Boolean decision about someone's freedom. Even in the unlikely case where the statistical biases are known and accounted for, it still isn't appropriate to over-interpret results about populations as if they apply to any particular individual. [1] Affordable Care Act ("Obamacare") [2] https://en.wikipedia.org/wiki/Redlining https://en.wikipedia.org/wiki/Redlining [3] https://www.youtube.com/watch?v=qXQA6D4JC0A https://www.youtube.com/watch?v=qXQA6D4JC0A [4] for definitions of "probable" somewhere between "an educated guess" and "meh; whatever"