5 ms·

California Bill Would Mandate 'Crime Prediction' Algorithms Instead of Cash Bail

9 points by elsewhen 6y ago

ThA0x2 6y ago
Can't wait for these to be decried as "racist" once the algorithms start denying bail to more Blacks than Whites. The algorithm could be 100% accurate and it would STILL be decried as "racist" because of the different outcomes between the races.
- colejohnson66 6y ago
  So you see no problem in a “black box” deciding whether you could go free or not after you commit a crime? Why not just throw out the whole justice system then and make rulings be done by a computer? > The algorithm could be 100% accurate and it would STILL be decried as "racist" because of the different outcomes between the races. This sentence is actually kindof racist. It implies that one race commits more crime. It may be true statistically, but if systematic racism exists, there will be biases in reporting.
  - ThA0x2 6y ago
    >So you see no problem in a “black box” deciding whether you could go free or not after you commit a crime? This is a straw man. Nonetheless, even if it was 100% transparent, FOSS, etc. people would still be screaming "RACIST!" >Why not just throw out the whole justice system then and make rulings be done by a computer? Non sequitur.
    - Gibbon1 6y ago
      The problem is the algorithms are designed to put people in a group and then the groups are treated differently. That's the mechanism underpinning racism. And other disparities. You take the current garbage and the algorithm spits it back out. Current garbage looks like this. I know three people that got popped for drugs. One served time and has a felony record. One had it reduced to a misdemeanor and did 10 hours of community service. And the third has no record at all. You can probably guess the socioeconomic status each of those three and you'd guess correctly. And I'm sure the algorithm will also guess correctly.
      - ThA0x2 6y ago
        >The problem is the algorithms are designed to put people in a group and then the groups are treated differently. That's not the problem, that's literally the solution. >That's the mechanism underpinning racism. And other disparities. Not at all. The mechanism underpinning racism is believing that race accounts for differences in human character or ability and that a particular race is superior to others. Disparities are not inherently racist. >Current garbage looks like this. I know three people that got popped for drugs. One served time and has a felony record. One had it reduced to a misdemeanor and did 10 hours of community service. And the third has no record at all. One had a long rap sheet and previously was in jail, one had a few priors, and the third had a clean record. You can probably guess the socioeconomic status each of those three and you'd guess correctly. We can both make up dumb anecdotes all day. >And I'm sure the algorithm will also guess correctly. The algorithm will correctly guess which of the three is more likely to flee, be a public threat, etc. Sounds like the algorithm is working.
        Gibbon1 6y ago
        > One had a long rap sheet and previously was in jail, one had a few priors, and the third had a clean record. Nope. You lose.
        ThA0x2 6y ago
        >We can both make up dumb anecdotes all day. Yep. I win. Glad you admit defeat.
        Gibbon1 6y ago
        Please do not interact with me.
        ThA0x2 6y ago
        Gibbon1's final admission of defeat. I won.
        6y ago
astrophysician 6y ago
As a data scientist, this is terrifying. A questionnaire is asking things like "have you only held low-wage jobs" to determine if someone should be locked up? The model isn't completely open to the public? (how that is justified is beyond me, this model literally decides whether or not you, as a citizen, are locked in a box). If it's not obvious: these methods easily reinforce all sorts of terrible biases in the data. Even with a perfectly transparent model, it's unclear what sort of data biases you are reinforcing, because getting unbiased data is extremely difficult if not impossible.
lsb 6y ago
This is a serious problem in many ways. Ethics can be a distraction from justice. Ethics sites the problem in the algorithm, whereas justice sites the problem in the power differential. People generally trust computers more than random human decisions, and computers can automate the repercussions of their decisions quickly. If you work hourly at a government job, and you get your hours yanked when you're judged a risk, without time for an appeal, things spiral out of your control, exacerbating oppression.

9 points by elsewhen 6y ago

Can't wait for these to be decried as "racist" once the algorithms start denying bail to more Blacks than Whites. The algorithm could be 100% accurate and it would STILL be decried as "racist" because of the different outcomes between the races.

colejohnson66 6y ago
So you see no problem in a “black box” deciding whether you could go free or not after you commit a crime? Why not just throw out the whole justice system then and make rulings be done by a computer? > The algorithm could be 100% accurate and it would STILL be decried as "racist" because of the different outcomes between the races. This sentence is actually kindof racist. It implies that one race commits more crime. It may be true statistically, but if systematic racism exists, there will be biases in reporting.
- ThA0x2 6y ago
  >So you see no problem in a “black box” deciding whether you could go free or not after you commit a crime? This is a straw man. Nonetheless, even if it was 100% transparent, FOSS, etc. people would still be screaming "RACIST!" >Why not just throw out the whole justice system then and make rulings be done by a computer? Non sequitur.
  - Gibbon1 6y ago
    The problem is the algorithms are designed to put people in a group and then the groups are treated differently. That's the mechanism underpinning racism. And other disparities. You take the current garbage and the algorithm spits it back out. Current garbage looks like this. I know three people that got popped for drugs. One served time and has a felony record. One had it reduced to a misdemeanor and did 10 hours of community service. And the third has no record at all. You can probably guess the socioeconomic status each of those three and you'd guess correctly. And I'm sure the algorithm will also guess correctly.
    - ThA0x2 6y ago
      >The problem is the algorithms are designed to put people in a group and then the groups are treated differently. That's not the problem, that's literally the solution. >That's the mechanism underpinning racism. And other disparities. Not at all. The mechanism underpinning racism is believing that race accounts for differences in human character or ability and that a particular race is superior to others. Disparities are not inherently racist. >Current garbage looks like this. I know three people that got popped for drugs. One served time and has a felony record. One had it reduced to a misdemeanor and did 10 hours of community service. And the third has no record at all. One had a long rap sheet and previously was in jail, one had a few priors, and the third had a clean record. You can probably guess the socioeconomic status each of those three and you'd guess correctly. We can both make up dumb anecdotes all day. >And I'm sure the algorithm will also guess correctly. The algorithm will correctly guess which of the three is more likely to flee, be a public threat, etc. Sounds like the algorithm is working.
      - Gibbon1 6y ago
        > One had a long rap sheet and previously was in jail, one had a few priors, and the third had a clean record. Nope. You lose.
        ThA0x2 6y ago
        >We can both make up dumb anecdotes all day. Yep. I win. Glad you admit defeat.
        Gibbon1 6y ago
        Please do not interact with me.
        ThA0x2 6y ago
        Gibbon1's final admission of defeat. I won.
      - 6y ago

astrophysician 6y ago

As a data scientist, this is terrifying. A questionnaire is asking things like "have you only held low-wage jobs" to determine if someone should be locked up? The model isn't completely open to the public? (how that is justified is beyond me, this model literally decides whether or not you, as a citizen, are locked in a box). If it's not obvious: these methods easily reinforce all sorts of terrible biases in the data. Even with a perfectly transparent model, it's unclear what sort of data biases you are reinforcing, because getting unbiased data is extremely difficult if not impossible.

lsb 6y ago

This is a serious problem in many ways. Ethics can be a distraction from justice. Ethics sites the problem in the algorithm, whereas justice sites the problem in the power differential. People generally trust computers more than random human decisions, and computers can automate the repercussions of their decisions quickly. If you work hourly at a government job, and you get your hours yanked when you're judged a risk, without time for an appeal, things spiral out of your control, exacerbating oppression.