7 ms·

California just replaced cash bail with algorithms

45 points by antr 8y ago

SargeZT 8y ago
This worked really well in New Jersey. However, California's implementation of county-by-county standards as opposed to a statewide system could still leave a lot of room for error.
- spuz 8y ago
  There is a good Planet Money episode about New Jersey's overhaul of the bail system: https://www.npr.org/sections/money/2018/08/29/643072388/episode-783-new-jersey-bails-out https://www.npr.org/sections/money/2018/08/29/643072388/epis...
- dsfyu404ed 8y ago
  Never underestimate the ability of politicians and special interest groups to screw up something that's been done successfully elsewhere. Optimism and expectations of spectacular failure are not mutually exclusive.
basch 8y ago
I post this too often, but its very applicable here. http://www.bbc.co.uk/blogs/adamcurtis/entries/78691781-c9b7-30a0-9a0a-3ff76e8bfe58 http://www.bbc.co.uk/blogs/adamcurtis/entries/78691781-c9b7-...
andrewaylett 8y ago
Sounds like the sort of thing GDPR's regulations of automated decision making is intended to deal with: https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/rights-related-to-automated-decision-making-including-profiling/ https://ico.org.uk/for-organisations/guide-to-the-general-da... Specifically, even if you're allowed by Article 22 to perform automated decision-making (and decisions on bail that are mandated by law, as in the case at hand, would be allowed) must still: * provide meaningful information about the logic involved in the decision-making process, as well as the significance and the envisaged consequences for the individual; * use appropriate mathematical or statistical procedures; ensure that individuals can: * obtain human intervention; * express their point of view; and * obtain an explanation of the decision and challenge it;
- chatmasta 8y ago
  GDPR does not apply to sovereign nations and states. The European Commission is not the boss of the world. That said, it may be a useful framework to adopt for uses such as regulating pre-trial risk assessment algorithms.
  - DerpyBaby123 8y ago
    Not OP, but they never claimed that what California is doing is unlawful because of GDPR. Their comment is still insightful because it shows an example of legislation that, if applied, could ensure more transparency and humanity to automated decision making. The lack of such legislation becomes more glaring as more and more examples are demonstrated.
will_brown 8y ago
A little over a year ago I was the victim of some crimes including: -kidnapping with a deadly weapon -assault with a deadly weapon/firearm -car jacking with a deadly weapon The defendant has bonded out twice in the case. The first time there was only a charge of grand theft of my car, and bond was revoked after the investigation and the other more serious charges were filed. Thereafter, the defendant bonded out again after all the charges were filed, and posted $300,000+ bond. As the victim (and a lawyer) I knew the defendant would be granted bond (although I believed the defendant was both a flight risk and danger to the community and bond should have been denied), so I requested gps tracking. Less than a week later the defendant was rearrested for violating the terms of bond...all based on the gps tracking. I’m not sure I trust an algorithm to determine if someone is a flight risk or danger to the community. Historically, the cash bond was like the carrot and stick to making sure a released defendant showed up to court. I think technology, like the gps tracker, can probably replace cash bond in almost all cases.
- jessaustin 8y ago
  Was the defendant rearrested by police or by bail agents?
  - will_brown 8y ago
    I think on the first violation/revocation of bond the defendant was arrested by police (but the defendant was in another county at the time, so they had to extradite back to the county). The 2nd violation/revocation of bond, I think the warrant was executed by police (but in the same county as the case).
- rdl 8y ago
  Were those crimes all one incident? If not I would seriously have considered moving.
  - will_brown 8y ago
    All one incident, while on vacation.
deleted 8y ago
[deleted]
Deckard256 8y ago
Our courts here in New Mexico got bamboozled by the use of the "Arnold Tool"which is basically the same thing. Anyways, Yada Yada Yada, we're now worst in nation for auto theft because of it and we have no recourse against it unless we want to change our state constitution because our Supreme Court mandated it as it was sold as being more fair. All it did was put repeat offenders on the street the next day. My car has been broken into three times since the change. Garbage in, garbage out.
- sambull 8y ago
  Your anecdote about petty theft is so valid
  - deleted 8y ago
    [deleted]
  - magduf 8y ago
    Auto theft is by definition not "petty theft".
    - sambull 8y ago
      Except they didnt steal his car just broke into it.
- tantalor 8y ago
  Er, wouldn't a repeat offense be a red flag for "no release"? It says so right in the article: a system to ascertain a suspect’s risk of flight or committing another crime during the trial process Sounds like your state is doing it wrong?
  - magduf 8y ago
    Sounds like his state is definitely doing it wrong. This is like building a car with square wheels and then proclaiming that cars are a terrible idea.
- 8y ago
mrcactu5 8y ago
Machine Learning Algorithms are severely racist towards defendants. We know, for example, that AI algorithms mistake Black people for primates... but they are no more racist than the criminal justice system at large. And they have a better chance of getting fixed. https://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white-guy-problem.html https://www.nytimes.com/2016/06/26/opinion/sunday/artificial...
- drharby 8y ago
  Wow citation needed, and a nyt opinion piece is not research proving your claim. "Algorithms are racist" ha!
  - tantalor 8y ago
    Here's a few, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... https://www.technologyreview.com/s/607955/inspecting-algorithms-for-bias/ https://www.technologyreview.com/s/607955/inspecting-algorit... http://fortune.com/longform/ai-bias-problem/ http://fortune.com/longform/ai-bias-problem/ https://www.pcmag.com/article/361661/artificial-intelligence-has-a-bias-problem-and-its-our-fau https://www.pcmag.com/article/361661/artificial-intelligence... Ethics and Bias in Machine Learning: A Technical Study of What Makes Us “Good” https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1042&context=jj_etds https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1...
  - drharby 8y ago
    Man - to hell with this forum, I ask for evidence on a claim that other posters have pointed out is a misapplication of other research 'webscam algos != cash bail algos' And I get downvoted!? This is tech news, something the general public can get easily manipulated into...and for demanding a degree of reasonable discourse and reference, i get downvoted? Bleh, this place is turning into a place i cant speak relatively freely in.
    - Stratoscope 8y ago
bjt2n3904 8y ago
There's this growing trend of... "Well look, this person in a lab coat with a clipboard did some complicated things with a computer, using all the trendy words like 'algorithms', 'machine learning', and 'big data'! Their results must be very accurate and unbiased, and much better than a human looking at things." I'm more and more convinced that most of these cases are just stuffing an excel spreadsheet into a black box, and shaking it. Hearing that we're using "algorithms" in criminal justice doesn't give me warm and fuzzy.
- LiterallyDoge 8y ago
  Or maybe we should be electing more people who understand how to read clipboards?
- IanDrake 8y ago
  >Their results must be very accurate and unbiased That is the assumption that scares me. The algorithm encodes the bias and applies it in every case. This feature will only save you from a shifting bias.
  - rectang 8y ago
    Up until now we were saddled with the extremely biased and unjust bond system. Calls to make the algorithms transparent are well taken -- but let's not go back.
    - cgriswald 8y ago
      Your thinking is very dangerous. Even if the algorithm is transparent and fair, that does not necessarily equate to a step forward because of the bias already present in the system. The algorithm will be[0] applied equally, sure, but that does not mean outcomes will be equal. For instance, consider that owning a major asset probably decreases flight risk. That seems fair. But, this means that poor people are probably more of a flight risk than a middle- or upper-class homeowner. So if the algorithm considers this, poor people would be disproportionately affected. Also consider that having family abroad or being from another country probably increases flight risk. Hence, immigrants would be disproportionately affected. But now, when there is an unjust outcome, there isn't even an option to avoid jail time (even if that option is difficult to reach or undesirable); and the human responsible for the decision to incarcerate can simply wash their hands of it, since the algorithm gave the suspect a grade of "D". No need for empathy, sympathy, or guilt. Then there are the pending discussions about what is 'fair' and the answer to those questions is politics... [0] - Edit: Could be
stanleydrew 8y ago
I think there are valid reasons to not trust an algorithm to determine whether someone is a flight risk or danger to the community. But there was already an algorithm that came up with the cash bail amount for a defendant, which was purportedly based on the defendant's flight risk and likelihood of endangering the communoty. I don't pretend to be an expert so I may be wrong but I think the algorithm boiled down to something like "let the judge decide". I think transparency around the algorithm is incredibly important, so anything that explicitly makes the properties of the algorithm part of the debate I think is good.
DevX101 8y ago
As governments start to implement more algorithms in public policy decisions, it becomes increasingly important that ALL algorithms be open sourced for public review unless there is a national security risk. Having a summary of how the algorithm works is necessary but not sufficient.
panic 8y ago
Several of this bill's original co-sponsors ended up opposing it due to the lack of accountability in the risk assessment system and the fact that it gives police even more power over the justice process: https://siliconvalleydebug.org/stories/silicon-valley-de-bug-s-letter-of-opposition-to-california-s-false-bail-reform-bill-sb10 https://siliconvalleydebug.org/stories/silicon-valley-de-bug...
gpm 8y ago
While there are some valid concerns about using algorithms I'm quite happy that they got rid of cash bail. Making presumed innocent people pay to get out of jail has never made any sense to me. For serious crimes, the number of people who say "well I would run for it to try and most of the rest of my life in prison, but I won't because then I (or whoever put the money up for me) won't get my bail back" has to be tiny. For non serious crimes the number of people motivated by bail and not by avoiding spending the rest of their life as a fugitive has to be tiny.
- posterboy 8y ago
  Isn't the rational that somebody needs to vouch for release and prove trust by risking loss of the money?
CoryG89 8y ago
Shouldn't the "algorithm" need to be published? Surely someone being held prisoner by a computer program should at least have the right to examine and potentially challenge the source code and data being used.
- JoeAltmaier 8y ago
  Actually, they're being held because they were arrested, right?
pseudolus 8y ago
This points to one of the dilemmas that societies will be hard pressed to confront. Are we capable of accepting decisions underpinned by algorithms even when the outcome is negative in a few cases? Inevitably an individual released as a result of one of these algorithms will commit a criminal act and the argument will arise that had an individual been responsible for making the evaluation and that had cash bail existed then such crime would not have occurred. I think people will have significant difficulty in accepting even the slightest of errors with algorithms (particularly with respect to violent crime) even if the overall result is lower than if humans had been the sole deciders. In many ways it's analogous with the ongoing debates about so-called driverless cars.
anfilt 8y ago
One step forward, and well who knows how many back.
kbos87 8y ago
So we are moving toward a world where one person gets out, while the other has to languish in prison for months or years awaiting trial because an algorithm said so. And the accused aren’t allowed to know what’s happening behind the scenes of the algorithm because we need to protect the competitive advantage of the vendor who sold the software to the state. This is total and complete insanity and elected officials should pay with their jobs and their reputations for even suggesting something so unjust.
- rectang 8y ago
  Where is your outrage about how poor people are wronged by the current system?
  - kbos87 8y ago
    I am absolutely outraged about the way the current system treats and traps the poor. But we need to be clear that this isn’t a fix; it’s a further obfuscation of the problem. I have little doubt that said algorithms will continue to wrong the poor at similar or greater rates - the only difference is that it will be significantly harder to detect.

45 points by antr 8y ago

This worked really well in New Jersey. However, California's implementation of county-by-county standards as opposed to a statewide system could still leave a lot of room for error.

spuz 8y ago
There is a good Planet Money episode about New Jersey's overhaul of the bail system: https://www.npr.org/sections/money/2018/08/29/643072388/episode-783-new-jersey-bails-out https://www.npr.org/sections/money/2018/08/29/643072388/epis...
dsfyu404ed 8y ago
Never underestimate the ability of politicians and special interest groups to screw up something that's been done successfully elsewhere. Optimism and expectations of spectacular failure are not mutually exclusive.

basch 8y ago

I post this too often, but its very applicable here. http://www.bbc.co.uk/blogs/adamcurtis/entries/78691781-c9b7-30a0-9a0a-3ff76e8bfe58 http://www.bbc.co.uk/blogs/adamcurtis/entries/78691781-c9b7-...

andrewaylett 8y ago

Sounds like the sort of thing GDPR's regulations of automated decision making is intended to deal with: https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/rights-related-to-automated-decision-making-including-profiling/ https://ico.org.uk/for-organisations/guide-to-the-general-da... Specifically, even if you're allowed by Article 22 to perform automated decision-making (and decisions on bail that are mandated by law, as in the case at hand, would be allowed) must still: * provide meaningful information about the logic involved in the decision-making process, as well as the significance and the envisaged consequences for the individual; * use appropriate mathematical or statistical procedures; ensure that individuals can: * obtain human intervention; * express their point of view; and * obtain an explanation of the decision and challenge it;

chatmasta 8y ago
GDPR does not apply to sovereign nations and states. The European Commission is not the boss of the world. That said, it may be a useful framework to adopt for uses such as regulating pre-trial risk assessment algorithms.
- DerpyBaby123 8y ago
  Not OP, but they never claimed that what California is doing is unlawful because of GDPR. Their comment is still insightful because it shows an example of legislation that, if applied, could ensure more transparency and humanity to automated decision making. The lack of such legislation becomes more glaring as more and more examples are demonstrated.

will_brown 8y ago

A little over a year ago I was the victim of some crimes including: -kidnapping with a deadly weapon -assault with a deadly weapon/firearm -car jacking with a deadly weapon The defendant has bonded out twice in the case. The first time there was only a charge of grand theft of my car, and bond was revoked after the investigation and the other more serious charges were filed. Thereafter, the defendant bonded out again after all the charges were filed, and posted $300,000+ bond. As the victim (and a lawyer) I knew the defendant would be granted bond (although I believed the defendant was both a flight risk and danger to the community and bond should have been denied), so I requested gps tracking. Less than a week later the defendant was rearrested for violating the terms of bond...all based on the gps tracking. I’m not sure I trust an algorithm to determine if someone is a flight risk or danger to the community. Historically, the cash bond was like the carrot and stick to making sure a released defendant showed up to court. I think technology, like the gps tracker, can probably replace cash bond in almost all cases.

jessaustin 8y ago
Was the defendant rearrested by police or by bail agents?
- will_brown 8y ago
  I think on the first violation/revocation of bond the defendant was arrested by police (but the defendant was in another county at the time, so they had to extradite back to the county). The 2nd violation/revocation of bond, I think the warrant was executed by police (but in the same county as the case).
rdl 8y ago
Were those crimes all one incident? If not I would seriously have considered moving.
- will_brown 8y ago
  All one incident, while on vacation.

deleted 8y ago

[deleted]

Deckard256 8y ago

Our courts here in New Mexico got bamboozled by the use of the "Arnold Tool"which is basically the same thing. Anyways, Yada Yada Yada, we're now worst in nation for auto theft because of it and we have no recourse against it unless we want to change our state constitution because our Supreme Court mandated it as it was sold as being more fair. All it did was put repeat offenders on the street the next day. My car has been broken into three times since the change. Garbage in, garbage out.

sambull 8y ago
Your anecdote about petty theft is so valid
- deleted 8y ago
  [deleted]
- magduf 8y ago
  Auto theft is by definition not "petty theft".
  - sambull 8y ago
    Except they didnt steal his car just broke into it.
tantalor 8y ago
Er, wouldn't a repeat offense be a red flag for "no release"? It says so right in the article: a system to ascertain a suspect’s risk of flight or committing another crime during the trial process Sounds like your state is doing it wrong?
- magduf 8y ago
  Sounds like his state is definitely doing it wrong. This is like building a car with square wheels and then proclaiming that cars are a terrible idea.
8y ago

mrcactu5 8y ago

Machine Learning Algorithms are severely racist towards defendants. We know, for example, that AI algorithms mistake Black people for primates... but they are no more racist than the criminal justice system at large. And they have a better chance of getting fixed. https://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white-guy-problem.html https://www.nytimes.com/2016/06/26/opinion/sunday/artificial...

drharby 8y ago
Wow citation needed, and a nyt opinion piece is not research proving your claim. "Algorithms are racist" ha!
- tantalor 8y ago
  Here's a few, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... https://www.technologyreview.com/s/607955/inspecting-algorithms-for-bias/ https://www.technologyreview.com/s/607955/inspecting-algorit... http://fortune.com/longform/ai-bias-problem/ http://fortune.com/longform/ai-bias-problem/ https://www.pcmag.com/article/361661/artificial-intelligence-has-a-bias-problem-and-its-our-fau https://www.pcmag.com/article/361661/artificial-intelligence... Ethics and Bias in Machine Learning: A Technical Study of What Makes Us “Good” https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1042&context=jj_etds https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1...
- drharby 8y ago
  Man - to hell with this forum, I ask for evidence on a claim that other posters have pointed out is a misapplication of other research 'webscam algos != cash bail algos' And I get downvoted!? This is tech news, something the general public can get easily manipulated into...and for demanding a degree of reasonable discourse and reference, i get downvoted? Bleh, this place is turning into a place i cant speak relatively freely in.
  - Stratoscope 8y ago

bjt2n3904 8y ago

There's this growing trend of... "Well look, this person in a lab coat with a clipboard did some complicated things with a computer, using all the trendy words like 'algorithms', 'machine learning', and 'big data'! Their results must be very accurate and unbiased, and much better than a human looking at things." I'm more and more convinced that most of these cases are just stuffing an excel spreadsheet into a black box, and shaking it. Hearing that we're using "algorithms" in criminal justice doesn't give me warm and fuzzy.

LiterallyDoge 8y ago
Or maybe we should be electing more people who understand how to read clipboards?
IanDrake 8y ago
>Their results must be very accurate and unbiased That is the assumption that scares me. The algorithm encodes the bias and applies it in every case. This feature will only save you from a shifting bias.
- rectang 8y ago
  Up until now we were saddled with the extremely biased and unjust bond system. Calls to make the algorithms transparent are well taken -- but let's not go back.
  - cgriswald 8y ago
    Your thinking is very dangerous. Even if the algorithm is transparent and fair, that does not necessarily equate to a step forward because of the bias already present in the system. The algorithm will be[0] applied equally, sure, but that does not mean outcomes will be equal. For instance, consider that owning a major asset probably decreases flight risk. That seems fair. But, this means that poor people are probably more of a flight risk than a middle- or upper-class homeowner. So if the algorithm considers this, poor people would be disproportionately affected. Also consider that having family abroad or being from another country probably increases flight risk. Hence, immigrants would be disproportionately affected. But now, when there is an unjust outcome, there isn't even an option to avoid jail time (even if that option is difficult to reach or undesirable); and the human responsible for the decision to incarcerate can simply wash their hands of it, since the algorithm gave the suspect a grade of "D". No need for empathy, sympathy, or guilt. Then there are the pending discussions about what is 'fair' and the answer to those questions is politics... [0] - Edit: Could be

stanleydrew 8y ago

I think there are valid reasons to not trust an algorithm to determine whether someone is a flight risk or danger to the community. But there was already an algorithm that came up with the cash bail amount for a defendant, which was purportedly based on the defendant's flight risk and likelihood of endangering the communoty. I don't pretend to be an expert so I may be wrong but I think the algorithm boiled down to something like "let the judge decide". I think transparency around the algorithm is incredibly important, so anything that explicitly makes the properties of the algorithm part of the debate I think is good.

DevX101 8y ago

As governments start to implement more algorithms in public policy decisions, it becomes increasingly important that ALL algorithms be open sourced for public review unless there is a national security risk. Having a summary of how the algorithm works is necessary but not sufficient.

panic 8y ago

Several of this bill's original co-sponsors ended up opposing it due to the lack of accountability in the risk assessment system and the fact that it gives police even more power over the justice process: https://siliconvalleydebug.org/stories/silicon-valley-de-bug-s-letter-of-opposition-to-california-s-false-bail-reform-bill-sb10 https://siliconvalleydebug.org/stories/silicon-valley-de-bug...

gpm 8y ago

While there are some valid concerns about using algorithms I'm quite happy that they got rid of cash bail. Making presumed innocent people pay to get out of jail has never made any sense to me. For serious crimes, the number of people who say "well I would run for it to try and most of the rest of my life in prison, but I won't because then I (or whoever put the money up for me) won't get my bail back" has to be tiny. For non serious crimes the number of people motivated by bail and not by avoiding spending the rest of their life as a fugitive has to be tiny.

posterboy 8y ago
Isn't the rational that somebody needs to vouch for release and prove trust by risking loss of the money?

CoryG89 8y ago

Shouldn't the "algorithm" need to be published? Surely someone being held prisoner by a computer program should at least have the right to examine and potentially challenge the source code and data being used.

JoeAltmaier 8y ago
Actually, they're being held because they were arrested, right?

pseudolus 8y ago

This points to one of the dilemmas that societies will be hard pressed to confront. Are we capable of accepting decisions underpinned by algorithms even when the outcome is negative in a few cases? Inevitably an individual released as a result of one of these algorithms will commit a criminal act and the argument will arise that had an individual been responsible for making the evaluation and that had cash bail existed then such crime would not have occurred. I think people will have significant difficulty in accepting even the slightest of errors with algorithms (particularly with respect to violent crime) even if the overall result is lower than if humans had been the sole deciders. In many ways it's analogous with the ongoing debates about so-called driverless cars.

anfilt 8y ago

One step forward, and well who knows how many back.

kbos87 8y ago

So we are moving toward a world where one person gets out, while the other has to languish in prison for months or years awaiting trial because an algorithm said so. And the accused aren’t allowed to know what’s happening behind the scenes of the algorithm because we need to protect the competitive advantage of the vendor who sold the software to the state. This is total and complete insanity and elected officials should pay with their jobs and their reputations for even suggesting something so unjust.

rectang 8y ago
Where is your outrage about how poor people are wronged by the current system?
- kbos87 8y ago
  I am absolutely outraged about the way the current system treats and traps the poor. But we need to be clear that this isn’t a fix; it’s a further obfuscation of the problem. I have little doubt that said algorithms will continue to wrong the poor at similar or greater rates - the only difference is that it will be significantly harder to detect.