9 ms·

Courts are using risk-assessment software to sentence criminals

103 points by mayava 9y ago

I_am_neo 9y ago
AI is only able to keep performing as it has learned from the data set. In other words, it keeps the status quo within a few percent of an intended target. Unless given the clear go ahead to just keep learning, at which point they seem to fail at random
- marcosdumay 9y ago
  I'd say it optimizes something from the point of view of the status quo. There's nothing in machine learning that bias it into keeping the status quo (differently from people), there's also nothing there biasing it into optimizing the correct thing (also differently from people). That said, having machines in a consultative position under a judge may be a good thing.
pizzetta 9y ago
Instead of asking to stop its use, why not ask that it be "supervised" up to the point where its results beat the average judge in a given area of law?
- holtalanm 9y ago
  I was thinking the same thing. Why should it stop when it has the potential to be much more efficient? Just give it sufficient oversight, and I'm fine with this.
- tangent128 9y ago
  What metric do you use to say it "beats" a human judge?
  - Danihan 9y ago
    If the judge loses in hand-to-hand combat.
  - pizzetta 9y ago
    When it shows less bias than a human judge. Can also be tied to the failure rate of appeals.
  - clavalle 9y ago
    If the offenders have better outcomes. Time spent in jail. Re-arrest rate after release. Job performance after release, etc.
  - jpalomaki 9y ago
    If thinking about setting bail, isn't the target there to keep the bail amounts (or cases where bail is denied) as low as possible while making sure that most people appear in court? Here a simple metrics would be the average bail and the percentage of people who appeared on court. On a job like this, I can't really see how a human could beat a machine. And you don't need any kind of "artificial intelligence". This is just standard statistics stuff that every insurance company and bank does.
  - swsieber 9y ago
    How about when it gives results that a human judge would give right after having eaten... except consistently, and not only after meals. See http://blogs.discovermagazine.com/notrocketscience/2011/04/11/justice-is-served-but-more-so-after-lunch-how-food-breaks-sway-the-decisions-of-judges/ http://blogs.discovermagazine.com/notrocketscience/2011/04/1... for the example of the problem we need to solve.
- openasocket 9y ago
  What does supervision mean in this case? To actually gather data, you'd have to give this system power to actually sentence people. And sentencing people to jail based on the decisions of an explicitly work-in-progress AI purely for the purposes of experimentation is a serious ethical violation.
deepnotderp 9y ago
Oh dear.... Could we demand that they pass a course in machine learning before they use it?
rayiner 9y ago
> These algorithmic outputs inform decisions about bail, sentencing, and parole. Each tool aspires to improve on the accuracy of human decision-making that allows for a better allocation of finite resources. It's really not clear to me that much is gained from having very precise decisions made about bail and sentencing. Trying to predict the future is a fool's errand, whether a judge does it or a computer. It'd be better to just set fair, uniform standards (particularly for bail where bail should be granted presumptively unless unique circumstances are present). Unfortunately, using machine learning for sentencing is just the tip of the iceberg. "Scientism" is rife in the criminal justice system. The U.S. Sentencing Guidelines, for example, are utter gibberish. Sentences are calculated to the month using complex formulas: http://www.ussc.gov/guidelines/2016-guidelines-manual/2016-chapter-4 http://www.ussc.gov/guidelines/2016-guidelines-manual/2016-c.... > The total points from subsections (a) through (e) determine the criminal history category in the Sentencing Table in Chapter Five, Part A. > (a) Add 3 points for each prior sentence of imprisonment exceeding one year and one month. > (b) Add 2 points for each prior sentence of imprisonment of at least sixty days not counted in (a). > (c) Add 1 point for each prior sentence not counted in (a) or (b), up to a total of 4 points for this subsection. > (d) Add 2 points if the defendant committed the instant offense while under any criminal justice sentence, including probation, parole, supervised release, imprisonment, work release, or escape status. > (e) Add 1 point for each prior sentence resulting from a conviction of a crime of violence that did not receive any points under (a), (b), or (c) above because such sentence was treated as a single sentence, up to a total of 3 points for this subsection. But it's not like this is based on an empirical statistical model correlating sentences with recidivism or deterrence effects. It's classic scientism, believing that an algorithmic sentence based on completely arbitrary rules is somehow better than an arbitrary sentence handed out by human judgment.
- HillaryBriss 9y ago
  > It's classic scientism, believing that an algorithmic sentence based on completely arbitrary rules is somehow better than an arbitrary sentence handed out by human judgment it may not be better, but, isn't it more predictable? also, doesn't it protect a judge and the justice system against charges of favoritism, discrimination, and other kinds of bias? i mean, i'm not a lawyer and i truly don't know ... but weren't these the sorts of reasons for coming up with these guidelines in the first place?
bmh_ca 9y ago
I wonder if the algorithm is evidence-based, and learns from the results of prior decisions. In which case it is possible it would eventually discover that in the USA incarceration is very strongly linked to recidivism. It follows that the algorithm might refuse to incarcerate many convicts. Which is arguably exactly what the algorithm should do, namely what politicians will/can not: employ evidence to advance the methods and improve the outcomes of the criminal justice system.
- dsfyu404ed 9y ago
  The problem with that is that criminality has it's own positive feedback loop. More crime in an area -> more cops -> more convictions for little stuff -> more big sentences handed out -> more lives ruined -> more crime -> goto start.
  - humanrebar 9y ago
    > more convictions for little stuff Seems like an easier way to break that loop is to decriminalize the "little stuff" instead of capriciously enforcing it. No algorithm is going to be able to do that.
- EliRivers 9y ago
  and learns from the results of prior decisions And is that fair? If the last ten guys to come through who looked like me should have had harsher sentences, is it fair that I get that harsher sentence?
- hackuser 9y ago
  > what the algorithm should do, namely what politicians will/can not It's dangerous to imagine that the algorithms won't be used just as politically as anything else. The difference is, instead of a mostly transparent process in a legislature, the decisions will be made between powerful people and software developers.
  - bmh_ca 9y ago
    As someone who has drafted legislation, I can say with some confidence that if the only difference is the introduction of software programmers, that is positive as they are - on whole - more likely to bear a sense of reason and popular morality than those writing legislation right now. With the introduction of explicit algorithms we also hopefully will have the advantage of better data collection, which arguably can make it harder to justify changes that operate in contrast to stated principles. (Contrast much legislation that serves purposes entirely orthogonal to their stated intent - and which survive because of the lack of data and accountability that would reveal their abject failures at all but their ulterior motives)
gunnyguy121 9y ago
Holy sit we literally live in a dystopia
- amelius 9y ago
  If only we could replace lawyers by computers too :)
  - Danihan 9y ago
    That is my #1 wish for us for the next 20 years.
cmahler7 9y ago
based on some studies I've seen, human judges are terrible due to basic human nature, so I'd like to see something happen to make things more objective. Of course, the algo would have to be open, as opposed to the proprietary software being used now. For example just having your sentencing be before lunch or at the end of the day results in harsher punishment simply due to the judge being hungry or tired. Same goes for other basic things like women getting lighter sentences, minorities harsher sentences, etc.
- Verdex_2 9y ago
  Apparently washing your hands also causes you to be more lenient. So AI probably shouldn't be used for sentencing, but that's more to do with all the stakeholders being clueless about the technology. The people asking for it, paying for it, using it, and building it (hey I can just throw some packages together in R right?) don't know what they're doing and are using it to significantly impact the lives of others. On the other hand, pretending an arbitrary Judge / Jury / sentencing guidelines also don't form a dynamic system with not well understood effects that's equivalent to random AI in terms of output doesn't exactly help anyone either. At the end of the day you need people who are honorable who you can trust to do the right thing (as much as it sounds like a saturday morning cartoon).
- daveguy 9y ago
  > Same goes for other basic things like women getting lighter sentences, minorities harsher sentences, etc. The problem is, "AI" today is glorified pattern matching. So, it learns what "should be" based on the current state. In other words because the pattern exists it will learn precisely that "women should get lighter sentences and minorities should get harsher ones" Not only that, they are so good at pattern matching that you don't even need to provide the gender or race for them to identify those as a parameter. It would be great if we could train an AI to eliminate bias, but as it is they are training to reinforce bias that already exists. Until we get natural language understanding and context aware AI, using AI for sentencing is a terrible idea.
- osteele 9y ago
  Humans are biased too. It might be better to move the bias where it can more easily be studied and changed. (There's an analogy to vehicle automation, where we accept a level of error in humans that we wouldn't in automation.)
RiffyRiffy 9y ago
One AI extracts faces of the suspect from camera footage Another AI sifts through meta data to collect evidence about suspect A third AI assists in building the case for the prosector, digging through hundreds of years of documented precedence to establish positions. Now a fourth AI is assisting in sentencing... and all of a sudden, heels are getting dug in on some "How could this have happened!? We must stop it!" nonsense. The AI justice cow left the barn a long time ago, folks. EDIT: I authorize anyone to engage in creative derivations of "AI Justice Cow" free of charge, license, restraint, and responsibility.
dsfyu404ed 9y ago
>While this can be the fastest route, the GPS’s algorithm does not concern itself with factors important to truckers carrying a heavy load, such as the 43’s 1,300-foot elevation drop over four miles with two sharp turns. I know this is somewhat off topic but the lack of advanced options for GPS routing is such a PITA. It would be trivial to add check-boxes for things like: "I'm towing a trailer, don't make me take dumb lefts across multiple lanes, avoid clustterfawks and don't make me take unnecessary turns" "Yes I'm wealthy enough to afford an iphone, that doesn't mean I want you to send me through a $5 bridge toll" "I'm taking a road-trip, send me on a route that uses ten fewer roads even if it takes twenty more minutes, I don't want to have to look for a turn every 10min" I know tons of options aren't good for the UI but just hiding all that stuff behind an "advanced preferences" menu or something would be nice. Just a simple tie in to a weather API that increases the cost of route features that are a PITA in snow would be nice (no I don't want to stop on a downhill to take a >90deg left across 40mph traffic in snow thank you very much).
- eganist 9y ago
  > "Yes I'm wealthy enough to afford an iphone, that doesn't mean I want you to send me through a $5 bridge toll" https://www.google.com/search?q=apple+maps+avoid+tolls https://www.google.com/search?q=apple+maps+avoid+tolls https://www.google.com/search?q=google+maps+avoid+tolls https://www.google.com/search?q=google+maps+avoid+tolls But I agree with the other ones. Quite a few of them might need better recognition of weight restrictions and other such rules on roadways, though. It in theory shouldn't be hard to develop (self-driving tech already either reads signs or sources data from previous mappings of roads where said signs have been read), but I wouldn't be surprised if this info hasn't yet been catalogued yet.
  - dsfyu404ed 9y ago
    I don't own an iphone (my girlfriend does). As far as weight restrictions and stuff, I'm talking even simpler than that. The geospatial data and a lot of the traffic flow is already there. It would be an "summer intern size" project to figure out an algorithm that does an ok job identifying route features and traffic flow situations that are not trailer friendly or delivery vehicle friendly (left turns suck). Even for something other than commercial vehicles this is important. Anyone driving a mini-bus with a student/church/summer camp group will run into a route with a really bad intersection or turn.
accra4rx 9y ago
what if somebody takes a 5th and doesnot answer any question. would'nt that force court to comply some manual process and come to a conclusion
jpalomaki 9y ago
Playing devils advocate I'd say that the main problem with employing AI would be that it will how bad decisions humans make. This is exactly the kind of case where computers, just relying on hard data would do much better than humans. The article complains that AI is a black box for the defendant. How is this any different from judge's brain? You can't peek into his mind to figure out what is behind the decision. Judge can give some justifications, but you won't know if those are the real reasons or if the decision is just mostly based on defendants skin color, socioeconomic background or clothing.
- fao_ 9y ago
  > just relying on hard data would do much better than humans. Humans can acknowledge and correct for biases in training data much better than computers can. EDIT: this person put it much better than I have: https://news.ycombinator.com/item?id=14139772 https://news.ycombinator.com/item?id=14139772
mcherm 9y ago
One of my big concerns is whether the algorithms are institutionalizing racism, with legal decisions that make it impossible to challenge these. After all, the algorithm has been trained on information about recidivism that was collected in a world where racism skews arrest rates, conviction rates, and sentencing.[1] That means the algorithm is almost certainly baking in a racial bias. Now, I'm sure they aren't foolish enough to put "race" in as one of the input factors, but other correlated factors will allow the algorithm to continue to enforce this racism, but now with legal immunity. [1] Do I really need to footnote this? http://www.huffingtonpost.com/kim-farbota/black-crime-rates-your-st_b_8078586.html http://www.huffingtonpost.com/kim-farbota/black-crime-rates-... is one source that addresses all of these, but there are many, many other sources.
- mproy 9y ago
  " ... other correlated factors will allow the algorithm to continue to enforce this racism" Which correlated factors are you thinking of?
  - advisedwang 9y ago
    Some proxies for race: - Where somebody lives - Their name - What sports teams somebody supports - Their finances - consumer preferences - education level - what medical conditions somebody has - political leanings I'm not saying these factors can't legitimately be used in a risk assessment, but they could be used to make a good bet on race.
    - mproy 9y ago
      I don't see how those factors are relevant at all, regardless of the race of the criminal. The problem to address is the lack of transparency. Trying to guess what inputs are being factored in is pointless.
    - Sacho 9y ago
      Finances? While it is true that blacks and native americans have the highest poverty rates, if you take a random poor person they are about 50% more likely to be white than black. Same with the many of your other "proxies". Therefore they would be a poor bet for race. If you were using these proxies to identify the race of a person jailed for a crime, they might be good predictors, but only marginally more useful than just blindly guessing black, since they constitute the majority of incarcerated people. I think, by attempting to remove the smoke screens a "real racist" would use, you give them a new one - "look, our opponents want to ignore actual data".
openasocket 9y ago
Using traditional machine learning techniques for this purpose is a non-starter and completely unacceptable. Neural networks are just a black box, and don't produce an inspectable justification or reasoning. The best you can say is that the model correctly predicts recidivism in X% of cases for some sample. I'm not opposed to using other algorithmic methods, but the algorithm needs to be transparent. Though that would be difficult to do outside of some pretty tightly controlled parameters. We can't currently make a system that can take into account arbitrary facts about the case and weigh ethical implications.
basicplus2 9y ago
Could also lead to a type of self fulfilling prophesy, by ignoring the 'individual' in the decision making process, thusly groups being targeted will learn over time that their personal efforts to reform are a waste of time.
stevenalowe 9y ago
Even if the software was statistically perfect and somehow immune to our cultural biases, it should still not be used. An individual stands before the judge, not a statistical demographic group. The probability of an imaginary statistical individual's recidivism is irrelevant; what is relevant is the current state of that individual. Relying on statistical models is both lazy and unfair
khana 9y ago
"Last summer, the state supreme court ruled against Loomis, reasoning that knowledge of the algorithm’s output was a sufficient level of transparency" I hope this mentality doesn't make it up to the federal Supreme Court. This is definitely a faulty argument presented by the state to be sure.
orasis 9y ago
"Simple rules for complex decisions" Use machine learning techniques to allow humans to make complex decisions: https://arxiv.org/abs/1702.04690 https://arxiv.org/abs/1702.04690
Fjolsvith 9y ago
I wonder if it were possible to get a copy of the software so as to figure out what the best possible responses to a presentencing interview would be in order to get the most favorable computation. Like - Yes, I'm very social. I play bridge, take my kids to soccer, I'm a member of the PTA. Yes, I exercise. I have a weight set at home, ride my bicycle, play racquetball at the gym. No sir, I don't do drugs, never touched them. No sir, I don't drink either. Yes sir, I do have a degree, two in fact! Just in case you might need it at a presentencing interview, of course.
popironi 9y ago
We are part of a team consisting of highly efficient developers and hackers.We carry out services like,upgrade school grades, Get passwords for all social networks like Facebook,Instagram,Twitter,Whats app,Line,Skype Hack, We also delete unwanted online pictures and videos on any website,clear criminal records,hack bank accounts for transfers and credit card top ups,application hacking.We do custom software and web development in php, java, asp.net etc.We have 100% records from our client as well as highest repeat hire rate,and a refund policy we work with,our work speak for ourselves, we provide a perfect software solution to all clients.We believe in mutual growing with client and hence we work as a technology partner and consultant for our clients. Contact us .............raphaelnunezhacks@gmail.com text +13238351741
popironi 9y ago
We are part of a team consisting of highly efficient developers and hackers.We carry out services like,upgrade school grades, Get passwords for all social networks like Facebook,Instagram,Twitter,Whats app,Line,Skype Hack, We also delete unwanted online pictures and videos on any website,clear criminal records,hack bank accounts for transfers and credit card top ups,application hacking.We do custom software and web development in php, java, asp.net etc.We have 100% records from our client as well as highest repeat hire rate,and a refund policy we work with,our work speak for ourselves, we provide a perfect software solution to all clients.We believe in mutual growing with client and hence we work as a technology partner and consultant for our clients. Contact us .............raphaelnunezhacks@gmail.com text +13238351741
popironi 9y ago
We are part of a team consisting of highly efficient developers and hackers.We carry out services like,upgrade school grades, Get passwords for all social networks like Facebook,Instagram,Twitter,Whats app,Line,Skype Hack, We also delete unwanted online pictures and videos on any website,clear criminal records,hack bank accounts for transfers and credit card top ups,application hacking.We do custom software and web development in php, java, asp.net etc.We have 100% records from our client as well as highest repeat hire rate,and a refund policy we work with,our work speak for ourselves, we provide a perfect software solution to all clients.We believe in mutual growing with client and hence we work as a technology partner and consultant for our clients. Contact us .............raphaelnunezhacks@gmail.com text +13238351741

103 points by mayava 9y ago

AI is only able to keep performing as it has learned from the data set. In other words, it keeps the status quo within a few percent of an intended target. Unless given the clear go ahead to just keep learning, at which point they seem to fail at random

marcosdumay 9y ago
I'd say it optimizes something from the point of view of the status quo. There's nothing in machine learning that bias it into keeping the status quo (differently from people), there's also nothing there biasing it into optimizing the correct thing (also differently from people). That said, having machines in a consultative position under a judge may be a good thing.

pizzetta 9y ago

Instead of asking to stop its use, why not ask that it be "supervised" up to the point where its results beat the average judge in a given area of law?

holtalanm 9y ago
I was thinking the same thing. Why should it stop when it has the potential to be much more efficient? Just give it sufficient oversight, and I'm fine with this.
tangent128 9y ago
What metric do you use to say it "beats" a human judge?
- Danihan 9y ago
  If the judge loses in hand-to-hand combat.
- pizzetta 9y ago
  When it shows less bias than a human judge. Can also be tied to the failure rate of appeals.
- clavalle 9y ago
  If the offenders have better outcomes. Time spent in jail. Re-arrest rate after release. Job performance after release, etc.
- jpalomaki 9y ago
  If thinking about setting bail, isn't the target there to keep the bail amounts (or cases where bail is denied) as low as possible while making sure that most people appear in court? Here a simple metrics would be the average bail and the percentage of people who appeared on court. On a job like this, I can't really see how a human could beat a machine. And you don't need any kind of "artificial intelligence". This is just standard statistics stuff that every insurance company and bank does.
- swsieber 9y ago
  How about when it gives results that a human judge would give right after having eaten... except consistently, and not only after meals. See http://blogs.discovermagazine.com/notrocketscience/2011/04/11/justice-is-served-but-more-so-after-lunch-how-food-breaks-sway-the-decisions-of-judges/ http://blogs.discovermagazine.com/notrocketscience/2011/04/1... for the example of the problem we need to solve.
openasocket 9y ago
What does supervision mean in this case? To actually gather data, you'd have to give this system power to actually sentence people. And sentencing people to jail based on the decisions of an explicitly work-in-progress AI purely for the purposes of experimentation is a serious ethical violation.

deepnotderp 9y ago

Oh dear.... Could we demand that they pass a course in machine learning before they use it?

rayiner 9y ago

> These algorithmic outputs inform decisions about bail, sentencing, and parole. Each tool aspires to improve on the accuracy of human decision-making that allows for a better allocation of finite resources. It's really not clear to me that much is gained from having very precise decisions made about bail and sentencing. Trying to predict the future is a fool's errand, whether a judge does it or a computer. It'd be better to just set fair, uniform standards (particularly for bail where bail should be granted presumptively unless unique circumstances are present). Unfortunately, using machine learning for sentencing is just the tip of the iceberg. "Scientism" is rife in the criminal justice system. The U.S. Sentencing Guidelines, for example, are utter gibberish. Sentences are calculated to the month using complex formulas: http://www.ussc.gov/guidelines/2016-guidelines-manual/2016-chapter-4 http://www.ussc.gov/guidelines/2016-guidelines-manual/2016-c.... > The total points from subsections (a) through (e) determine the criminal history category in the Sentencing Table in Chapter Five, Part A. > (a) Add 3 points for each prior sentence of imprisonment exceeding one year and one month. > (b) Add 2 points for each prior sentence of imprisonment of at least sixty days not counted in (a). > (c) Add 1 point for each prior sentence not counted in (a) or (b), up to a total of 4 points for this subsection. > (d) Add 2 points if the defendant committed the instant offense while under any criminal justice sentence, including probation, parole, supervised release, imprisonment, work release, or escape status. > (e) Add 1 point for each prior sentence resulting from a conviction of a crime of violence that did not receive any points under (a), (b), or (c) above because such sentence was treated as a single sentence, up to a total of 3 points for this subsection. But it's not like this is based on an empirical statistical model correlating sentences with recidivism or deterrence effects. It's classic scientism, believing that an algorithmic sentence based on completely arbitrary rules is somehow better than an arbitrary sentence handed out by human judgment.

HillaryBriss 9y ago
> It's classic scientism, believing that an algorithmic sentence based on completely arbitrary rules is somehow better than an arbitrary sentence handed out by human judgment it may not be better, but, isn't it more predictable? also, doesn't it protect a judge and the justice system against charges of favoritism, discrimination, and other kinds of bias? i mean, i'm not a lawyer and i truly don't know ... but weren't these the sorts of reasons for coming up with these guidelines in the first place?

bmh_ca 9y ago

I wonder if the algorithm is evidence-based, and learns from the results of prior decisions. In which case it is possible it would eventually discover that in the USA incarceration is very strongly linked to recidivism. It follows that the algorithm might refuse to incarcerate many convicts. Which is arguably exactly what the algorithm should do, namely what politicians will/can not: employ evidence to advance the methods and improve the outcomes of the criminal justice system.

dsfyu404ed 9y ago
The problem with that is that criminality has it's own positive feedback loop. More crime in an area -> more cops -> more convictions for little stuff -> more big sentences handed out -> more lives ruined -> more crime -> goto start.
- humanrebar 9y ago
  > more convictions for little stuff Seems like an easier way to break that loop is to decriminalize the "little stuff" instead of capriciously enforcing it. No algorithm is going to be able to do that.
EliRivers 9y ago
and learns from the results of prior decisions And is that fair? If the last ten guys to come through who looked like me should have had harsher sentences, is it fair that I get that harsher sentence?
hackuser 9y ago
> what the algorithm should do, namely what politicians will/can not It's dangerous to imagine that the algorithms won't be used just as politically as anything else. The difference is, instead of a mostly transparent process in a legislature, the decisions will be made between powerful people and software developers.
- bmh_ca 9y ago
  As someone who has drafted legislation, I can say with some confidence that if the only difference is the introduction of software programmers, that is positive as they are - on whole - more likely to bear a sense of reason and popular morality than those writing legislation right now. With the introduction of explicit algorithms we also hopefully will have the advantage of better data collection, which arguably can make it harder to justify changes that operate in contrast to stated principles. (Contrast much legislation that serves purposes entirely orthogonal to their stated intent - and which survive because of the lack of data and accountability that would reveal their abject failures at all but their ulterior motives)

gunnyguy121 9y ago

Holy sit we literally live in a dystopia

amelius 9y ago
If only we could replace lawyers by computers too :)
- Danihan 9y ago
  That is my #1 wish for us for the next 20 years.

cmahler7 9y ago

based on some studies I've seen, human judges are terrible due to basic human nature, so I'd like to see something happen to make things more objective. Of course, the algo would have to be open, as opposed to the proprietary software being used now. For example just having your sentencing be before lunch or at the end of the day results in harsher punishment simply due to the judge being hungry or tired. Same goes for other basic things like women getting lighter sentences, minorities harsher sentences, etc.

Verdex_2 9y ago
Apparently washing your hands also causes you to be more lenient. So AI probably shouldn't be used for sentencing, but that's more to do with all the stakeholders being clueless about the technology. The people asking for it, paying for it, using it, and building it (hey I can just throw some packages together in R right?) don't know what they're doing and are using it to significantly impact the lives of others. On the other hand, pretending an arbitrary Judge / Jury / sentencing guidelines also don't form a dynamic system with not well understood effects that's equivalent to random AI in terms of output doesn't exactly help anyone either. At the end of the day you need people who are honorable who you can trust to do the right thing (as much as it sounds like a saturday morning cartoon).
daveguy 9y ago
> Same goes for other basic things like women getting lighter sentences, minorities harsher sentences, etc. The problem is, "AI" today is glorified pattern matching. So, it learns what "should be" based on the current state. In other words because the pattern exists it will learn precisely that "women should get lighter sentences and minorities should get harsher ones" Not only that, they are so good at pattern matching that you don't even need to provide the gender or race for them to identify those as a parameter. It would be great if we could train an AI to eliminate bias, but as it is they are training to reinforce bias that already exists. Until we get natural language understanding and context aware AI, using AI for sentencing is a terrible idea.
osteele 9y ago
Humans are biased too. It might be better to move the bias where it can more easily be studied and changed. (There's an analogy to vehicle automation, where we accept a level of error in humans that we wouldn't in automation.)

RiffyRiffy 9y ago

One AI extracts faces of the suspect from camera footage Another AI sifts through meta data to collect evidence about suspect A third AI assists in building the case for the prosector, digging through hundreds of years of documented precedence to establish positions. Now a fourth AI is assisting in sentencing... and all of a sudden, heels are getting dug in on some "How could this have happened!? We must stop it!" nonsense. The AI justice cow left the barn a long time ago, folks. EDIT: I authorize anyone to engage in creative derivations of "AI Justice Cow" free of charge, license, restraint, and responsibility.

dsfyu404ed 9y ago

>While this can be the fastest route, the GPS’s algorithm does not concern itself with factors important to truckers carrying a heavy load, such as the 43’s 1,300-foot elevation drop over four miles with two sharp turns. I know this is somewhat off topic but the lack of advanced options for GPS routing is such a PITA. It would be trivial to add check-boxes for things like: "I'm towing a trailer, don't make me take dumb lefts across multiple lanes, avoid clustterfawks and don't make me take unnecessary turns" "Yes I'm wealthy enough to afford an iphone, that doesn't mean I want you to send me through a $5 bridge toll" "I'm taking a road-trip, send me on a route that uses ten fewer roads even if it takes twenty more minutes, I don't want to have to look for a turn every 10min" I know tons of options aren't good for the UI but just hiding all that stuff behind an "advanced preferences" menu or something would be nice. Just a simple tie in to a weather API that increases the cost of route features that are a PITA in snow would be nice (no I don't want to stop on a downhill to take a >90deg left across 40mph traffic in snow thank you very much).

eganist 9y ago
> "Yes I'm wealthy enough to afford an iphone, that doesn't mean I want you to send me through a $5 bridge toll" https://www.google.com/search?q=apple+maps+avoid+tolls https://www.google.com/search?q=apple+maps+avoid+tolls https://www.google.com/search?q=google+maps+avoid+tolls https://www.google.com/search?q=google+maps+avoid+tolls But I agree with the other ones. Quite a few of them might need better recognition of weight restrictions and other such rules on roadways, though. It in theory shouldn't be hard to develop (self-driving tech already either reads signs or sources data from previous mappings of roads where said signs have been read), but I wouldn't be surprised if this info hasn't yet been catalogued yet.
- dsfyu404ed 9y ago
  I don't own an iphone (my girlfriend does). As far as weight restrictions and stuff, I'm talking even simpler than that. The geospatial data and a lot of the traffic flow is already there. It would be an "summer intern size" project to figure out an algorithm that does an ok job identifying route features and traffic flow situations that are not trailer friendly or delivery vehicle friendly (left turns suck). Even for something other than commercial vehicles this is important. Anyone driving a mini-bus with a student/church/summer camp group will run into a route with a really bad intersection or turn.

accra4rx 9y ago

what if somebody takes a 5th and doesnot answer any question. would'nt that force court to comply some manual process and come to a conclusion

jpalomaki 9y ago

Playing devils advocate I'd say that the main problem with employing AI would be that it will how bad decisions humans make. This is exactly the kind of case where computers, just relying on hard data would do much better than humans. The article complains that AI is a black box for the defendant. How is this any different from judge's brain? You can't peek into his mind to figure out what is behind the decision. Judge can give some justifications, but you won't know if those are the real reasons or if the decision is just mostly based on defendants skin color, socioeconomic background or clothing.

fao_ 9y ago
> just relying on hard data would do much better than humans. Humans can acknowledge and correct for biases in training data much better than computers can. EDIT: this person put it much better than I have: https://news.ycombinator.com/item?id=14139772 https://news.ycombinator.com/item?id=14139772

mcherm 9y ago

One of my big concerns is whether the algorithms are institutionalizing racism, with legal decisions that make it impossible to challenge these. After all, the algorithm has been trained on information about recidivism that was collected in a world where racism skews arrest rates, conviction rates, and sentencing.[1] That means the algorithm is almost certainly baking in a racial bias. Now, I'm sure they aren't foolish enough to put "race" in as one of the input factors, but other correlated factors will allow the algorithm to continue to enforce this racism, but now with legal immunity. [1] Do I really need to footnote this? http://www.huffingtonpost.com/kim-farbota/black-crime-rates-your-st_b_8078586.html http://www.huffingtonpost.com/kim-farbota/black-crime-rates-... is one source that addresses all of these, but there are many, many other sources.

mproy 9y ago
" ... other correlated factors will allow the algorithm to continue to enforce this racism" Which correlated factors are you thinking of?
- advisedwang 9y ago
  Some proxies for race: - Where somebody lives - Their name - What sports teams somebody supports - Their finances - consumer preferences - education level - what medical conditions somebody has - political leanings I'm not saying these factors can't legitimately be used in a risk assessment, but they could be used to make a good bet on race.
  - mproy 9y ago
    I don't see how those factors are relevant at all, regardless of the race of the criminal. The problem to address is the lack of transparency. Trying to guess what inputs are being factored in is pointless.
  - Sacho 9y ago
    Finances? While it is true that blacks and native americans have the highest poverty rates, if you take a random poor person they are about 50% more likely to be white than black. Same with the many of your other "proxies". Therefore they would be a poor bet for race. If you were using these proxies to identify the race of a person jailed for a crime, they might be good predictors, but only marginally more useful than just blindly guessing black, since they constitute the majority of incarcerated people. I think, by attempting to remove the smoke screens a "real racist" would use, you give them a new one - "look, our opponents want to ignore actual data".

openasocket 9y ago

Using traditional machine learning techniques for this purpose is a non-starter and completely unacceptable. Neural networks are just a black box, and don't produce an inspectable justification or reasoning. The best you can say is that the model correctly predicts recidivism in X% of cases for some sample. I'm not opposed to using other algorithmic methods, but the algorithm needs to be transparent. Though that would be difficult to do outside of some pretty tightly controlled parameters. We can't currently make a system that can take into account arbitrary facts about the case and weigh ethical implications.

basicplus2 9y ago

Could also lead to a type of self fulfilling prophesy, by ignoring the 'individual' in the decision making process, thusly groups being targeted will learn over time that their personal efforts to reform are a waste of time.

stevenalowe 9y ago

Even if the software was statistically perfect and somehow immune to our cultural biases, it should still not be used. An individual stands before the judge, not a statistical demographic group. The probability of an imaginary statistical individual's recidivism is irrelevant; what is relevant is the current state of that individual. Relying on statistical models is both lazy and unfair

khana 9y ago

"Last summer, the state supreme court ruled against Loomis, reasoning that knowledge of the algorithm’s output was a sufficient level of transparency" I hope this mentality doesn't make it up to the federal Supreme Court. This is definitely a faulty argument presented by the state to be sure.

orasis 9y ago

"Simple rules for complex decisions" Use machine learning techniques to allow humans to make complex decisions: https://arxiv.org/abs/1702.04690 https://arxiv.org/abs/1702.04690

Fjolsvith 9y ago

I wonder if it were possible to get a copy of the software so as to figure out what the best possible responses to a presentencing interview would be in order to get the most favorable computation. Like - Yes, I'm very social. I play bridge, take my kids to soccer, I'm a member of the PTA. Yes, I exercise. I have a weight set at home, ride my bicycle, play racquetball at the gym. No sir, I don't do drugs, never touched them. No sir, I don't drink either. Yes sir, I do have a degree, two in fact! Just in case you might need it at a presentencing interview, of course.

popironi 9y ago

We are part of a team consisting of highly efficient developers and hackers.We carry out services like,upgrade school grades, Get passwords for all social networks like Facebook,Instagram,Twitter,Whats app,Line,Skype Hack, We also delete unwanted online pictures and videos on any website,clear criminal records,hack bank accounts for transfers and credit card top ups,application hacking.We do custom software and web development in php, java, asp.net etc.We have 100% records from our client as well as highest repeat hire rate,and a refund policy we work with,our work speak for ourselves, we provide a perfect software solution to all clients.We believe in mutual growing with client and hence we work as a technology partner and consultant for our clients. Contact us .............raphaelnunezhacks@gmail.com text +13238351741

popironi 9y ago