8 ms·

I could not disagree more with these comments. Psychologists are just now starting to study the phenomenon of "algorithm aversion", where people irrationally tr

I could not disagree more with these comments. Psychologists are just now starting to study the phenomenon of "algorithm aversion", where people irrationally trust human judgement far more than algorithms. Even after watching an algorithm do far better in many examples. The reality is humans are far worse. We are biased by all sorts of things. Unattractive people were found to get twice as long sentences as attractive ones. Judges were found to give much harsher sentences right before lunch time, when they were hungry. Doing interviews was found to decrease the performance of human judges, in domains like hiring and determining parolle. As opposed to just looking at the facts. Even very simple statistical algorithms far outperform humans in almost every domain. As early as 1928, a simple statistical rule predicted recidivism better than prison psychologists. They predict the success of college students, job applicants, outcomes of medical treatment, etc, far better than human experts. Human experts never even beat the most basic statistical baseline. You should never ever trust human judges. They are neither fair nor accurate. In such an important domain as this, where better predictions reduce the time people spend in prison and crime, there is no excuse not to use them. Anything that gets low risk people out of prison is good. I believe that any rules that apply to algorithms should apply to humans too. We are algorithms too after all. If algorithms have to be blind to race and gender, so should human judges. If economic information is bad to use, humans should be blind to it also. If we have a right to see why an algorithm made a decision the way it did, we should be able to inspect human brains to. Perhaps put judges and parolle officers in an MRI.

coldtea 10y ago
>I could not disagree more with these comments. Psychologists are just now starting to study the phenomenon of "algorithm aversion", where people irrationally trust human judgement far more than algorithms. Even after watching an algorithm do far better in many examples. Those algorithms are also designed by humans and codify biases and prejudices and injustices that are considered OK by society into them. Besides, it's like going for a run. Other might might be far better at it, but we want to do it ourselves, even badly, than just be spectators in our own life. Same for humans becoming spectators while algorithms run business "better". One quality that's not accounted for in accessing better is "more participative".
- diogenes78 10y ago
  > Those algorithms are also designed by humans But the humans would have to explicitly build the biases into the algorithm. Many human biases are unconscious, so we aren't aware that our conclusions are biased by race, a subject's attractiveness, etc. Going with an algorithm would minimize unconscious biases, and prevent us from reaching a conclusion based on emotion and then just back-fitting a rational-sounding argument.
  - coldtea 10y ago
    >But the humans would have to explicitly build the biases into the algorithm. Explicitly as in coding them, but not necessarily as doing it consciously. After all biases are just how people see reality, so it would be natural to code based on one's biases. Also it's less the biases (that are inevitable), and more the blind following of biases (algorithmically) that I'm concerned about. >Going with an algorithm would minimize unconscious biases, and prevent us from reaching a conclusion based on emotion The latter is not necessarily (or even commonly) bad, it's more of a citation needed kind of thing. I'd rather we DO reach conclusions based on emotions. Naked logic sees everything as black and white, it's emotions that make civilisation nuanced (including empathy, pity, gut instinct, abhorrence, etc).
    - sdenton4 10y ago
      Racists have emotions, too, and often make decisions based on them. 'Feeling' that someone is suspicious is a classic cover for racial profiling.
      - coldtea 10y ago
        >Racists have emotions, too, and often make decisions based on them And knives can be used both for murdering people and spreading jam on your bread. The answer to "racist emotions" are emotions of empathy and love towards the other, and not some objective logic that one can follow to fix racism. After all, one could always use objective logic to justify racism if one wants to -- e.g. "blacks are involved in a higher percentage of violent crimes compared to their population". That's a totally objective statement, it just skips trying to find the reasons why that might be so, and uses the numbers to justify thinking of them as violent inherently.
rhizome 10y ago
We are biased by all sorts of things. Right, so who makes the algorithms?
- z3r0_29a 10y ago
  You point of reference, your past experience, your his-tory. We are seriously in bad shape, not only do we have no right to privacy, and can we be prosecuted for mere words alone.. Now they will try to prove we are guilty prior to committing an offense, and we are doomed.
- Houshalter 10y ago
  Objective data.
  - rhizome 10y ago
    Who decides whether it's objective?
Smerity 10y ago
Stating that method A is problematic does not automatically mean method B is better. > The reality is humans are far worse Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. If you give a machine learning model a dataset where one feature appears entirely indicative (remember: correlation is not causation), it can overfit to that, even if that does not reflect reality. I highly recommend reading "How big data is unfair: understanding unintended sources of unfairness in data driven decision making"[1], by Moritz Hardt, a Google machine learning researcher who has published on the topic (see: Fairness, Accountability, Transparency). It is a non-technical and general introduction to some of the many issues that can result in bias and prejudice in machine learning models. To summarize, "machine learning is not, by default, fair or just in any meaningful way". Algorithms and machine learning models _can_ be biased, for many reasons. Without proper analysis, we don't know whether it's a good or bad model, full stop. [1]: https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de#.ykd1vedz1 https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de...
- zamalek 10y ago
  Additionally, before we can approach AI justice we'd need artificial intuition. If you know what the local minima are, you could game the model to convict or defend anyone. The AI would need a way to determine that it is being gamed. For example: taking pictures of bullet casings framed in a specific way might guarantee convictions. Sometimes there is a minority report.
- Houshalter 10y ago
  >Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. 516 is more than enough to fit a simple model. As long as you use cross validation and hold out tests to make sure you aren't fitting. 516 data points is more than a person needs to see to be called an "expert". Many of the algorithms I referenced used fewer data points, or even totally unoptimized weights, and still beat human experts. >An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. We have ground truth though. Whether someone will be convicted is a fairly objective measure. Even if it's slightly biased, it's still the best possible indicator we have of whether or not someone should be released. If you had a time machine that could go into the future and see who would be convicted, would you still argue against using that information, because it might be biased? Leaving people to rot in prison, even if all the statistics point to them being very low risk, is just wrong. >"machine learning is not, by default, fair or just in any meaningful way" Humans are not, by default, fair or just in any meaningful way. Nor are they accurate at prediction. Any argument you can possibly use against algorithms applies even more to humans. That's my entire point. You should trust humans far, far less than you do.
_rpd 10y ago
Yes, surely accurate modeling is the first step towards effective, efficient intervention to improve outcomes. Right now many interventions are little more than guesses that may actually be making things worse. It's a ridiculous state of affairs for 2016.
nl 10y ago
I couldn't disagree more strongly. I agree that human judgement is often flawed and biased. This particular algorithm appears to be always racially biased[1], and wasn't even tesed for it. Substituting flawed human judgement for systematic bias is a bad, bad idea. [1] https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm...
- Houshalter 10y ago
  It's not racially biased. It doesn't even use race as a feature. It just so happens that blacks statistically have worse criminal records, and people with worse criminal records tend to commit more crimes in the future. A white person with the same record is not treated any differently. And the decision is based on objective data and statistics, not flawed human judgement. Human judges are incredibly biased and their predictions are barely better than chance. It's insanity that humans were ever allowed to make important decisions like this.
  - nl 10y ago
    It just so happens that blacks statistically have worse criminal records, and people with worse criminal records tend to commit more crimes in the future. Right, but there is plenty of evidence that this (the statistically worse criminal records) is an artefact of sentencing[1]. Using that biased feature as an input is going to make it worse: the algorithm will recommend higher sentences because previously blacks had previously had more severe sentences. This is almost the exact definition of implicit bias[2] [1] eg http://www.wsj.com/articles/SB10001424127887324432004578304463789858002 http://www.wsj.com/articles/SB100014241278873244320045783044... but plenty of other examples. [2] http://plato.stanford.edu/entries/implicit-bias/ http://plato.stanford.edu/entries/implicit-bias/
    - Houshalter 10y ago
      Well first of all that article is exactly why we should use algorithms. Judges' sentences are arbitrary and biased, and not very correlated with the actual degree of risk. We should set sentences algorithmically, based on how likely the person is to commit a crime again if they go free. Not how the judge feels that day. Second, a criminal record is the crimes you committed in the past, not the sentence a judge gave. The algorithm would see your record and determine your level of risk, along with other objective factors. The algorithm has no bias. It sees the data and uses it to make the most accurate predictions possible. It doesn't care about race, and it is blind to it. A white criminal with the same record, should be given the same prediction as a similar black criminal. And the predictions it makes should be pretty accurate and objective.
panic 10y ago
Nothing on the planet can do the job of a judge perfectly. There will always be edge cases and vague situations. Human judges are sometimes unfair and inaccurate, but we can listen to their reasoning, criticize it, continue the discussion by appealing their decision, and so on. How do you criticize the reasoning of a statistical algorithm?
- vintermann 10y ago
  You can't listen to their reasoning. You can listen to their justifications, which aren't the same thing at all. The attraction of an algorithm is that you can control strictly what information goes in, and what objective is targeted. A judge may - consciously or not - use the information of the accused's skin color. A judge may - again consciously or not - target objectives besides lowering recidivism. An algorithm isn't always better, but when it produces unwanted results, we can criticize it, think about why it went wrong, improve it. When a judge makes mistakes, there's not any easy way to improve him. I think artificial objectivity is "bigger" than artificial intelligence. It's not that ML algorithms are so much better than human brains at what they do - it's that we can have more confidence about how it does it. Oddly enough, considering how they're often criticized for opacity.
  - mk89 10y ago
    > An algorithm isn't always better, but when it produces unwanted results, we can criticize it, think about why it went wrong, improve it. When a judge makes mistakes, there's not any easy way to improve him. I believe that now that there is much fuzz about AI you can improve lots of things. I would like to see in 200 years if you can - no, you can't, why? For the same reason you hardly can change things now: "the algorithm has been studied, improved, tested over and over" -> this is the law. But who says it's correct or fair? A judge, exactly because he is a human, can improve. How? For example, with a feedback system or by requiring explanations for his judgement (based on facts, not moods) - if he doesn't do so, he shall be accountable for his actions. I am not too much into law or politics, but things are that way since always and hardly have changed. But we know how things will go. Algorithms/machines will slowly replace everything, because they lower the costs with the promise that things will be more dynamic and that life will improve - however, the usual gang of people will manage to be "more equal" than others, also when the algorithm works perfectly. > The attraction of an algorithm is that you can control strictly what information goes in [...] How is this attractive? Information can be easily manipulated, delayed, withheld. Essentially, you are just shifting the accountability - from the judge to the guy who feeds the algorithm. And by the way, we are talking about judges who don't apply the law appropriately. Let's talk about those judges who applying the law put their lives at risk - or simply get killed. Shall we? Can we at least trust those judges?
werkljdvopbd 10y ago
The problem with putting machines in control, is that it centralizes power. In these kinds of complex situations, I would rather have thousands of humans each making suboptimal decisions, than decisions by half a dozen algorithms designed by a small group of people. It's the same reason many people oppose electronic voting. Sure, letting humans count votes seems inefficient, and might enable some small scale fraud, but it's much more difficult to commit large scale fraud if there are many people involved. When you use voting machines, you put a tremendous amount of power into the hands of the people who design and build those machines.
- Houshalter 10y ago
  It's easy enough to make the machines open source. And anyway, I can't imagine there is any incentive for there to be some kind of mass conspiracy involved here.
kabouseng 10y ago
Lots of replies here that the algorithms can also be biased since humans design them or the data they are built from is biased. And this is certainly correct. But I believe an even bigger advantage of algorithms is repeatability. Yes initially the algorithms might be biased, but they will always given the same input, deliver the same verdict. And over time we will be able to perfect the algorithms. It's like automated testing. Initially at first automated testing is far less effective than manual testing, but as you add and improve your tests they outperform manual testing almost always. And just like with testing you reduce the need for manual testing, algorithms like these do not have to completely eliminate human judges, but can reduce the amount of cases they are needed for, with improvements in sentencing and processing times.
- soundwave106 10y ago
  The problem I have with this, is the assumption that the "we" perfecting the algorithm is going to be doing so with the best moral and ethical intentions. In this case, we are dealing with an area (law enforcement) which has in the past been misused for political / power purposes. In my opinion, such areas definitely needs some degree of checks and balances to prevent power abuse. Reducing risk assessment to a ML style single algorithmic number would give the people in charge of the algorithm an awful lot of power. I do agree that if humans were entirely moral and ethical, ML can be a huge net gain. I'm not sure I trust humanity that much, personally.
CM30 10y ago
You then have the same issues as with electronic voting. How can you be sure your 'fair' algorithm is running for each individual court case? That it hasn't been tampered with, swapped out, hacked or otherwise changed? Do you give the entire court room the ability to look at the machine and what it's running? How do you then stop any of them messing around with it when you're not looking? You've then got the unfortunate implications that an automated system might create. Okay, the court system very much isn't neutral in regards to race, wealth, gender, nationality, religion or anything else. How do you stop an automated system discriminating based on those factors? Because there are probably different statistics and likelyhoods of different groups committing crimes. And there are various other issues too. Do you trust a small group of people, a company, some open source contributors or an individual to program a system to make 'fairer' decisions than a country's worth of judges? Because the amount of factors that go into a court case means that a piece of software has to deal with a thousands or even millions of possibilities. Which provide a place for an unscrupulous, lazy or shortsighted programmer to screw things up. And there's also the old trust issue. People tend to trust other people more than machines, especially those seemingly associated with a faceless corporation, large government or unknown force. A system that replaces people, no matter how good the algorithm, would get a lot of bad press in this sort of situation. People won't be happy about being sentenced to prison by a computer program. Heck, even a judge using a smartphone app would cause an uproar about how 'soulless technology is replacing human intuition' or something similar. As for trying to make human judges blind to race and gender. That's its own challenge. Facial expressions and courtroom behaviour can indicate whether someone is guilty or not about as much as what they actually say. Try and anonymise that, and you end up in a situation where those who can act regretful over the phone/without being soon have an advantage over those that can't. There are also situations where being blind to race and gender would be a bad thing. Economic information can cause bias, but it can also be used to treat a defendant more fairly as well. If someone was really poor/starving on the streets and stole to stay alive, shouldn't that be indicated somehow? There's a difference between someone like that and a rich guy who stole because he was bored or wanted more.
- Houshalter 10y ago
  How is that any different from other machines used in court? From transcripts/recordings, to forensic tests like DNA machines and algorithms. But in the worst case, you can print the algorithm out on paper. They are generally simple linear models or decision trees. You could easily verify the output by hand, albeit a bit tediously. In the case of more complicated models like random forests or ensembles, you can generate lots of artificial data, and then train a simpler model to approximate its output as best as possible. Then you can print out that model. In the old days, they printed out actuarial tables. Big tables that you would look up your case, and find the recommended output. >Try and anonymise that, and you end up in a situation where those who can act regretful over the phone/without being soon have an advantage over those that can't. That's exactly the problem. People vastly overestimate how well they can read facial expressions. I recall studies that officers couldn't tell liars from nonliars better than chance, but they believed they could.
deleted 10y ago
[deleted]
pc86 10y ago
I count at least seven statements that should be cited. I've heard the "harsher sentences before lunch" thing too but have never seen it cited. I've never heard ugly people get 2x the punishment of attractive people.
- joncrocks 10y ago
  http://www.pnas.org/content/108/17/6889.full http://www.pnas.org/content/108/17/6889.full
- Houshalter 10y ago
  I agree. Sorry I was on mobile and didn't have them on hand, but I will have sources soon. I'm currently writing an article on this so it was fresh in my memory.
- Houshalter 10y ago
  Ok here they are: The algorithm aversion paper is here: http://opim.wharton.upenn.edu/risk/library/WPAF201410-AlgorthimAversion-Dietvorst-Simmons-Massey.pdf http://opim.wharton.upenn.edu/risk/library/WPAF201410-Algort... The attraction bias is found here: http://lesswrong.com/lw/lj/the_halo_effect/ http://lesswrong.com/lw/lj/the_halo_effect/ The hungry judge study is found here: http://www.scientificamerican.com/article/lunchtime-leniency/ http://www.scientificamerican.com/article/lunchtime-leniency... That algorithms almost always do better than humans is backed up by this: http://lesswrong.com/lw/3gv/statistical_prediction_rules_outperform_expert/ http://lesswrong.com/lw/3gv/statistical_prediction_rules_out... and https://meehl.dl.umn.edu/sites/g/files/pua1696/f/167grovemeehlclinstix.pdf https://meehl.dl.umn.edu/sites/g/files/pua1696/f/167grovemee... which also mentions the 1928 study.

coldtea 10y ago

>I could not disagree more with these comments. Psychologists are just now starting to study the phenomenon of "algorithm aversion", where people irrationally trust human judgement far more than algorithms. Even after watching an algorithm do far better in many examples. Those algorithms are also designed by humans and codify biases and prejudices and injustices that are considered OK by society into them. Besides, it's like going for a run. Other might might be far better at it, but we want to do it ourselves, even badly, than just be spectators in our own life. Same for humans becoming spectators while algorithms run business "better". One quality that's not accounted for in accessing better is "more participative".

diogenes78 10y ago
> Those algorithms are also designed by humans But the humans would have to explicitly build the biases into the algorithm. Many human biases are unconscious, so we aren't aware that our conclusions are biased by race, a subject's attractiveness, etc. Going with an algorithm would minimize unconscious biases, and prevent us from reaching a conclusion based on emotion and then just back-fitting a rational-sounding argument.
- coldtea 10y ago
  >But the humans would have to explicitly build the biases into the algorithm. Explicitly as in coding them, but not necessarily as doing it consciously. After all biases are just how people see reality, so it would be natural to code based on one's biases. Also it's less the biases (that are inevitable), and more the blind following of biases (algorithmically) that I'm concerned about. >Going with an algorithm would minimize unconscious biases, and prevent us from reaching a conclusion based on emotion The latter is not necessarily (or even commonly) bad, it's more of a citation needed kind of thing. I'd rather we DO reach conclusions based on emotions. Naked logic sees everything as black and white, it's emotions that make civilisation nuanced (including empathy, pity, gut instinct, abhorrence, etc).
  - sdenton4 10y ago
    Racists have emotions, too, and often make decisions based on them. 'Feeling' that someone is suspicious is a classic cover for racial profiling.
    - coldtea 10y ago
      >Racists have emotions, too, and often make decisions based on them And knives can be used both for murdering people and spreading jam on your bread. The answer to "racist emotions" are emotions of empathy and love towards the other, and not some objective logic that one can follow to fix racism. After all, one could always use objective logic to justify racism if one wants to -- e.g. "blacks are involved in a higher percentage of violent crimes compared to their population". That's a totally objective statement, it just skips trying to find the reasons why that might be so, and uses the numbers to justify thinking of them as violent inherently.

rhizome 10y ago

We are biased by all sorts of things. Right, so who makes the algorithms?

z3r0_29a 10y ago
You point of reference, your past experience, your his-tory. We are seriously in bad shape, not only do we have no right to privacy, and can we be prosecuted for mere words alone.. Now they will try to prove we are guilty prior to committing an offense, and we are doomed.
Houshalter 10y ago
Objective data.
- rhizome 10y ago
  Who decides whether it's objective?

Smerity 10y ago

Stating that method A is problematic does not automatically mean method B is better. > The reality is humans are far worse Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. If you give a machine learning model a dataset where one feature appears entirely indicative (remember: correlation is not causation), it can overfit to that, even if that does not reflect reality. I highly recommend reading "How big data is unfair: understanding unintended sources of unfairness in data driven decision making"[1], by Moritz Hardt, a Google machine learning researcher who has published on the topic (see: Fairness, Accountability, Transparency). It is a non-technical and general introduction to some of the many issues that can result in bias and prejudice in machine learning models. To summarize, "machine learning is not, by default, fair or just in any meaningful way". Algorithms and machine learning models _can_ be biased, for many reasons. Without proper analysis, we don't know whether it's a good or bad model, full stop. [1]: https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de#.ykd1vedz1 https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de...

zamalek 10y ago
Additionally, before we can approach AI justice we'd need artificial intuition. If you know what the local minima are, you could game the model to convict or defend anyone. The AI would need a way to determine that it is being gamed. For example: taking pictures of bullet casings framed in a specific way might guarantee convictions. Sometimes there is a minority report.
Houshalter 10y ago
>Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. 516 is more than enough to fit a simple model. As long as you use cross validation and hold out tests to make sure you aren't fitting. 516 data points is more than a person needs to see to be called an "expert". Many of the algorithms I referenced used fewer data points, or even totally unoptimized weights, and still beat human experts. >An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. We have ground truth though. Whether someone will be convicted is a fairly objective measure. Even if it's slightly biased, it's still the best possible indicator we have of whether or not someone should be released. If you had a time machine that could go into the future and see who would be convicted, would you still argue against using that information, because it might be biased? Leaving people to rot in prison, even if all the statistics point to them being very low risk, is just wrong. >"machine learning is not, by default, fair or just in any meaningful way" Humans are not, by default, fair or just in any meaningful way. Nor are they accurate at prediction. Any argument you can possibly use against algorithms applies even more to humans. That's my entire point. You should trust humans far, far less than you do.

_rpd 10y ago

Yes, surely accurate modeling is the first step towards effective, efficient intervention to improve outcomes. Right now many interventions are little more than guesses that may actually be making things worse. It's a ridiculous state of affairs for 2016.

nl 10y ago

I couldn't disagree more strongly. I agree that human judgement is often flawed and biased. This particular algorithm appears to be always racially biased[1], and wasn't even tesed for it. Substituting flawed human judgement for systematic bias is a bad, bad idea. [1] https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm...

Houshalter 10y ago
It's not racially biased. It doesn't even use race as a feature. It just so happens that blacks statistically have worse criminal records, and people with worse criminal records tend to commit more crimes in the future. A white person with the same record is not treated any differently. And the decision is based on objective data and statistics, not flawed human judgement. Human judges are incredibly biased and their predictions are barely better than chance. It's insanity that humans were ever allowed to make important decisions like this.
- nl 10y ago
  It just so happens that blacks statistically have worse criminal records, and people with worse criminal records tend to commit more crimes in the future. Right, but there is plenty of evidence that this (the statistically worse criminal records) is an artefact of sentencing[1]. Using that biased feature as an input is going to make it worse: the algorithm will recommend higher sentences because previously blacks had previously had more severe sentences. This is almost the exact definition of implicit bias[2] [1] eg http://www.wsj.com/articles/SB10001424127887324432004578304463789858002 http://www.wsj.com/articles/SB100014241278873244320045783044... but plenty of other examples. [2] http://plato.stanford.edu/entries/implicit-bias/ http://plato.stanford.edu/entries/implicit-bias/
  - Houshalter 10y ago
    Well first of all that article is exactly why we should use algorithms. Judges' sentences are arbitrary and biased, and not very correlated with the actual degree of risk. We should set sentences algorithmically, based on how likely the person is to commit a crime again if they go free. Not how the judge feels that day. Second, a criminal record is the crimes you committed in the past, not the sentence a judge gave. The algorithm would see your record and determine your level of risk, along with other objective factors. The algorithm has no bias. It sees the data and uses it to make the most accurate predictions possible. It doesn't care about race, and it is blind to it. A white criminal with the same record, should be given the same prediction as a similar black criminal. And the predictions it makes should be pretty accurate and objective.

panic 10y ago

Nothing on the planet can do the job of a judge perfectly. There will always be edge cases and vague situations. Human judges are sometimes unfair and inaccurate, but we can listen to their reasoning, criticize it, continue the discussion by appealing their decision, and so on. How do you criticize the reasoning of a statistical algorithm?

vintermann 10y ago
You can't listen to their reasoning. You can listen to their justifications, which aren't the same thing at all. The attraction of an algorithm is that you can control strictly what information goes in, and what objective is targeted. A judge may - consciously or not - use the information of the accused's skin color. A judge may - again consciously or not - target objectives besides lowering recidivism. An algorithm isn't always better, but when it produces unwanted results, we can criticize it, think about why it went wrong, improve it. When a judge makes mistakes, there's not any easy way to improve him. I think artificial objectivity is "bigger" than artificial intelligence. It's not that ML algorithms are so much better than human brains at what they do - it's that we can have more confidence about how it does it. Oddly enough, considering how they're often criticized for opacity.
- mk89 10y ago
  > An algorithm isn't always better, but when it produces unwanted results, we can criticize it, think about why it went wrong, improve it. When a judge makes mistakes, there's not any easy way to improve him. I believe that now that there is much fuzz about AI you can improve lots of things. I would like to see in 200 years if you can - no, you can't, why? For the same reason you hardly can change things now: "the algorithm has been studied, improved, tested over and over" -> this is the law. But who says it's correct or fair? A judge, exactly because he is a human, can improve. How? For example, with a feedback system or by requiring explanations for his judgement (based on facts, not moods) - if he doesn't do so, he shall be accountable for his actions. I am not too much into law or politics, but things are that way since always and hardly have changed. But we know how things will go. Algorithms/machines will slowly replace everything, because they lower the costs with the promise that things will be more dynamic and that life will improve - however, the usual gang of people will manage to be "more equal" than others, also when the algorithm works perfectly. > The attraction of an algorithm is that you can control strictly what information goes in [...] How is this attractive? Information can be easily manipulated, delayed, withheld. Essentially, you are just shifting the accountability - from the judge to the guy who feeds the algorithm. And by the way, we are talking about judges who don't apply the law appropriately. Let's talk about those judges who applying the law put their lives at risk - or simply get killed. Shall we? Can we at least trust those judges?

werkljdvopbd 10y ago

The problem with putting machines in control, is that it centralizes power. In these kinds of complex situations, I would rather have thousands of humans each making suboptimal decisions, than decisions by half a dozen algorithms designed by a small group of people. It's the same reason many people oppose electronic voting. Sure, letting humans count votes seems inefficient, and might enable some small scale fraud, but it's much more difficult to commit large scale fraud if there are many people involved. When you use voting machines, you put a tremendous amount of power into the hands of the people who design and build those machines.

Houshalter 10y ago
It's easy enough to make the machines open source. And anyway, I can't imagine there is any incentive for there to be some kind of mass conspiracy involved here.

kabouseng 10y ago

Lots of replies here that the algorithms can also be biased since humans design them or the data they are built from is biased. And this is certainly correct. But I believe an even bigger advantage of algorithms is repeatability. Yes initially the algorithms might be biased, but they will always given the same input, deliver the same verdict. And over time we will be able to perfect the algorithms. It's like automated testing. Initially at first automated testing is far less effective than manual testing, but as you add and improve your tests they outperform manual testing almost always. And just like with testing you reduce the need for manual testing, algorithms like these do not have to completely eliminate human judges, but can reduce the amount of cases they are needed for, with improvements in sentencing and processing times.

soundwave106 10y ago
The problem I have with this, is the assumption that the "we" perfecting the algorithm is going to be doing so with the best moral and ethical intentions. In this case, we are dealing with an area (law enforcement) which has in the past been misused for political / power purposes. In my opinion, such areas definitely needs some degree of checks and balances to prevent power abuse. Reducing risk assessment to a ML style single algorithmic number would give the people in charge of the algorithm an awful lot of power. I do agree that if humans were entirely moral and ethical, ML can be a huge net gain. I'm not sure I trust humanity that much, personally.

CM30 10y ago

You then have the same issues as with electronic voting. How can you be sure your 'fair' algorithm is running for each individual court case? That it hasn't been tampered with, swapped out, hacked or otherwise changed? Do you give the entire court room the ability to look at the machine and what it's running? How do you then stop any of them messing around with it when you're not looking? You've then got the unfortunate implications that an automated system might create. Okay, the court system very much isn't neutral in regards to race, wealth, gender, nationality, religion or anything else. How do you stop an automated system discriminating based on those factors? Because there are probably different statistics and likelyhoods of different groups committing crimes. And there are various other issues too. Do you trust a small group of people, a company, some open source contributors or an individual to program a system to make 'fairer' decisions than a country's worth of judges? Because the amount of factors that go into a court case means that a piece of software has to deal with a thousands or even millions of possibilities. Which provide a place for an unscrupulous, lazy or shortsighted programmer to screw things up. And there's also the old trust issue. People tend to trust other people more than machines, especially those seemingly associated with a faceless corporation, large government or unknown force. A system that replaces people, no matter how good the algorithm, would get a lot of bad press in this sort of situation. People won't be happy about being sentenced to prison by a computer program. Heck, even a judge using a smartphone app would cause an uproar about how 'soulless technology is replacing human intuition' or something similar. As for trying to make human judges blind to race and gender. That's its own challenge. Facial expressions and courtroom behaviour can indicate whether someone is guilty or not about as much as what they actually say. Try and anonymise that, and you end up in a situation where those who can act regretful over the phone/without being soon have an advantage over those that can't. There are also situations where being blind to race and gender would be a bad thing. Economic information can cause bias, but it can also be used to treat a defendant more fairly as well. If someone was really poor/starving on the streets and stole to stay alive, shouldn't that be indicated somehow? There's a difference between someone like that and a rich guy who stole because he was bored or wanted more.

Houshalter 10y ago
How is that any different from other machines used in court? From transcripts/recordings, to forensic tests like DNA machines and algorithms. But in the worst case, you can print the algorithm out on paper. They are generally simple linear models or decision trees. You could easily verify the output by hand, albeit a bit tediously. In the case of more complicated models like random forests or ensembles, you can generate lots of artificial data, and then train a simpler model to approximate its output as best as possible. Then you can print out that model. In the old days, they printed out actuarial tables. Big tables that you would look up your case, and find the recommended output. >Try and anonymise that, and you end up in a situation where those who can act regretful over the phone/without being soon have an advantage over those that can't. That's exactly the problem. People vastly overestimate how well they can read facial expressions. I recall studies that officers couldn't tell liars from nonliars better than chance, but they believed they could.

deleted 10y ago

[deleted]

pc86 10y ago

I count at least seven statements that should be cited. I've heard the "harsher sentences before lunch" thing too but have never seen it cited. I've never heard ugly people get 2x the punishment of attractive people.

joncrocks 10y ago
http://www.pnas.org/content/108/17/6889.full http://www.pnas.org/content/108/17/6889.full
Houshalter 10y ago
I agree. Sorry I was on mobile and didn't have them on hand, but I will have sources soon. I'm currently writing an article on this so it was fresh in my memory.
Houshalter 10y ago
Ok here they are: The algorithm aversion paper is here: http://opim.wharton.upenn.edu/risk/library/WPAF201410-AlgorthimAversion-Dietvorst-Simmons-Massey.pdf http://opim.wharton.upenn.edu/risk/library/WPAF201410-Algort... The attraction bias is found here: http://lesswrong.com/lw/lj/the_halo_effect/ http://lesswrong.com/lw/lj/the_halo_effect/ The hungry judge study is found here: http://www.scientificamerican.com/article/lunchtime-leniency/ http://www.scientificamerican.com/article/lunchtime-leniency... That algorithms almost always do better than humans is backed up by this: http://lesswrong.com/lw/3gv/statistical_prediction_rules_outperform_expert/ http://lesswrong.com/lw/3gv/statistical_prediction_rules_out... and https://meehl.dl.umn.edu/sites/g/files/pua1696/f/167grovemeehlclinstix.pdf https://meehl.dl.umn.edu/sites/g/files/pua1696/f/167grovemee... which also mentions the 1928 study.