3 ms·

I wonder to what extent this creates a feedback loop that isn't fully acknowledged/understood. As long as the prediction is only one element of decision and no

by cmehdy 6y ago

I wonder to what extent this creates a feedback loop that isn't fully acknowledged/understood. As long as the prediction is only one element of decision and not the decision itself, it can probably be useful for judges. If it becomes "the algorithm says", then the feedback loop could have unpredictable consequences. By nature of a world with finite resources, I also wonder whether the 2.7 million USD could have made a better impact on likelihood to show up if they had been used to put in place or extend existing support structures for the majority of the crimes those courts see. It's still worth it to research how technology can help of course, but it's also hard to put in any kind of comparative study when only doing one thing at a time.

zdragnar 6y ago
In a city of 8 million people, it is not likely that $2.7 million would substantially shift underlying social issues to reduce crime or failure to appear for court dates. In the example case given, prosecutors appeared to have no evidence whatsoever that the defendant would skip court; the defense presented the algorithm and explained why he likely got the score he did. I imagine judges may come to heavily weigh the score, but cannot imagine any would ever give such an algorithm a blank check to decide for them.
- cmehdy 6y ago
  I might be misunderstanding the cost there but isn't it more the amount of people who were going through that system that matters, and not the city size? By that I mean that we could also look at what 2.7 million dollars (or even that flat cost + operating costs evened out over the period of time of the study) would do, were it put towards every one of those defendants in dire conditions. To ensure for example that they have at least a temporary place to stay, social programs in place, someone to follow their case and offer a reinforced legal counsel (which is available to everyone for free for sure, but I hope nobody kids themselves about how overworked those people are - I could bet everything I have that anyone with a good income would always pick a private lawyer over duty counsel). As for your last point: it certainly doesn't seem to be the case right now but I can very well see that entering the realm of possibilities. You can't imagine that a system that performs on paper really well and is thrown into a field where people are incredibly over-worked would end up being relied on a little bit too much? I mean, we see that sort of thing in place for way less reliable things with people who actually know better (i.e. how many scripts and old bits of code hold together much of the world). And in the medical field the development and usage of AI is on a steady rise ("chatbots" in psychology, predictive modeling for decision making and disease diagnosis, "telehealth" assessments using data from IoT devices). If AI gave people more work rather than less, it would be of no use after all.
  - deleted 6y ago
    [deleted]
  - nobody9999 6y ago
    >and offer a reinforced legal counsel (which is available to everyone for free for sure, but I hope nobody kids themselves about how overworked those people are - I could bet everything I have that anyone with a good income would always pick a private lawyer over duty counsel). Where did you get the idea that everyone gets free legal counsel for civil or criminal actions? In many, if not most places in the US, in criminal actions, you must prove that you don't have the means to pay a lawyer before one is "appointed by the court." In fact, many places have very low limits on income/assets before providing free counsel. The requirements surrounding this are a hodgepodge of state and local rules[0]. As for civil actions, most places do not provide legal counsel[1] at all. [0] https://www.findlaw.com/hirealawyer/do-you-need-a-lawyer/do-you-qualify-for-free-legal-aid.html https://www.findlaw.com/hirealawyer/do-you-need-a-lawyer/do-... [1] https://en.wikipedia.org/wiki/Legal_aid_in_the_United_States#A_history_of_civil_legal_aid_in_the_United_States https://en.wikipedia.org/wiki/Legal_aid_in_the_United_States...
    - Nitrolo 6y ago
      I believe they were referring to England, where you do get legal counsel for free (though my understanding is that restrictions to that have been placed in recent years).
      - cmehdy 6y ago
        I was also thinking of Canada, France, and Brazil. But nobody9999 is right, for the most part it's not a right in the world.
  - zdragnar 6y ago
    What you are advocating for would not last long at all as a one-time source of funding. On the other hand, letting people out without bail without relying on bail loan sharks, in perpetuity, will do much good on itd own. They don't need the bail loan, they might not lose a job from not showing up to work if they couldnt make bail, etc. I think, also, that chatbots are an unfair comparison. Those are specifically designed to decrease triage time for non-emergency scenarios. This is a fundamentally different scenario, where it provides evidence to counter the prosecution. It will always fall to the judge to weigh the score versus what evidence the prosecution provides. If the prosecution has no good reason to demand punishingly high bail, and the model continues to hold up well over time, I see no reason to assume that using it leads to bad outcomes. Remember that the most overworked people are the public defenders, and this is an aide for them.
jamesrom 6y ago
> As long as the prediction is only one element of decision and not the decision itself, it can probably be useful for judges. Whatever reasoning a judge may have to overrule the algorithm must be quantifiable -- otherwise it would not exist in the judge's mind. Given enough data and compute, judges will be make worse decisions than algorithms, even on the edge cases. Free the data. Train the nets. Highlight and eliminate any source of human bias and error we can find. Measure outcomes. Train again. It's the only rational way forward.
tchalla 6y ago
In short, the society needs to decide the success metric for the algorithm. There seems to be little discussion on it.

by cmehdy 6y ago

zdragnar 6y ago

In a city of 8 million people, it is not likely that $2.7 million would substantially shift underlying social issues to reduce crime or failure to appear for court dates. In the example case given, prosecutors appeared to have no evidence whatsoever that the defendant would skip court; the defense presented the algorithm and explained why he likely got the score he did. I imagine judges may come to heavily weigh the score, but cannot imagine any would ever give such an algorithm a blank check to decide for them.

cmehdy 6y ago
I might be misunderstanding the cost there but isn't it more the amount of people who were going through that system that matters, and not the city size? By that I mean that we could also look at what 2.7 million dollars (or even that flat cost + operating costs evened out over the period of time of the study) would do, were it put towards every one of those defendants in dire conditions. To ensure for example that they have at least a temporary place to stay, social programs in place, someone to follow their case and offer a reinforced legal counsel (which is available to everyone for free for sure, but I hope nobody kids themselves about how overworked those people are - I could bet everything I have that anyone with a good income would always pick a private lawyer over duty counsel). As for your last point: it certainly doesn't seem to be the case right now but I can very well see that entering the realm of possibilities. You can't imagine that a system that performs on paper really well and is thrown into a field where people are incredibly over-worked would end up being relied on a little bit too much? I mean, we see that sort of thing in place for way less reliable things with people who actually know better (i.e. how many scripts and old bits of code hold together much of the world). And in the medical field the development and usage of AI is on a steady rise ("chatbots" in psychology, predictive modeling for decision making and disease diagnosis, "telehealth" assessments using data from IoT devices). If AI gave people more work rather than less, it would be of no use after all.
- deleted 6y ago
  [deleted]
- nobody9999 6y ago
  >and offer a reinforced legal counsel (which is available to everyone for free for sure, but I hope nobody kids themselves about how overworked those people are - I could bet everything I have that anyone with a good income would always pick a private lawyer over duty counsel). Where did you get the idea that everyone gets free legal counsel for civil or criminal actions? In many, if not most places in the US, in criminal actions, you must prove that you don't have the means to pay a lawyer before one is "appointed by the court." In fact, many places have very low limits on income/assets before providing free counsel. The requirements surrounding this are a hodgepodge of state and local rules[0]. As for civil actions, most places do not provide legal counsel[1] at all. [0] https://www.findlaw.com/hirealawyer/do-you-need-a-lawyer/do-you-qualify-for-free-legal-aid.html https://www.findlaw.com/hirealawyer/do-you-need-a-lawyer/do-... [1] https://en.wikipedia.org/wiki/Legal_aid_in_the_United_States#A_history_of_civil_legal_aid_in_the_United_States https://en.wikipedia.org/wiki/Legal_aid_in_the_United_States...
  - Nitrolo 6y ago
    I believe they were referring to England, where you do get legal counsel for free (though my understanding is that restrictions to that have been placed in recent years).
    - cmehdy 6y ago
      I was also thinking of Canada, France, and Brazil. But nobody9999 is right, for the most part it's not a right in the world.
- zdragnar 6y ago
  What you are advocating for would not last long at all as a one-time source of funding. On the other hand, letting people out without bail without relying on bail loan sharks, in perpetuity, will do much good on itd own. They don't need the bail loan, they might not lose a job from not showing up to work if they couldnt make bail, etc. I think, also, that chatbots are an unfair comparison. Those are specifically designed to decrease triage time for non-emergency scenarios. This is a fundamentally different scenario, where it provides evidence to counter the prosecution. It will always fall to the judge to weigh the score versus what evidence the prosecution provides. If the prosecution has no good reason to demand punishingly high bail, and the model continues to hold up well over time, I see no reason to assume that using it leads to bad outcomes. Remember that the most overworked people are the public defenders, and this is an aide for them.

jamesrom 6y ago

> As long as the prediction is only one element of decision and not the decision itself, it can probably be useful for judges. Whatever reasoning a judge may have to overrule the algorithm must be quantifiable -- otherwise it would not exist in the judge's mind. Given enough data and compute, judges will be make worse decisions than algorithms, even on the edge cases. Free the data. Train the nets. Highlight and eliminate any source of human bias and error we can find. Measure outcomes. Train again. It's the only rational way forward.

tchalla 6y ago

In short, the society needs to decide the success metric for the algorithm. There seems to be little discussion on it.