4 ms·

>Their results must be very accurate and unbiased That is the assumption that scares me. The algorithm encodes the bias and applies it in every case. This fe

>Their results must be very accurate and unbiased That is the assumption that scares me. The algorithm encodes the bias and applies it in every case. This feature will only save you from a shifting bias.

rectang 8y ago
Up until now we were saddled with the extremely biased and unjust bond system. Calls to make the algorithms transparent are well taken -- but let's not go back.
- cgriswald 8y ago
  Your thinking is very dangerous. Even if the algorithm is transparent and fair, that does not necessarily equate to a step forward because of the bias already present in the system. The algorithm will be[0] applied equally, sure, but that does not mean outcomes will be equal. For instance, consider that owning a major asset probably decreases flight risk. That seems fair. But, this means that poor people are probably more of a flight risk than a middle- or upper-class homeowner. So if the algorithm considers this, poor people would be disproportionately affected. Also consider that having family abroad or being from another country probably increases flight risk. Hence, immigrants would be disproportionately affected. But now, when there is an unjust outcome, there isn't even an option to avoid jail time (even if that option is difficult to reach or undesirable); and the human responsible for the decision to incarcerate can simply wash their hands of it, since the algorithm gave the suspect a grade of "D". No need for empathy, sympathy, or guilt. Then there are the pending discussions about what is 'fair' and the answer to those questions is politics... [0] - Edit: Could be
  - rectang 8y ago
    Those critiques are all indeed things to watch out for, but progress is possible -- see New Jersey. The status quo is a roaring injustice and ongoing disaster, and by insisting on the perfect over the good you are condemning millions of people to continue suffering indefinitely. How is your thinking not "very dangerous" to all those wronged day after day by the current system?
    - cgriswald 8y ago
      You've completely misunderstood my point if you think I'm advocating for the perfect at the cost of the good. I'm advocating caution rather than a blind faith that this solution is ipso facto forward progress, that just needs some tweaking. The danger here is doubling down on the existing injustices, removing the limited accountability and positive exceptions that exist, and effectively sweeping conversation under the rug or arguing ideologies about the fairness of tweaks to the algorithms at the cost of real human lives.
      - rectang 8y ago
        You're making a distinction without a difference -- whether it's because of "caution" or whatever, no progress will be made if you have your way. According to you the solution lies in "politics". Yet, this law is the product of California's government, a political body -- is that not legitimate "politics"? So what would make for a "political" solution that would qualify for your stamp of approval? Chasing a consensus with the "tough on crime" crowd who benefit from felony disenfranchisement disproportionately suppressing minority votes? No such consensus will ever happen.
        cgriswald 8y ago
        Again, you either misunderstand or misrepresent my points. > According to you the solution lies in "politics". My post was critical of the role of politics in determining the fairness of the algorithm. It removes the human lives factor and provides a platform for politicians and ideologues ("tough on crime" or otherwise) to either do nothing, argue about the ideology of fairness, or ignore ground truth about outcomes and pre-existing bias in the system.
cobbzilla 8y ago
The magnitude of bias being applied today is so widely distributed as to be essentially immeasurable, so I don't know if algorithm-driven policy will leave us any better or worse off. In an open system, the bias could be identified, a patch created and approved, and a new release deployed. As biases shift, more patches are made. Ideally we find smaller and smaller "bugs" as time goes by. Alas, we are seemingly very far from that today. There is probably great value in controlling that algorithm, and like gerrymandering district maps, the politicians have no incentive to create such an open system. I fear that algorithm-driven bail might become just another tool for politicians and power brokers to bludgeon their opponents with.
- bjt2n3904 8y ago
  From the camp of people that decided to let algorithms draw voting districts: "just redistrict, lol. But after the election"
Someone1234 8y ago
True. It is however a first step, and we can look at the results of the current implementation and adjust as needed. I imagine academics will be watching closely in particular. That's always been the challenge in the current system, because you're leaving responsibility up to individuals it was a hard system to adjust even if it isn't working fairly (e.g. you aren't going to just randomly fire judges because minorities got a slightly unfair deal).

rectang 8y ago

Up until now we were saddled with the extremely biased and unjust bond system. Calls to make the algorithms transparent are well taken -- but let's not go back.

cgriswald 8y ago
Your thinking is very dangerous. Even if the algorithm is transparent and fair, that does not necessarily equate to a step forward because of the bias already present in the system. The algorithm will be[0] applied equally, sure, but that does not mean outcomes will be equal. For instance, consider that owning a major asset probably decreases flight risk. That seems fair. But, this means that poor people are probably more of a flight risk than a middle- or upper-class homeowner. So if the algorithm considers this, poor people would be disproportionately affected. Also consider that having family abroad or being from another country probably increases flight risk. Hence, immigrants would be disproportionately affected. But now, when there is an unjust outcome, there isn't even an option to avoid jail time (even if that option is difficult to reach or undesirable); and the human responsible for the decision to incarcerate can simply wash their hands of it, since the algorithm gave the suspect a grade of "D". No need for empathy, sympathy, or guilt. Then there are the pending discussions about what is 'fair' and the answer to those questions is politics... [0] - Edit: Could be
- rectang 8y ago
  Those critiques are all indeed things to watch out for, but progress is possible -- see New Jersey. The status quo is a roaring injustice and ongoing disaster, and by insisting on the perfect over the good you are condemning millions of people to continue suffering indefinitely. How is your thinking not "very dangerous" to all those wronged day after day by the current system?
  - cgriswald 8y ago
    You've completely misunderstood my point if you think I'm advocating for the perfect at the cost of the good. I'm advocating caution rather than a blind faith that this solution is ipso facto forward progress, that just needs some tweaking. The danger here is doubling down on the existing injustices, removing the limited accountability and positive exceptions that exist, and effectively sweeping conversation under the rug or arguing ideologies about the fairness of tweaks to the algorithms at the cost of real human lives.
    - rectang 8y ago
      You're making a distinction without a difference -- whether it's because of "caution" or whatever, no progress will be made if you have your way. According to you the solution lies in "politics". Yet, this law is the product of California's government, a political body -- is that not legitimate "politics"? So what would make for a "political" solution that would qualify for your stamp of approval? Chasing a consensus with the "tough on crime" crowd who benefit from felony disenfranchisement disproportionately suppressing minority votes? No such consensus will ever happen.
      - cgriswald 8y ago
        Again, you either misunderstand or misrepresent my points. > According to you the solution lies in "politics". My post was critical of the role of politics in determining the fairness of the algorithm. It removes the human lives factor and provides a platform for politicians and ideologues ("tough on crime" or otherwise) to either do nothing, argue about the ideology of fairness, or ignore ground truth about outcomes and pre-existing bias in the system.

cobbzilla 8y ago

The magnitude of bias being applied today is so widely distributed as to be essentially immeasurable, so I don't know if algorithm-driven policy will leave us any better or worse off. In an open system, the bias could be identified, a patch created and approved, and a new release deployed. As biases shift, more patches are made. Ideally we find smaller and smaller "bugs" as time goes by. Alas, we are seemingly very far from that today. There is probably great value in controlling that algorithm, and like gerrymandering district maps, the politicians have no incentive to create such an open system. I fear that algorithm-driven bail might become just another tool for politicians and power brokers to bludgeon their opponents with.

bjt2n3904 8y ago
From the camp of people that decided to let algorithms draw voting districts: "just redistrict, lol. But after the election"

Someone1234 8y ago

True. It is however a first step, and we can look at the results of the current implementation and adjust as needed. I imagine academics will be watching closely in particular. That's always been the challenge in the current system, because you're leaving responsibility up to individuals it was a hard system to adjust even if it isn't working fairly (e.g. you aren't going to just randomly fire judges because minorities got a slightly unfair deal).