4 ms·

Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting l

Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. As much as I hate regulations I think this may be a good place to legislate in order to protect those wrongly affected. It makes me think about the mess the derivatives markets made in 2008

taneq 7y ago
It’s not like your bank manager used to give you a phone call to explain in depth why your loan was declined. Most of these situations were already impenetrable from a human standpoint. Humans are far more ‘black box’ than neutral nets, we just happen to be able to construct plausible explanations in parallel.
- BoiledCabbage 7y ago
  No, they were impenetrable from the customer perspective, but either the human making the decision or the software being used can answer. With a black box model, nobody can answer.
  - taneq 7y ago
    From the post I responded to: > the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. I was pointing out that it’s already like this. Whether it’s a black box ML model or a capricious bank employee makes no difference if there’s no transparency either way.
- _jal 7y ago
  Bank managers, however, can at least theoretically be hauled in to court and questioned.
  - taneq 7y ago
    That's true, but as mentioned above, the explanations humans give of how they reached their conclusions are parallel construction rather than genuine introspection. We really don't understand how our brains work, either when they're generating their conclusions or when they're generating an explanation for those conclusions.
    - _jal 7y ago
      Yes, I'm (somewhat) aware of the research. I'm unaware of any legal jurisdiction that recognizes it, let alone tries to address it somehow. There's reality, there's legal reality, and then there's what legal reality does to reality.

taneq 7y ago

It’s not like your bank manager used to give you a phone call to explain in depth why your loan was declined. Most of these situations were already impenetrable from a human standpoint. Humans are far more ‘black box’ than neutral nets, we just happen to be able to construct plausible explanations in parallel.

BoiledCabbage 7y ago
No, they were impenetrable from the customer perspective, but either the human making the decision or the software being used can answer. With a black box model, nobody can answer.
- taneq 7y ago
  From the post I responded to: > the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. I was pointing out that it’s already like this. Whether it’s a black box ML model or a capricious bank employee makes no difference if there’s no transparency either way.
_jal 7y ago
Bank managers, however, can at least theoretically be hauled in to court and questioned.
- taneq 7y ago
  That's true, but as mentioned above, the explanations humans give of how they reached their conclusions are parallel construction rather than genuine introspection. We really don't understand how our brains work, either when they're generating their conclusions or when they're generating an explanation for those conclusions.
  - _jal 7y ago
    Yes, I'm (somewhat) aware of the research. I'm unaware of any legal jurisdiction that recognizes it, let alone tries to address it somehow. There's reality, there's legal reality, and then there's what legal reality does to reality.