4 ms·

I don't really agree with you. > If you were rejected by a bank for a home loan, you would want to know why your creditworthiness wasn’t evaluated positively (

I don't really agree with you. > If you were rejected by a bank for a home loan, you would want to know why your creditworthiness wasn’t evaluated positively (and banks must explain precisely why to regulators). This is why we don't use humans to evaluate credits, but precise algorithms. Humans are just applying the algorithms, they are not evaluating themselves with their gut feeling. I don't see why this would change with AI. Regulations prevents unexplainable tools to be used there, so deep learning black box models will not be used, similarly to why humans are not used today. But in cases where performance is required, but not explainability, then deep learning will strive. And I believe that cases where explanation is required is only a very small subset of areas where AI could be useful. > And if a self-driving car made a decision in a collision Would you prefer a car that crashes once every 100 million miles but is not explainable, or a car that crashes once every 100 miles but is interpretable can explain why it crashed ?

TheOtherHobbes 6y ago
Isn't it interesting that an industry like banking, built on those beautiful, precise algorithms, blows up so regularly?
- loopz 6y ago
  You're really thinking of finance. Pure banking is solved long ago.
Reebz 6y ago
That's ok, but based on your example, how would a human apply an algorithm that it can't determine why it worked? We're not talking about "Not Hotdog" here. Application of a model in a real-world at-scale scenario is a lot more than running inference and walking away. At a bank credit decisions are evaluated by humans, frequently and often. These reviews are conducted in the forms of sampling audits, control processes, and other scenarios that involve internal bank employees and external regulators. In each case, humans will inspect the details of what occurred. This would be impossible with any type black-box model (SSL, deep NN, etc.).
- elcomet 6y ago
  I'm not sure I understand your comment. Bank credits are auditable, because there are some precise algorithm around them. Humans created the algorithm specifically so that it is interpretable. Then they must apply it the same way to everybody. They can't just say "oh, I think this person is more likely to actually reimburse, I play golf with him and I trust him !" This is a specific case where no black box will ever be used (at least I hope).
  - loopz 6y ago
    There's incentive to use complex black boxes everywhere, if more profitable.
    - elcomet 6y ago
      Of course, and my point is that it's perfectly fine, and even desirable, as better models lead to better outcomes. Of course, for cases where explainability is needed, either required by law (such as banking), or by common sense, then black boxes will not be deployed.

TheOtherHobbes 6y ago

Isn't it interesting that an industry like banking, built on those beautiful, precise algorithms, blows up so regularly?

loopz 6y ago
You're really thinking of finance. Pure banking is solved long ago.

Reebz 6y ago

That's ok, but based on your example, how would a human apply an algorithm that it can't determine why it worked? We're not talking about "Not Hotdog" here. Application of a model in a real-world at-scale scenario is a lot more than running inference and walking away. At a bank credit decisions are evaluated by humans, frequently and often. These reviews are conducted in the forms of sampling audits, control processes, and other scenarios that involve internal bank employees and external regulators. In each case, humans will inspect the details of what occurred. This would be impossible with any type black-box model (SSL, deep NN, etc.).

elcomet 6y ago
I'm not sure I understand your comment. Bank credits are auditable, because there are some precise algorithm around them. Humans created the algorithm specifically so that it is interpretable. Then they must apply it the same way to everybody. They can't just say "oh, I think this person is more likely to actually reimburse, I play golf with him and I trust him !" This is a specific case where no black box will ever be used (at least I hope).
- loopz 6y ago
  There's incentive to use complex black boxes everywhere, if more profitable.
  - elcomet 6y ago
    Of course, and my point is that it's perfectly fine, and even desirable, as better models lead to better outcomes. Of course, for cases where explainability is needed, either required by law (such as banking), or by common sense, then black boxes will not be deployed.