8 ms·
State of the art performance is being broken in multiple fields rapidly these days. However, AI explainability has a long way to go. Scaling opaqueness makes th
by Reebz 6y ago
State of the art performance is being broken in multiple fields rapidly these days. However, AI explainability has a long way to go. Scaling opaqueness makes this problem worse.
Fine tuning labels black-box style is a terrifying concept to most who are working in fields where great risk must be managed to avoid unintended and disparate impact. Facebook making an oopsies suggesting my friends face in a photo instead of mine with a SSL trained model may seem trivial, but this non-human oversight is concerning for other applications.
If you were rejected by a bank for a home loan, you would want to know why your creditworthiness wasn’t evaluated positively (and banks must explain precisely why to regulators). And if a self-driving car made a decision in a collision, or a CV model performing cancer screening in X-rays that gave a bad result - ...or recommending politically divisive content... - or a thousand other real world examples that have human impact.
- visarga 6y agoHigh risk models should work with humans in the loop, not autonomous.
- a_imho 6y agoWould this rule out self driving cars from the get go?
- 7_my_mind 6y agoDepends on what you mean by self driving cars. To me it is not even clear that driving assist tech will ever be advanced enough for cars to drive themselves reliably.* But that aside. Human in control + driving assists is already safer than fully manual driving. The question is whether the opposite is even safer. It is obvious to me that self driving with the human as backup is a joke. Humans cannot react fast enough in real emergencies. So if you want the car to mostly drive itself, it has to actually always drive itself. And to me it is not clear whether this is really going to be safer than the human+driving_assist scenario. *Of course I am talking about tech based on the current DL approaches. If we had AGI, then the question does not even need to be asked.
- Reebz 6y agoYes, this is the point I am trying to make. Self-supervised models using already opaque techniques built by an ethically flexible company is not a good recipe.
- loopz 6y agoA sufficiently complex algo black box could fool the people.
- asamiam 6y agoWe want our models to be opaque in the same way we like to ask people why they made a certain decision or act in the way they do. What's interesting to me is if you want to know about a person you'll get better information by asking their closest friends than themselves. Perhaps there's something in the black box nature of our own self understanding similar to these hyper complex function approximators. Judges make harsher rulings when they're hungry. A regulator might decline a loan and justify it b/c they had a bad experience with someone that reminds them of the person they're dealing with. AI makes bad decisions in boundary conditions or when they're trained on biased data. I think it's good to strive for opacity in every case but in complex decision spaces I'm not sure if we'll ever get fully satisfactory explanations. Which is a meandering way of making the non-point that models making judgments have made me re-evaluate what I trust and why, and I think in many cases I'd rather trust a black box if I know what's gone into it and what's come out.
- elcomet 6y agoI don't really agree with you. > If you were rejected by a bank for a home loan, you would want to know why your creditworthiness wasn’t evaluated positively (and banks must explain precisely why to regulators). This is why we don't use humans to evaluate credits, but precise algorithms. Humans are just applying the algorithms, they are not evaluating themselves with their gut feeling. I don't see why this would change with AI. Regulations prevents unexplainable tools to be used there, so deep learning black box models will not be used, similarly to why humans are not used today. But in cases where performance is required, but not explainability, then deep learning will strive. And I believe that cases where explanation is required is only a very small subset of areas where AI could be useful. > And if a self-driving car made a decision in a collision Would you prefer a car that crashes once every 100 million miles but is not explainable, or a car that crashes once every 100 miles but is interpretable can explain why it crashed ?
- TheOtherHobbes 6y agoIsn't it interesting that an industry like banking, built on those beautiful, precise algorithms, blows up so regularly?
- loopz 6y agoYou're really thinking of finance. Pure banking is solved long ago.
- Reebz 6y agoThat's ok, but based on your example, how would a human apply an algorithm that it can't determine why it worked? We're not talking about "Not Hotdog" here. Application of a model in a real-world at-scale scenario is a lot more than running inference and walking away. At a bank credit decisions are evaluated by humans, frequently and often. These reviews are conducted in the forms of sampling audits, control processes, and other scenarios that involve internal bank employees and external regulators. In each case, humans will inspect the details of what occurred. This would be impossible with any type black-box model (SSL, deep NN, etc.).