3 ms·
Neural networks are NOT hard to understand. Tools like LIME[1] have been around for a few years, and work really well for inspecting and understanding the deci
by nl 7y ago
Neural networks are NOT hard to understand.
Tools like LIME[1] have been around for a few years, and work really well for inspecting and understanding the decisions made by NNs.
It's true that complex decisions are hard to explain, but LIME does a better job than (say) anyway of trying to explain a SVN+Human coded feature.
LIME has been out since 2016. It and similar techniques are widely used in production systems.
It's time this "NNs are hard to understand" idea died.
[1] https://github.com/marcotcr/lime https://github.com/marcotcr/lime
- p1esk 7y agoHow does it explain adversarial examples where changing a single pixel leads to class prediction change?
- nl 7y agohttps://openreview.net/pdf?id=BkpiPMbA- https://openreview.net/pdf?id=BkpiPMbA- Code: https://github.com/sunblaze-ucb/decision-boundaries https://github.com/sunblaze-ucb/decision-boundaries
- p1esk 7y agoThe problem is we have very little understanding of how decision boundaries are formed, and very little intuition about how the input manifold “looks like”. That’s why it’s so difficult to make sure the boundaries have large enough margin (i.e. robust for all reasonable samples). After reading papers like this it feels like while we make some progress towards understanding some aspects of NN operation, we uncover deeper aspects that we don’t understand. This process itself looks like gradient descent (saddles everywhere!), and we are still pretty far from the global minima. :)
- nl 7y agoSure. But have you ever tried explaining a SVN decision to an actual human before? I'm not convinced that a NN is any worse than that.