3 ms·
How does it explain adversarial examples where changing a single pixel leads to class prediction change?
by p1esk 7y ago
How does it explain adversarial examples where changing a single pixel leads to class prediction change?
- nl 7y agohttps://openreview.net/pdf?id=BkpiPMbA- https://openreview.net/pdf?id=BkpiPMbA- Code: https://github.com/sunblaze-ucb/decision-boundaries https://github.com/sunblaze-ucb/decision-boundaries
- p1esk 7y agoThe problem is we have very little understanding of how decision boundaries are formed, and very little intuition about how the input manifold “looks like”. That’s why it’s so difficult to make sure the boundaries have large enough margin (i.e. robust for all reasonable samples). After reading papers like this it feels like while we make some progress towards understanding some aspects of NN operation, we uncover deeper aspects that we don’t understand. This process itself looks like gradient descent (saddles everywhere!), and we are still pretty far from the global minima. :)
- nl 7y agoSure. But have you ever tried explaining a SVN decision to an actual human before? I'm not convinced that a NN is any worse than that.