4 ms·
Interesting that simple optimization ends up with high-frequency noise similar to adversarial attacks on neural nets. While I agree that the practicality of th
by Kronopath 9y ago
Interesting that simple optimization ends up with high-frequency noise similar to adversarial attacks on neural nets.
While I agree that the practicality of these visualizations mean that you have to fight against this high-frequency "cheating", I can't help but shake the feeling that what these optimization visualizations are showing us is correct. This is what the neuron responds to, whether you like it or not. Put in another way, the problem doesn't seem to be with the visualization but with the network itself.
Has there been any research in making neural networks that are robust to adversarial examples?
- somesnm 9y agoThere are was a Kaggle competition on Defences against Adversarial attacks by Google Brain for NIPS 2017 https://www.kaggle.com/c/nips-2017-defense-against-adversarial-attack/data https://www.kaggle.com/c/nips-2017-defense-against-adversari...
- michaf 9y agoThe article mentions pooling layers as one source for the high frequency patterns. Jeffrey Hinton recently introduced capsule networks (https://news.ycombinator.com/item?id=15609402 https://news.ycombinator.com/item?id=15609402), in part because he wants to get rid of pooling layers. Maybe this approach is effective to counter (at least visually indistinguishable) adversarial examples. edit: ok, someone already tested it, and it does not seem to help that much: https://github.com/jaesik817/adv_attack_capsnet https://github.com/jaesik817/adv_attack_capsnet