3 ms·

Ensembles are also vulnerable to adversarial images. From one of the canonical papers on adversarial images: > In addition, the specific nature of these pert

Ensembles are also vulnerable to adversarial images. From one of the canonical papers on adversarial images: > In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input. > ... suggest that adversarial examples are somewhat universal and not just the results of overfitting to a particular model or to the specific selection of the training set https://arxiv.org/abs/1312.6199 https://arxiv.org/abs/1312.6199 More recent research shows that one can make adversarial images which fool ensembles. Once you have these images there is a very high chance it will also fool any individual architecture you pass it on to next. https://twitter.com/goodfellow_ian/status/790330213963923456 https://twitter.com/goodfellow_ian/status/790330213963923456 These attacks show the inherent flaws of ML models (not just convnets, as per http://karpathy.github.io/2015/03/30/breaking-convnets/ http://karpathy.github.io/2015/03/30/breaking-convnets/ ), and as far as I know, they are not trivial to work around. Research on adversarial images is still very active, in part because the military would like to rely on deep learning, and does not like to mistake a tank for a puppy (and vice versa).

nl 10y ago
https://arxiv.org/abs/1412.6572 https://arxiv.org/abs/1412.6572 is a Goodfellow et. al. (ie, the person who's Tweet you note above) paper where they show adversarial examples do apply across architectures (as you state above), but they show that using the adversarial examples during training has a regularization effect and improves performance: https://arxiv.org/pdf/1511.05432v3.pdf https://arxiv.org/pdf/1511.05432v3.pdf takes a similar approach: We show that adversarial training of ANNs is in fact robustification of the network optimization, and that our proposed framework generalizes previous approaches for increasing local stability of ANNs. Experimental results reveal that our approach increases the robustness of the network to existing adversarial examples, while making it harder to generate new ones. Furthermore, our algorithm improves the accuracy of the network also on the original test data. I agree this is an area of active research, but not that it is an inherent flaw.

nl 10y ago

https://arxiv.org/abs/1412.6572 https://arxiv.org/abs/1412.6572 is a Goodfellow et. al. (ie, the person who's Tweet you note above) paper where they show adversarial examples do apply across architectures (as you state above), but they show that using the adversarial examples during training has a regularization effect and improves performance: https://arxiv.org/pdf/1511.05432v3.pdf https://arxiv.org/pdf/1511.05432v3.pdf takes a similar approach: We show that adversarial training of ANNs is in fact robustification of the network optimization, and that our proposed framework generalizes previous approaches for increasing local stability of ANNs. Experimental results reveal that our approach increases the robustness of the network to existing adversarial examples, while making it harder to generate new ones. Furthermore, our algorithm improves the accuracy of the network also on the original test data. I agree this is an area of active research, but not that it is an inherent flaw.