4 ms·
Abstract: Given a state-of-the-art deep neural network classifier, we show the existence of a universal (image-agnostic) and very small perturbation vector that
by legatus 10y ago
Abstract: Given a state-of-the-art deep neural network classifier, we show the existence of a universal (image-agnostic) and very small perturbation vector that causes natural images to be misclassified with high probability. We propose a systematic algorithm for computing universal perturbations, and show that state-of-the-art deep neural networks are highly vulnerable to such perturbations, albeit being quasi-imperceptible to the human eye. We further empirically analyze these universal perturbations and show, in particular, that they generalize very well across neural networks. The surprising existence of universal perturbations reveals important geometric correlations among the high-dimensional decision boundary of classifiers. It further outlines potential security breaches with the existence of single directions in the input space that adversaries can possibly exploit to break a classifier on most natural images.
- kough 10y agoSuper interesting. I'm on mobile and haven't had time to read the whole paper yet - would it be feasible to continuously compute these perturbation vectors during training and include them as part of a larger heuristic? For instance, to incorporate the objective of maximizing the size of the perturbation vector necessary for misclassification? The goal being to end up with a net that is more resistant to such perturbations.
- sherjilozair 10y agoShort answer: No. Computing these perturbations requires an expensive optimization with multiple passes through the dataset, and this would be prohibitively expensive to do in the inner-most loop of training. There are other work in the literature describing faster algorithms to compute these perturbations, which makes it possible to use them while training. See, eg.: https://arxiv.org/abs/1412.6572 https://arxiv.org/abs/1412.6572
- joshuawarner32 10y agoSee also: https://arxiv.org/abs/1511.04599 https://arxiv.org/abs/1511.04599 IMO, (at least) two pieces of research on the subject means that the short answer really is "yes". Maybe not the exact technique used in the paper in the original post, but conceptually similar techniques.
- sherjilozair 10y agoIt's easier to find fooling perturbations of one image, but not of the whole dataset. I assumed the question was can we use the universal perturbations for robust training? The answer to that is still "no", I think.
- Hydraulix989 10y agoI suspect that doing this will only get you one step ahead in a cat-and-mouse game. You're only fighting one kind of perturbation.