14 ms·
Short answer: No. Computing these perturbations requires an expensive optimization with multiple passes through the dataset, and this would be prohibitively exp
by sherjilozair 10y ago
Short answer: No. Computing these perturbations requires an expensive optimization with multiple passes through the dataset, and this would be prohibitively expensive to do in the inner-most loop of training.
There are other work in the literature describing faster algorithms to compute these perturbations, which makes it possible to use them while training. See, eg.: https://arxiv.org/abs/1412.6572 https://arxiv.org/abs/1412.6572
- joshuawarner32 10y agoSee also: https://arxiv.org/abs/1511.04599 https://arxiv.org/abs/1511.04599 IMO, (at least) two pieces of research on the subject means that the short answer really is "yes". Maybe not the exact technique used in the paper in the original post, but conceptually similar techniques.
- sherjilozair 10y agoIt's easier to find fooling perturbations of one image, but not of the whole dataset. I assumed the question was can we use the universal perturbations for robust training? The answer to that is still "no", I think.
- Hydraulix989 10y agoI suspect that doing this will only get you one step ahead in a cat-and-mouse game. You're only fighting one kind of perturbation.