3 ms·
It's very intersting, but once you understand how these attacks are performed, not too surprising (at least the attacks that I'm familiar with, there may be oth
by colincooke 6y ago
It's very intersting, but once you understand how these attacks are performed, not too surprising (at least the attacks that I'm familiar with, there may be others that are different). Basically (and I'm drastically oversimplifying here), there are a class of succesful techniques that can run an optimization to maximize classification loss for a certain model architecture/parameter set by modulating the content of the image (for example by adding noise). This loss critically depends on having the parameters of the model in hand, and is very hard to avoid.
So it's a little more sophisticated than just adding random noise, it's adding very specific quantities of noise to very specific locations, which are based on perfect knowledge of how the predictive system (the deep model) works.
Is this stuff interesting? Absolutely. Is it worth studying? Yes, again. Does it mean that CNNs as we know them are poor computer vision systems and fundementally flawed? No. It's a limitation of existing deep models, and one which may be overcome eventually.