3 ms·

What's new in this paper is that they do not have access to the neural net activations. They need only send it images, and get back the class the nnet predicts

• gjm11 11y ago
  But argonaut wasn't talking about having access to the neural net activations, but about having access to individual pixels in the image (which corresponds to individual photoreceptor cells in the human retina). Crafting these NN-defeating images seems to depend on tweaking individual pixels. I'm not sure how far it really does depend on that -- whether, e.g., you can use a more sophisticated version of the same approach to make an image that fools the NN reliably even if it's slightly offset, out of focus, stretched, etc. I wouldn't be astonished if you could. If not, that suggests an obvious defence mechanism: feed the NN not the raw image but several slightly perturbed ones, and take a majority vote or something. It's also worth observing that there are attacks on the human visual system, found by trying things out and observing their effects. We call them optical illusions, and some of them are very convincing.