3 ms·
Breaking Linear Classifiers on ImageNet
- jostmey 12y agoI'd be interested in seeing if this same problem appears in generative models. It seems like a flaw of discriminative models. Convolutional Boltzmann machines have been developed, and neural networks trained with Backpropagation can be pre-trained with generative models. Would this help alleviate the problem?
- Houshalter 12y agoThe original paper found that nets pretrained as autoencoders were the least susceptible to it, but still failed. IIRC the follow up paper (http://arxiv.org/abs/1412.6572 http://arxiv.org/abs/1412.6572) tested it or at least claimed it wasn't a solution.
- panic 12y agoCan our own visual system be fooled in a similar way, or is the response of the neurons involved too non-linear?
- dwiel 12y agoI think so, we just also have the capability to "double take" and pay closer attention and get a new point of view when the thing we're seeing is unexpected. Its hard to say if a similar hack would work on people since we can't easily run an high speed optimization over the human visual cortex. Maybe some mice?
- compbio 12y agoI do not think there is a one-to-one connection to be made between neural nets and the brain. That said, it is possible to fool our own visual system in a similar way: http://en.wikipedia.org/wiki/Gestalt_psychology#Properties http://en.wikipedia.org/wiki/Gestalt_psychology#Properties If you haven't seen this picture before: http://www.moillusions.com/wp-content/uploads/photos1.blogger.com/blogger/5639/2020/400/dalmation.gif http://www.moillusions.com/wp-content/uploads/photos1.blogge... it should appear as meaningless noise. After you spot the Dalmation dog, you will notice it in the future.
- Houshalter 12y agoPossibly. We can't backpropagate through all the neurons in a human brain. If we could we might be able to create real BLITs http://www.infinityplus.co.uk/stories/blit.htm http://www.infinityplus.co.uk/stories/blit.htm .
- bastomi29 12y agohttp://gunungberapiaktif.tripod.com/gunungberapiaktif/ http://gunungberapiaktif.tripod.com/gunungberapiaktif/
- irascible 12y agoWhat are the implications of this with self driving cars or biometric security devices? Also.. In the "incomprehensible noise" pattern classes, I can pretty clearly see features that I would identify as a robin, cheetah, armadillo, and lesser panda. I don't think this means the classifiers are being fooled, they are just not being trained to recognize the class of "noisy image that looks kind of like something but is actally noise"
- wodenokoto 12y agoI think the required noise / subspaces are unlikely to be met in real world scenarios. That is, you won't take a picture of a spider where the sun just happens to shine in the right way and your lens is in just the right angle to make an image recognized as a giraf. It may become an attack vector, but one the is very difficult to fit to the classifier you are targeting (your car may be using online learning, so there is a chance that an image generated on one self driving car won't work on another of the same model) then you have to show this carefully generated image to the car, but if you show it to the camera, it will already be distorted beyond its fooling capability. due to angle, lighting, lenses etc.
- wodenokoto 12y agoI tried replicated Nguyen et. al's experiment on MNIST. I decided to do it using different classifiers and that "state of the art" was less than 5 percent 0-1 error on testing set. I ended up with a ~98% accurate Gaussian kernel SVM classifier and my generated images all looked like numbers. I also got ~96% using KNN, but haven't figured out how I would measure confidence in such a classifier (average distance?) Any ideas?