5 ms·

Adversarial noise is becoming quite a crises in the deep learning field and causing lot of heated debates. There have been effort for defenses using distillatio

Adversarial noise is becoming quite a crises in the deep learning field and causing lot of heated debates. There have been effort for defenses using distillation and denoised auto-encoders but only thing that has worked well is actually training on adversarial noise itself. However the bigger insight is that deep networks aren't learning high level features. This paper has very strong evidence that deep networks are just more fancy statistical models as opposed to developing more abstract representations. When if you make network more deeper, you are just developing more higher capacity models as opposed to more higher level features. This is quite damning... If you were hoping current form of deep learning will open the door to AGI, your hopes should be shattered by now. Some of the things to investigate are (1) can proper regularization help turn this around (for example, forcing gradients to be as small as possible) (2) are there any other architectures using gradient descent that might actually develop abstract features. This is definitely crisis time in the field and therefore more interesting than ever :).

red75prime 9y ago
It's also interesting whether there are people with apperceptive visual agnosia who are susceptible to adversarial examples.
- taeric 9y ago
  Agreed it is an interesting question. Humans are just far less likely to build a mono culture of vision capabilities. Such that the applicability of any adversarial examples is likely far more limited.
- sharemywin 9y ago
  aren't optical illusions human adversarial examples?
  - hackinthebochs 9y ago
    It's probably fair to see them as such, but they operate in the semantic domain rather than the pixel domain.
    - taeric 9y ago
      Do they? Specifically the graffiti style illusions that appear 3d to people. Those seem to work by specifically lining up the "pixels" you see.
      - hackinthebochs 9y ago
        It's not a seemingly random (meaningless) perturbations that cause misclassification/misrecognition, but a very specific (meaningful) tuning of features that serve to trigger some misrecognition. The distinction here is whether the modified features themselves have semantic information (i.e. have a representation in the semantic domain with respect to the content of the image). In the case of graffiti illusions, it seems that it does: the misrecognition is due to a particular kind of coherence with the observers perspective and the relative positioning/alignment of the features in the image. Both perspective and relative alignment of features is meaningful.
        moultano 9y ago
        There's no semantic reason why we see this image as moving, but everyone does, even animals. http://24.p3k.hu/app/uploads/2014/05/Rotsnake1.gif http://24.p3k.hu/app/uploads/2014/05/Rotsnake1.gif
        hackinthebochs 9y ago
        >There's no semantic reason It's hard to be certain because we don't know the details of our visual system.
        taeric 9y ago
        I think you are somewhat shifting the goalposts, though. For adversarial images, you are changing what you know the system is looking at. In such a way that confuses it to "see" something else. For humans, we just happen to somewhat understand the image that is being used against us. How is this different?
nerdponx 9y ago
more fancy statistical models This would be true even if there were intermediate representations being learned.
fnbr 9y ago
Adversarial noise is interesting, but I think it's more of a toy problem that will go away as we switch to more sophisticated models with life-long learning that are able to develop complicated, hierarchical/compositional classifications. For instance, there was a talk at NIPS this year where the researcher developed virtual stickers that can be added to a photo and cause the image to be misclassified (eg you have a "banana" sticker that you can apply to any picture to make the classifier classify that picture as a banana). The underlying issue, IMO, is that the classifier has only a limited number of categories from which to choose, and can't develop new categories (such as "adversarially perturbed picture"). The modified image does look more like a banana than anything else, and as the classifier never trained on adversarial images as a separate category it doesn't really make sense to make any other classifications.

red75prime 9y ago

It's also interesting whether there are people with apperceptive visual agnosia who are susceptible to adversarial examples.

taeric 9y ago
Agreed it is an interesting question. Humans are just far less likely to build a mono culture of vision capabilities. Such that the applicability of any adversarial examples is likely far more limited.
sharemywin 9y ago
aren't optical illusions human adversarial examples?
- hackinthebochs 9y ago
  It's probably fair to see them as such, but they operate in the semantic domain rather than the pixel domain.
  - taeric 9y ago
    Do they? Specifically the graffiti style illusions that appear 3d to people. Those seem to work by specifically lining up the "pixels" you see.
    - hackinthebochs 9y ago
      It's not a seemingly random (meaningless) perturbations that cause misclassification/misrecognition, but a very specific (meaningful) tuning of features that serve to trigger some misrecognition. The distinction here is whether the modified features themselves have semantic information (i.e. have a representation in the semantic domain with respect to the content of the image). In the case of graffiti illusions, it seems that it does: the misrecognition is due to a particular kind of coherence with the observers perspective and the relative positioning/alignment of the features in the image. Both perspective and relative alignment of features is meaningful.
      - moultano 9y ago
        There's no semantic reason why we see this image as moving, but everyone does, even animals. http://24.p3k.hu/app/uploads/2014/05/Rotsnake1.gif http://24.p3k.hu/app/uploads/2014/05/Rotsnake1.gif
        hackinthebochs 9y ago
        >There's no semantic reason It's hard to be certain because we don't know the details of our visual system.
      - taeric 9y ago
        I think you are somewhat shifting the goalposts, though. For adversarial images, you are changing what you know the system is looking at. In such a way that confuses it to "see" something else. For humans, we just happen to somewhat understand the image that is being used against us. How is this different?

nerdponx 9y ago

more fancy statistical models This would be true even if there were intermediate representations being learned.

fnbr 9y ago

Adversarial noise is interesting, but I think it's more of a toy problem that will go away as we switch to more sophisticated models with life-long learning that are able to develop complicated, hierarchical/compositional classifications. For instance, there was a talk at NIPS this year where the researcher developed virtual stickers that can be added to a photo and cause the image to be misclassified (eg you have a "banana" sticker that you can apply to any picture to make the classifier classify that picture as a banana). The underlying issue, IMO, is that the classifier has only a limited number of categories from which to choose, and can't develop new categories (such as "adversarially perturbed picture"). The modified image does look more like a banana than anything else, and as the classifier never trained on adversarial images as a separate category it doesn't really make sense to make any other classifications.