3 ms·

To be clear, this is not overfitting. This is non-smooth (poor?) generalizability. Overfitting is related to, but not equivalent to, generalization. I'm very t

To be clear, this is not overfitting. This is non-smooth (poor?) generalizability. Overfitting is related to, but not equivalent to, generalization. I'm very that skeptical data augmentation is the answer to this. The space of adversarial examples is much larger than the space of true population examples. Boosting adversarial examples would only push the classifier to work really hard to properly classify examples that don't even belong to the population distribution. I suspect it would be equally susceptible to adversarial attacks.

p4wnc6 11y ago
Thank you, your comment along with the one further down by `Houshalter` helped me see something I was missing. I think it is still an interesting point to reflect on whether or not the adversarial examples "belong" in the population distribution or not. Take the example of the distorted Stop sign in the paper. It comes from the space of adversarial examples, but I think most of us would agree, as far as human perception goes, that that image should be considered part of whatever space comprises positive examples (if the system is trying to replicate human performance). Humans see it unequivocally as a Stop sign, so regardless of subtleties that make it non-natural, from the point of view of "I know it when I see it" it should be considered as if it were natural. So then the next question is whether all adversarial examples have to have that property. I mean, they obviously have to be good enough to fool the oracle, and potential any humans who check them out to see what's going on. In the other link that `Houshalter` shared, however, there is the example of random noise being classified as an ostrich. So in that case, we can see that input is non-sensical for the particular label. There would probably be some spectrum moving from obvious images to noise/gibberish images, all in the adversarial example space. And finding the line between them probably could not be any easier than the original learning problem in the first place. I'm just thinking out loud.
- argonaut 11y ago
  The counterpoint is that it does not belong to the space of naturally observed images.
  - p4wnc6 11y ago
    If an adversary made it and somehow got it in front of my eyeballs, that's natural enough for me.
    - argonaut 11y ago
      But this paper does not (metaphorically) get these things in front of the computer eyeballs (which would fall in the space of naturally observed images). It gets them directly implanted (metaphorically) on the computer's retinal photoreceptors (not in the space of naturally observed images). Perhaps that distinction is not important, but my guess is it is.
      - zardo 11y ago
        If you can find it on the internet, it occurs in the world we want programs to operate in.

p4wnc6 11y ago

Thank you, your comment along with the one further down by `Houshalter` helped me see something I was missing. I think it is still an interesting point to reflect on whether or not the adversarial examples "belong" in the population distribution or not. Take the example of the distorted Stop sign in the paper. It comes from the space of adversarial examples, but I think most of us would agree, as far as human perception goes, that that image should be considered part of whatever space comprises positive examples (if the system is trying to replicate human performance). Humans see it unequivocally as a Stop sign, so regardless of subtleties that make it non-natural, from the point of view of "I know it when I see it" it should be considered as if it were natural. So then the next question is whether all adversarial examples have to have that property. I mean, they obviously have to be good enough to fool the oracle, and potential any humans who check them out to see what's going on. In the other link that `Houshalter` shared, however, there is the example of random noise being classified as an ostrich. So in that case, we can see that input is non-sensical for the particular label. There would probably be some spectrum moving from obvious images to noise/gibberish images, all in the adversarial example space. And finding the line between them probably could not be any easier than the original learning problem in the first place. I'm just thinking out loud.

argonaut 11y ago
The counterpoint is that it does not belong to the space of naturally observed images.
- p4wnc6 11y ago
  If an adversary made it and somehow got it in front of my eyeballs, that's natural enough for me.
  - argonaut 11y ago
    But this paper does not (metaphorically) get these things in front of the computer eyeballs (which would fall in the space of naturally observed images). It gets them directly implanted (metaphorically) on the computer's retinal photoreceptors (not in the space of naturally observed images). Perhaps that distinction is not important, but my guess is it is.
    - zardo 11y ago
      If you can find it on the internet, it occurs in the world we want programs to operate in.