4 ms·

I don't think adversarial examples give any evidence of relevant problems with these models because they occur on a very specific subset of images that can only

I don't think adversarial examples give any evidence of relevant problems with these models because they occur on a very specific subset of images that can only be discovered using detailed knowledge of how these networks process images. For all we know, humans have similar problems on some obscure subset of images, but we can't find human's adversarial examples because we don't have detailed knowledge of how the brain processes images.

adrianN 8y ago
I think adversarial examples for humans are called "optical illusions".
- ccvannorman 8y ago
  There is a categorical difference between "a [specifically designed] image that can be construed as a duck or a rabbit" and "a human can regularly mis-categorize random pictures of ducks as rabbits if a weird filter is overlayed". The first is well-known and fun and trite -- the second is unheard of and probably impossible for humans, yet provably possible for trained computers.
  - fons 8y ago
    The point outlined is that we don't know enough about how we identify objects to discard a simple adversarial attack; probably not a filter-based but maybe something else.
  - cameldrv 8y ago
    It's called camoflauge. The natural world is full of adversarial examples.
  - dontreact 8y ago
    "probably impossible for humans" Based on what?
  - fenomas 8y ago
    I'd imagine GP was referring to "humans perceive straight lines to be curved when certain shapes are overlayed", or "humans perceive shapes of the same color to be different colors when filters are applied" sorts of optical illusions. There are plenty of those, and I personally I think they're probably analogous to how adversarial filters fool AI classifiers.
  - adrianN 8y ago
    It's really easy to cause humans to misclassify all kinds of images as containing faces ;). Humans also regularly misclassify random noise as words. You can even suggest which words we hear by telling us what the noise is supposed to be.
  - deleted 8y ago
    [deleted]
- ars 8y ago
  The difference is that humans are aware that there is an illusion happening, they just can't help seeing it.
  - adaml_623 8y ago
    Or sometimes not aware https://www.dw.com/en/man-falls-into-black-hole-art-exhibit-in-portugal/a-45197790 https://www.dw.com/en/man-falls-into-black-hole-art-exhibit-...
mmirate 8y ago
The important difference here is that most adversarial examples for the human perception: (a) do not occur frequently in nature, (b) are not frequently - if at all - produced in man-made architecture or transit-constructions, (c) often contain repetitive and regular geometric and chromatic patterns which further make them stand out from everything else, and (d) practically cannot be produced by digital (ergo noisy/less-than-perfect) images of any common real-world scenario. In short: optical illusions don't accidentally occur in places where they can be seen by meatbag drivers.
- dontreact 8y ago
  I don’t see how you can make any claim about “most human adversarial examples”. There is a huge space of images and we have explored a negligible part of it. Also a) and b) empirically seem to be true of the test sets people have collected thus far of the natural world for these models. In short, we have no evidence that adversarial examples of the type being studied occur commonly in images collected by self driving cars.
  - mannykannot 8y ago
    The issue with regard to self-driving cars is that these cases demonstrate a disturbing level of fragility: we don't have a good handle on where the boundary between acceptable and chaotic responses lies. You hypothesize that there are comparable examples for humans somewhere out there in the domain of all possible images, but the fact that, for all the countless cases of people looking at things that have occurred in humanity's existence, no-one has found a good example, suggests that, from the pragmatic point of view that you propose, image-recognition software has some catching-up to do. Maybe a system that seeks consensus among several differently-trained models would be more robust.
    - de_watcher 8y ago
      The difference is that you can calculate an adversarial example for our classifiers, but it's too slow to calculate on a human. Even if you could, the result would be specific to that particular person, so it won't work as good on others. And these bastards learn while you're constructing the example (which isn't fair at all to a helpless classifier that's just sitting there and doesn't change).
mannykannot 8y ago
I think we do actually know enough about how the human mind does process images to have some idea of what is different. It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it, such as making hypotheses as to what's going on and checking them out, and/or seeking to get a better view (or other evidence) in a way that is specifically designed to resolve the uncertainty. It is this higher-level semantic analysis that is missing from current image processing software. In these discussions, someone always mentions optical illusions, but only humans (so far) understand the concept of 'optical illusion', and recognize that they are experiencing them.
- thaumasiotes 8y ago
  > It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it This is true, but step one is "move your head" (or in your words, "get a better view" -- but you get more value from just the fact that your head is in a different place than from the possibility of a better angle on whatever you're looking at). That strategy doesn't work at all when you're trying to classify static images rather than physical objects.
  - TeMPOraL 8y ago
    GP's statement applies as much to observing objects in 3D space as it does to looking at photos, where just moving your head ain't gonna help you much. Optical illusions are great to study this process, because most of them are delivered in form of flat images on paper or computer screen.
    - thaumasiotes 8y ago
      Optical illusions are delivered as flat images because moving your head doesn't affect those.
ben_w 8y ago
Recent update — we do know, and humans are vulnerable to perturbations created to fool a multitude of existing AI: https://arxiv.org/pdf/1802.08195.pdf https://arxiv.org/pdf/1802.08195.pdf
UncleMeat 8y ago
Not true. There are black and grey box adversarial techniques as well.

adrianN 8y ago

I think adversarial examples for humans are called "optical illusions".

ccvannorman 8y ago
There is a categorical difference between "a [specifically designed] image that can be construed as a duck or a rabbit" and "a human can regularly mis-categorize random pictures of ducks as rabbits if a weird filter is overlayed". The first is well-known and fun and trite -- the second is unheard of and probably impossible for humans, yet provably possible for trained computers.
- fons 8y ago
  The point outlined is that we don't know enough about how we identify objects to discard a simple adversarial attack; probably not a filter-based but maybe something else.
- cameldrv 8y ago
  It's called camoflauge. The natural world is full of adversarial examples.
- dontreact 8y ago
  "probably impossible for humans" Based on what?
- fenomas 8y ago
  I'd imagine GP was referring to "humans perceive straight lines to be curved when certain shapes are overlayed", or "humans perceive shapes of the same color to be different colors when filters are applied" sorts of optical illusions. There are plenty of those, and I personally I think they're probably analogous to how adversarial filters fool AI classifiers.
- adrianN 8y ago
  It's really easy to cause humans to misclassify all kinds of images as containing faces ;). Humans also regularly misclassify random noise as words. You can even suggest which words we hear by telling us what the noise is supposed to be.
- deleted 8y ago
  [deleted]
ars 8y ago
The difference is that humans are aware that there is an illusion happening, they just can't help seeing it.
- adaml_623 8y ago
  Or sometimes not aware https://www.dw.com/en/man-falls-into-black-hole-art-exhibit-in-portugal/a-45197790 https://www.dw.com/en/man-falls-into-black-hole-art-exhibit-...

mmirate 8y ago

The important difference here is that most adversarial examples for the human perception: (a) do not occur frequently in nature, (b) are not frequently - if at all - produced in man-made architecture or transit-constructions, (c) often contain repetitive and regular geometric and chromatic patterns which further make them stand out from everything else, and (d) practically cannot be produced by digital (ergo noisy/less-than-perfect) images of any common real-world scenario. In short: optical illusions don't accidentally occur in places where they can be seen by meatbag drivers.

dontreact 8y ago
I don’t see how you can make any claim about “most human adversarial examples”. There is a huge space of images and we have explored a negligible part of it. Also a) and b) empirically seem to be true of the test sets people have collected thus far of the natural world for these models. In short, we have no evidence that adversarial examples of the type being studied occur commonly in images collected by self driving cars.
- mannykannot 8y ago
  The issue with regard to self-driving cars is that these cases demonstrate a disturbing level of fragility: we don't have a good handle on where the boundary between acceptable and chaotic responses lies. You hypothesize that there are comparable examples for humans somewhere out there in the domain of all possible images, but the fact that, for all the countless cases of people looking at things that have occurred in humanity's existence, no-one has found a good example, suggests that, from the pragmatic point of view that you propose, image-recognition software has some catching-up to do. Maybe a system that seeks consensus among several differently-trained models would be more robust.
  - de_watcher 8y ago
    The difference is that you can calculate an adversarial example for our classifiers, but it's too slow to calculate on a human. Even if you could, the result would be specific to that particular person, so it won't work as good on others. And these bastards learn while you're constructing the example (which isn't fair at all to a helpless classifier that's just sitting there and doesn't change).

mannykannot 8y ago

I think we do actually know enough about how the human mind does process images to have some idea of what is different. It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it, such as making hypotheses as to what's going on and checking them out, and/or seeking to get a better view (or other evidence) in a way that is specifically designed to resolve the uncertainty. It is this higher-level semantic analysis that is missing from current image processing software. In these discussions, someone always mentions optical illusions, but only humans (so far) understand the concept of 'optical illusion', and recognize that they are experiencing them.

thaumasiotes 8y ago
> It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it This is true, but step one is "move your head" (or in your words, "get a better view" -- but you get more value from just the fact that your head is in a different place than from the possibility of a better angle on whatever you're looking at). That strategy doesn't work at all when you're trying to classify static images rather than physical objects.
- TeMPOraL 8y ago
  GP's statement applies as much to observing objects in 3D space as it does to looking at photos, where just moving your head ain't gonna help you much. Optical illusions are great to study this process, because most of them are delivered in form of flat images on paper or computer screen.
  - thaumasiotes 8y ago
    Optical illusions are delivered as flat images because moving your head doesn't affect those.

ben_w 8y ago

Recent update — we do know, and humans are vulnerable to perturbations created to fool a multitude of existing AI: https://arxiv.org/pdf/1802.08195.pdf https://arxiv.org/pdf/1802.08195.pdf

UncleMeat 8y ago

Not true. There are black and grey box adversarial techniques as well.