37 ms·

Cool illusion; I figured it out and so will most humans. I don't think the claim that these sorts of illusions are of the same nature as, say, single pixel atta

Cool illusion; I figured it out and so will most humans. I don't think the claim that these sorts of illusions are of the same nature as, say, single pixel attacks against neural networks is justified. On the other hand, it is also true that it's not clear whether such attacks on our current crop of primitive neural networks will work on more advanced autonomous systems that regulate themselves.

yters 9y ago
In theory it seems that with sufficient understanding of any ML system, regardless of sophistication, such examples can be constructed. It is essentially the brain in the vat problem. Computers have no access to objective reality, whereas we cannot say the same regarding the human mind without making strong assumptions about the nature of the human mind. To put this in comp sci language, we can think of DNNs as proofs that certain inputs belong to certain classes. Since no proof system can be both complete and consistent, something outside the proof system can always either provide unprovable examples or adversarial examples.
- danharaj 9y ago
  I think the proper test is how stable the learning system is in the face of adversarial examples. I have no indications either way that we could, using current ideas, create a system that learns and is autonomous like a human brain while regulating itself from going too far off the rails. An optical illusion won't lead to you thinking you're the king of France, nor do there seem to be basilisks that will actually crash a brain. It remains to be seen if the same holds for artificial learning systems. Of course! The gap between current learning systems and even the brain of a rodent is vast and we have yet no idea how to get there.
  - yters 9y ago
    The incompleteness/inconsistency theorem I mention seems to indicate no such ML system is possible. It will either encounter examples it cannot deal with, or it will be deceived. ML can only be guaranteed to work within very limited domains.
  - munificent 9y ago
    > An optical illusion won't lead to you thinking you're the king of France I'm not sure what you're getting at here. By definition, optical illusions are perceptual mistakes limited to the optical system. It is certainly possible for an optical illusion to trick someone into doing something they don't intend to do because they are fooled into believing the illusion. Imagine painting a set of stairs with a disorienting pattern of shading that makes them appear off, leading you to trip and fall down them. I don't think it's necessarily for a perceptual illusion to hijack your entire belief system in order to be considered an effective adversarial attack.
    - yters 9y ago
      The whole point of these examples is the DNN gives erroneous classifications with near complete certainty. This looks to me like hijacking a belief system.

yters 9y ago

In theory it seems that with sufficient understanding of any ML system, regardless of sophistication, such examples can be constructed. It is essentially the brain in the vat problem. Computers have no access to objective reality, whereas we cannot say the same regarding the human mind without making strong assumptions about the nature of the human mind. To put this in comp sci language, we can think of DNNs as proofs that certain inputs belong to certain classes. Since no proof system can be both complete and consistent, something outside the proof system can always either provide unprovable examples or adversarial examples.

danharaj 9y ago
I think the proper test is how stable the learning system is in the face of adversarial examples. I have no indications either way that we could, using current ideas, create a system that learns and is autonomous like a human brain while regulating itself from going too far off the rails. An optical illusion won't lead to you thinking you're the king of France, nor do there seem to be basilisks that will actually crash a brain. It remains to be seen if the same holds for artificial learning systems. Of course! The gap between current learning systems and even the brain of a rodent is vast and we have yet no idea how to get there.
- yters 9y ago
  The incompleteness/inconsistency theorem I mention seems to indicate no such ML system is possible. It will either encounter examples it cannot deal with, or it will be deceived. ML can only be guaranteed to work within very limited domains.
- munificent 9y ago
  > An optical illusion won't lead to you thinking you're the king of France I'm not sure what you're getting at here. By definition, optical illusions are perceptual mistakes limited to the optical system. It is certainly possible for an optical illusion to trick someone into doing something they don't intend to do because they are fooled into believing the illusion. Imagine painting a set of stairs with a disorienting pattern of shading that makes them appear off, leading you to trip and fall down them. I don't think it's necessarily for a perceptual illusion to hijack your entire belief system in order to be considered an effective adversarial attack.
  - yters 9y ago
    The whole point of these examples is the DNN gives erroneous classifications with near complete certainty. This looks to me like hijacking a belief system.