3 ms·

It’s really surprising to me how easily AI can be fooled. Maybe there is a fundamental difference between our visual system and what is represented in a visual

by car 6y ago

It’s really surprising to me how easily AI can be fooled. Maybe there is a fundamental difference between our visual system and what is represented in a visual recognition CNN. Could it be the complexity of billions of cells vs. the simplification of an AI, or something about the biology we haven’t yet accounted for?

iujjkfjdkkdkf 6y ago
We compare what we see against our internal model of the world, a CNN pattern matches, it doesn't think critically about the result that comes out.
TaylorAlexander 6y ago
Our CNN models are not meant to replicate the human visual system. They are a convenient mathematical tool that is quite unlike our brain. It’s not even necessary to use convolutions, though they remain the most popular method.
yumraj 6y ago
Because there is no I in AI. At a mile high conceptual level, AI is nothing but a program created by a computer based on the data it is provided Which is why it is extremely easy to fool using techniques that it is not trained to handle, today, but might be able to handle tomorrow. It is a race...
- car 6y ago
  Right. I’m naive in assigning more capability to these models than they posess. I found a quote from Geoff Hinton where he talked about this last year. From [1]: “I can take an image and a tiny bit of noise and CNNs will recognize it as something completely different and I can hardly see that it’s changed. That seems really bizarre and I take that as evidence that CNNs are actually using very different information from us to recognize images,” Hinton said in his keynote speech at the AAAI Conference. “It’s not that it’s wrong, they’re just doing it in a very different way, and their very different way has some differences in how it generalizes,” Hinton says. [1] https://bdtechtalks.com/2020/03/02/geoffrey-hinton-convnets-cnn-limits/ https://bdtechtalks.com/2020/03/02/geoffrey-hinton-convnets-...
- andrewjl 6y ago
  > Because there is no I in AI. This. In a nutshell every sort of algorithm we call "AI" today is reductive pattern matcher. This limitation isn't due to computational capacity or even, IMO, algorithm design, but due to our collective lack of understanding of how intelligence itself works. We'll get there eventually, but not for a long while.
- IshKebab 6y ago
  > Because there is no I in AI. This a common refrain but fairly obviously untrue. It assumes there's some secret sauce in human brains that makes us "intelligent" whereas AI is "just a machine". It's pretty clear that human brains are just programs. Extraordinarily complicated highly optimised programs, sure. But nobody has even found a shred of evidence that there's anything fundamentally different to programs in them. Thinking otherwise is along the same lines as thinking that animals don't have feelings. Every time there's an advance in AI the "it's not really intelligent" goalpost shifts. Clearly intelligence is a continuum.
  - car 6y ago
    I think you are right, it would be surprising if we discovered something outside our understanding of physics (god) to give us what we perceive as consciousness. It's been hypothesized that some unknown quantum effect might provide us this ability, but there is no indication of that yet. My general thought is, that since our brain is made of matter, it can be dissected and understood and eventually copied. Except, we are reverse engineering millions of years of evolution, which is exceedingly hard! We have had access to the information about all the proteins that make our brain cells for almost two decades, and still, their function has to be teased out in year long experiments. Not to speak of understanding the workings of the Homo Sapiens brain as a whole. It is also an analog 'computer', the performance of which we have no perceivable chance to match, at least in the near and most likely also distant future.
  - A12-B 6y ago
    I don't think AI is unintelligent, but human brains aren't just programs. We don't even have a definitive answer to how neuron signals are 'stored' in the brain. There is no equivalent to a memory address in there. It's an open problem. How could one say human brains are basically just programs if we don't even operate on the same hardware as programs, and the hardware we do operate on is too complex to be understood by us?
    - IshKebab 6y ago
      I think you might have misunderstood. When I say they're programs I mean that the answer a brain outputs could also be computed by a (sufficiently powerful) computer program. I don't mean that they're literally dealing in bits and bytes. Artificial neural networks don't have "memory addresses" to store data in the same way that a conventional program does either. But they can still store data. GPT-3 knows the first page of Harry Potter, but if you feel through its weights you won't find any of the text. The knowledge is distributed somehow (in a way that we don't fully understand). Despite that GPT-3 is clearly a program.
- LockAndLol 6y ago
  > At a mile high conceptual level, AI is nothing but a program created by a computer based on the data it is provided Your brain is but a preprogrammed, biological computer that reacts to data obtained from its interfaces and attach peripherals.
amelius 6y ago
There are optical illusions ... https://michaelbach.de/ot/ https://michaelbach.de/ot/
tomaszs 6y ago
Image recognition is currently done with so called weak UI. It was not a popular term, however it became such some years ago so that marketing can sell machine learning as an artificial intelligence. However machine learning is nowhere near what we consider as an AI, equivalent of our intelligence. You can compare machine learning with training a hamster to jump on a command. If you will repeat learning process a lot of time hamster will jump. But change anything in the environment and he won't. Machine learning is just a hamster that is trained thousands of times. It can do one thing, sometimes quite good, but still it is as intelligent as a hamster. Machine learning does not aim to become an intelligence. It is just a well trained hamster. Nothing more. It is just a fuzzy algorytm. That is why it is so easy to fool the algorytm. I hesitate to call machine learning any kind of AI just for the reason it generates such confusion. If it comes to developing real AI we are nowhere near currently. However we enjoy machine based models that are easier to brute force train with processing power he have today
colincooke 6y ago
It's very intersting, but once you understand how these attacks are performed, not too surprising (at least the attacks that I'm familiar with, there may be others that are different). Basically (and I'm drastically oversimplifying here), there are a class of succesful techniques that can run an optimization to maximize classification loss for a certain model architecture/parameter set by modulating the content of the image (for example by adding noise). This loss critically depends on having the parameters of the model in hand, and is very hard to avoid. So it's a little more sophisticated than just adding random noise, it's adding very specific quantities of noise to very specific locations, which are based on perfect knowledge of how the predictive system (the deep model) works. Is this stuff interesting? Absolutely. Is it worth studying? Yes, again. Does it mean that CNNs as we know them are poor computer vision systems and fundementally flawed? No. It's a limitation of existing deep models, and one which may be overcome eventually.
kerng 6y ago
This talk from Grayhat conference last year gave me a lot of insights on how powerful but brittle ML is: https://m.youtube.com/watch?v=-SV80sIBhqY https://m.youtube.com/watch?v=-SV80sIBhqY

by car 6y ago

iujjkfjdkkdkf 6y ago

We compare what we see against our internal model of the world, a CNN pattern matches, it doesn't think critically about the result that comes out.

TaylorAlexander 6y ago

Our CNN models are not meant to replicate the human visual system. They are a convenient mathematical tool that is quite unlike our brain. It’s not even necessary to use convolutions, though they remain the most popular method.

yumraj 6y ago

Because there is no I in AI. At a mile high conceptual level, AI is nothing but a program created by a computer based on the data it is provided Which is why it is extremely easy to fool using techniques that it is not trained to handle, today, but might be able to handle tomorrow. It is a race...

car 6y ago
Right. I’m naive in assigning more capability to these models than they posess. I found a quote from Geoff Hinton where he talked about this last year. From [1]: “I can take an image and a tiny bit of noise and CNNs will recognize it as something completely different and I can hardly see that it’s changed. That seems really bizarre and I take that as evidence that CNNs are actually using very different information from us to recognize images,” Hinton said in his keynote speech at the AAAI Conference. “It’s not that it’s wrong, they’re just doing it in a very different way, and their very different way has some differences in how it generalizes,” Hinton says. [1] https://bdtechtalks.com/2020/03/02/geoffrey-hinton-convnets-cnn-limits/ https://bdtechtalks.com/2020/03/02/geoffrey-hinton-convnets-...
andrewjl 6y ago
> Because there is no I in AI. This. In a nutshell every sort of algorithm we call "AI" today is reductive pattern matcher. This limitation isn't due to computational capacity or even, IMO, algorithm design, but due to our collective lack of understanding of how intelligence itself works. We'll get there eventually, but not for a long while.
IshKebab 6y ago
> Because there is no I in AI. This a common refrain but fairly obviously untrue. It assumes there's some secret sauce in human brains that makes us "intelligent" whereas AI is "just a machine". It's pretty clear that human brains are just programs. Extraordinarily complicated highly optimised programs, sure. But nobody has even found a shred of evidence that there's anything fundamentally different to programs in them. Thinking otherwise is along the same lines as thinking that animals don't have feelings. Every time there's an advance in AI the "it's not really intelligent" goalpost shifts. Clearly intelligence is a continuum.
- car 6y ago
  I think you are right, it would be surprising if we discovered something outside our understanding of physics (god) to give us what we perceive as consciousness. It's been hypothesized that some unknown quantum effect might provide us this ability, but there is no indication of that yet. My general thought is, that since our brain is made of matter, it can be dissected and understood and eventually copied. Except, we are reverse engineering millions of years of evolution, which is exceedingly hard! We have had access to the information about all the proteins that make our brain cells for almost two decades, and still, their function has to be teased out in year long experiments. Not to speak of understanding the workings of the Homo Sapiens brain as a whole. It is also an analog 'computer', the performance of which we have no perceivable chance to match, at least in the near and most likely also distant future.
- A12-B 6y ago
  I don't think AI is unintelligent, but human brains aren't just programs. We don't even have a definitive answer to how neuron signals are 'stored' in the brain. There is no equivalent to a memory address in there. It's an open problem. How could one say human brains are basically just programs if we don't even operate on the same hardware as programs, and the hardware we do operate on is too complex to be understood by us?
  - IshKebab 6y ago
    I think you might have misunderstood. When I say they're programs I mean that the answer a brain outputs could also be computed by a (sufficiently powerful) computer program. I don't mean that they're literally dealing in bits and bytes. Artificial neural networks don't have "memory addresses" to store data in the same way that a conventional program does either. But they can still store data. GPT-3 knows the first page of Harry Potter, but if you feel through its weights you won't find any of the text. The knowledge is distributed somehow (in a way that we don't fully understand). Despite that GPT-3 is clearly a program.
LockAndLol 6y ago
> At a mile high conceptual level, AI is nothing but a program created by a computer based on the data it is provided Your brain is but a preprogrammed, biological computer that reacts to data obtained from its interfaces and attach peripherals.

amelius 6y ago

There are optical illusions ... https://michaelbach.de/ot/ https://michaelbach.de/ot/

tomaszs 6y ago

Image recognition is currently done with so called weak UI. It was not a popular term, however it became such some years ago so that marketing can sell machine learning as an artificial intelligence. However machine learning is nowhere near what we consider as an AI, equivalent of our intelligence. You can compare machine learning with training a hamster to jump on a command. If you will repeat learning process a lot of time hamster will jump. But change anything in the environment and he won't. Machine learning is just a hamster that is trained thousands of times. It can do one thing, sometimes quite good, but still it is as intelligent as a hamster. Machine learning does not aim to become an intelligence. It is just a well trained hamster. Nothing more. It is just a fuzzy algorytm. That is why it is so easy to fool the algorytm. I hesitate to call machine learning any kind of AI just for the reason it generates such confusion. If it comes to developing real AI we are nowhere near currently. However we enjoy machine based models that are easier to brute force train with processing power he have today

colincooke 6y ago

It's very intersting, but once you understand how these attacks are performed, not too surprising (at least the attacks that I'm familiar with, there may be others that are different). Basically (and I'm drastically oversimplifying here), there are a class of succesful techniques that can run an optimization to maximize classification loss for a certain model architecture/parameter set by modulating the content of the image (for example by adding noise). This loss critically depends on having the parameters of the model in hand, and is very hard to avoid. So it's a little more sophisticated than just adding random noise, it's adding very specific quantities of noise to very specific locations, which are based on perfect knowledge of how the predictive system (the deep model) works. Is this stuff interesting? Absolutely. Is it worth studying? Yes, again. Does it mean that CNNs as we know them are poor computer vision systems and fundementally flawed? No. It's a limitation of existing deep models, and one which may be overcome eventually.

kerng 6y ago

This talk from Grayhat conference last year gave me a lot of insights on how powerful but brittle ML is: https://m.youtube.com/watch?v=-SV80sIBhqY https://m.youtube.com/watch?v=-SV80sIBhqY