7 ms·

This latest craze of "AI" research seems to be fueled by a sudden glut of computational power (GPUs) that wasn't available previously. I think that most technic

This latest craze of "AI" research seems to be fueled by a sudden glut of computational power (GPUs) that wasn't available previously. I think that most technical people would agree that the mid 2020s is extremely ambitious. I'd also argue that we're actually more likely to experience another AI winter. The frightening part of the current deep learning research is how susceptible they are to adversarial attacks. Adding small amounts of noise causes misclassification in images, and some papers even explore the inevitability of adversarial examples [1]. This is especially frightening given the amount of autonomous vehicle work being done. I could imagine a situation in which the sensor noise varies just enough to cause such an error. Obviously, the systems will have redundancies built in, but I'm convinced the self-driving cars are still a ways off as well. EDIT: As others, have stated just adding noise is not enough and it is often used to generalize the model. The paper does discuss that the perturbations can be incredibly small to cause this deviation and that the set of such deviations may be larger than expected especially for complex images. Regarding the AI winter, I suppose I should have defined it as a reduction in the amount of research and the extent of the progress being made in the area rather than the utility of such research. [1] https://arxiv.org/abs/1809.02104 https://arxiv.org/abs/1809.02104

njoubert 8y ago
This is a common misconception. It is not a small amount of noise that causes misclassification of images. It is a carefully designed and quite unique pattern that causes misclassification. It only looks like noise to the human eye, but it really isn't. Yes, neural networks are susceptible to adversarial attacks. No, just adding noise to an image doesn't break neural networks.
- pmoriarty 8y ago
  Adding small amounts of noise is actually sometimes used to improve the performance of various AI techniques. It helps prevent overfitting. In fact, if your technique or model is seriously affected by a little noise this is usually enough to brand it brittle and maybe even a failure, as it's a sign of overfitting. Anyone working in this field knows to look for this and will try to make what they create more robust. The design of visual captchas is one obvious indication of just how successful AI techniques have been at image recognition in the presence of noise. It's no longer enough to make them a little noisy. In order to resist being solved by mechanical means, visual captchas have to include so much noise that even humans have problems recognizing them.
- ipsa 8y ago
  Read it as: A small amount of carefully constructed noise. Then you are correct to literature and pop-science. No misconception needed. There are 1-pixel attacks now. Randomly shuffling a small amount of pixels around can cause predictions to shift. The issue is that there is no scene understanding. No common sense. No 3D modeling. Just 10x10 pattern matching on a very large fuzzy database of natural images (which works really really well in most cases). The hype of ML is driven by 3 things: Big companies vying for AI dominance, militaries that want to finally use neural nets that work, and international competition between the West and the East to be the first to largely automate their economies (or AGI if you want to call it that). Catalysts were big data hoarding, GPU training on ImageNet, and then AlphaGo.
  - SubiculumCode 8y ago
    I am not a machine learning expert, but could not these adversarial example issues be resolved by solving an image classification problem by (1)producing multiple non-equivalent classification solutions with adequate accuracy, then (2)fusion (e.g. voting) to produce a consensus classification? (3) Maybe random shuffling of which X of Z solutions get to vote in each classification attempt. What might fool one solution might not fool another, and adversarial examples seem to depend on idiosyncrasies of a particular solution.
    - hahajk 8y ago
      Or just adding a small amount of random noise to the input, which would wipe out the carefully constructed attack.
      - ipsa 8y ago
        You can try out this technique at https://github.com/google/unrestricted-adversarial-examples https://github.com/google/unrestricted-adversarial-examples My guess is it would have the same result as adding noise to the normal images too (resulting in a slightly worse performance overall).
    - freeone3000 8y ago
- apatters 8y ago
  Can I have a tool to add this noise please? So that Facebook et al. can't find me and build a profile on me based on random images that I didn't even know existed?
  - ipsa 8y ago
    https://cvdazzle.com/ https://cvdazzle.com/
    - jakecopp 8y ago
      That is a really cool research/art project!
    - Animats 8y ago
      Nice. I just sent that to a friend who has a hair salon in SF.
    - apatters 8y ago
      This is the most cyberpunk thing I've ever seen
    - Izkata 8y ago
      This style made an appearance on Elementary a few years ago: https://www.youtube.com/watch?v=A1_9aHo0S30 https://www.youtube.com/watch?v=A1_9aHo0S30
- ericd 8y ago
  Sounds like those networks need an adversarial network or two to improve their performance and make them less susceptible to attack.
adventured 8y ago
> I'd also argue that we're actually more likely to experience another AI winter. We'll experience an AI winter again like we experienced an Internet winter in 2001-2004. Which is to say, not really at all. AI is now being widely commercialized for the benefit of consumers and businesses. That process will not stop, even if the hype train deflates before rising again at a later date. There is large, tangible commercial value in AI at the current general level of capability and near-term potential. That will result in pursuing maxing out whatever this era is capable of, before the next leap occurs at some point down the road. It's a progress track of higher highs during the exploratory boom and higher lows during the winter.
darawk 8y ago
> This latest craze of "AI" research seems to be fueled by a sudden glut of computational power (GPUs) that wasn't available previously. I think that most technical people would agree that the mid 2020s is extremely ambitious. I'd also argue that we're actually more likely to experience another AI winter. I think that is extremely unlikely. "AI" (read: machine learning) is actually being used for business purposes now, it's delivering enormous value to nearly every business on the planet. We're now in a long phase of descending the gradient of the current batch of broad techniques. This is likely a decently long gradient, with lots of marginal improvements to be made for a long time. And whereas with research projects, people don't care much about marginal improvements, they really do for business use-cases. For those reasons, I think AI/ML is basically here to stay just as much as basic biological research, or physics, or whatever is, if not more.
- RodericDay 8y ago
  > it's delivering enormous value to nearly every business on the planet lol
- logifail 8y ago
  >> "AI" (read: machine learning) [is] delivering enormous value to nearly every business on the planet That statement appears to contain two fairly bold claims - could you share sources?
  - adrianN 8y ago
    Every big website uses some kind of machine learning to prevent fraud. Banks do the same. Data mining is used everywhere to improve customer experience. Data mining is used in industrial applications for preventative maintenance.
  - overlords 8y ago
    Not OP, but of course even 0.1% improvements are worth millions to search engines, social network feeds, financial forecasters, and self driving car companies. Also worth thousands or millions to manufacturing processes, and to small businesses, which might be using ecommerce optimization systems through providers.
dontreact 8y ago
I don't think adversarial examples give any evidence of relevant problems with these models because they occur on a very specific subset of images that can only be discovered using detailed knowledge of how these networks process images. For all we know, humans have similar problems on some obscure subset of images, but we can't find human's adversarial examples because we don't have detailed knowledge of how the brain processes images.
- adrianN 8y ago
  I think adversarial examples for humans are called "optical illusions".
  - ccvannorman 8y ago
    There is a categorical difference between "a [specifically designed] image that can be construed as a duck or a rabbit" and "a human can regularly mis-categorize random pictures of ducks as rabbits if a weird filter is overlayed". The first is well-known and fun and trite -- the second is unheard of and probably impossible for humans, yet provably possible for trained computers.
    - fons 8y ago
      The point outlined is that we don't know enough about how we identify objects to discard a simple adversarial attack; probably not a filter-based but maybe something else.
    - cameldrv 8y ago
      It's called camoflauge. The natural world is full of adversarial examples.
    - dontreact 8y ago
      "probably impossible for humans" Based on what?
    - fenomas 8y ago
      I'd imagine GP was referring to "humans perceive straight lines to be curved when certain shapes are overlayed", or "humans perceive shapes of the same color to be different colors when filters are applied" sorts of optical illusions. There are plenty of those, and I personally I think they're probably analogous to how adversarial filters fool AI classifiers.
mannykannot 8y ago
I don't think it is just a matter of computational power: I have been quite surprised by how effective word embedding, for example, has been in abetting language translation (note that I am not claiming that it has solved the problem; I am too familiar with the problems of idiomatic Vietnamese-English translation.) Of course, if you had different intuitions (or more knowledge) than me, you might not be so impressed. Having said that, I agree that the projections seem highly optimistic, but maybe I will be surprised again.
est 8y ago
> how susceptible they are to adversarial attacks. Adding small amounts of noise causes misclassification in images, and some papers even explore the inevitability of adversarial examples What if, in a distant future, computers turn out to be the correct one, humans's perception are biased?
- red75prime 8y ago
  In what sense an image of a rabbit perturbed by adversarial noise can be _correctly_ recognized as a duck? There can be a general consensus that the image looks like a duck at most. And if humans see a rabbit and AIs see a duck there just won't be consensus.
Dirlewanger 8y ago
>I'm convinced the self-driving cars are still a ways off as well. Technology-wise, absolutely they are. The problem is that in actuality, they aren't. Companies will continue to push as hard as they can for as wide of a launch as they can, while governments (and any kind of sorely-needed oversight) will be ages behind.

njoubert 8y ago

This is a common misconception. It is not a small amount of noise that causes misclassification of images. It is a carefully designed and quite unique pattern that causes misclassification. It only looks like noise to the human eye, but it really isn't. Yes, neural networks are susceptible to adversarial attacks. No, just adding noise to an image doesn't break neural networks.

pmoriarty 8y ago
Adding small amounts of noise is actually sometimes used to improve the performance of various AI techniques. It helps prevent overfitting. In fact, if your technique or model is seriously affected by a little noise this is usually enough to brand it brittle and maybe even a failure, as it's a sign of overfitting. Anyone working in this field knows to look for this and will try to make what they create more robust. The design of visual captchas is one obvious indication of just how successful AI techniques have been at image recognition in the presence of noise. It's no longer enough to make them a little noisy. In order to resist being solved by mechanical means, visual captchas have to include so much noise that even humans have problems recognizing them.
ipsa 8y ago
Read it as: A small amount of carefully constructed noise. Then you are correct to literature and pop-science. No misconception needed. There are 1-pixel attacks now. Randomly shuffling a small amount of pixels around can cause predictions to shift. The issue is that there is no scene understanding. No common sense. No 3D modeling. Just 10x10 pattern matching on a very large fuzzy database of natural images (which works really really well in most cases). The hype of ML is driven by 3 things: Big companies vying for AI dominance, militaries that want to finally use neural nets that work, and international competition between the West and the East to be the first to largely automate their economies (or AGI if you want to call it that). Catalysts were big data hoarding, GPU training on ImageNet, and then AlphaGo.
- SubiculumCode 8y ago
  I am not a machine learning expert, but could not these adversarial example issues be resolved by solving an image classification problem by (1)producing multiple non-equivalent classification solutions with adequate accuracy, then (2)fusion (e.g. voting) to produce a consensus classification? (3) Maybe random shuffling of which X of Z solutions get to vote in each classification attempt. What might fool one solution might not fool another, and adversarial examples seem to depend on idiosyncrasies of a particular solution.
  - hahajk 8y ago
    Or just adding a small amount of random noise to the input, which would wipe out the carefully constructed attack.
    - ipsa 8y ago
      You can try out this technique at https://github.com/google/unrestricted-adversarial-examples https://github.com/google/unrestricted-adversarial-examples My guess is it would have the same result as adding noise to the normal images too (resulting in a slightly worse performance overall).
  - freeone3000 8y ago
apatters 8y ago
Can I have a tool to add this noise please? So that Facebook et al. can't find me and build a profile on me based on random images that I didn't even know existed?
- ipsa 8y ago
  https://cvdazzle.com/ https://cvdazzle.com/
  - jakecopp 8y ago
    That is a really cool research/art project!
  - Animats 8y ago
    Nice. I just sent that to a friend who has a hair salon in SF.
  - apatters 8y ago
    This is the most cyberpunk thing I've ever seen
  - Izkata 8y ago
    This style made an appearance on Elementary a few years ago: https://www.youtube.com/watch?v=A1_9aHo0S30 https://www.youtube.com/watch?v=A1_9aHo0S30
ericd 8y ago
Sounds like those networks need an adversarial network or two to improve their performance and make them less susceptible to attack.

adventured 8y ago

> I'd also argue that we're actually more likely to experience another AI winter. We'll experience an AI winter again like we experienced an Internet winter in 2001-2004. Which is to say, not really at all. AI is now being widely commercialized for the benefit of consumers and businesses. That process will not stop, even if the hype train deflates before rising again at a later date. There is large, tangible commercial value in AI at the current general level of capability and near-term potential. That will result in pursuing maxing out whatever this era is capable of, before the next leap occurs at some point down the road. It's a progress track of higher highs during the exploratory boom and higher lows during the winter.

darawk 8y ago

> This latest craze of "AI" research seems to be fueled by a sudden glut of computational power (GPUs) that wasn't available previously. I think that most technical people would agree that the mid 2020s is extremely ambitious. I'd also argue that we're actually more likely to experience another AI winter. I think that is extremely unlikely. "AI" (read: machine learning) is actually being used for business purposes now, it's delivering enormous value to nearly every business on the planet. We're now in a long phase of descending the gradient of the current batch of broad techniques. This is likely a decently long gradient, with lots of marginal improvements to be made for a long time. And whereas with research projects, people don't care much about marginal improvements, they really do for business use-cases. For those reasons, I think AI/ML is basically here to stay just as much as basic biological research, or physics, or whatever is, if not more.

RodericDay 8y ago
> it's delivering enormous value to nearly every business on the planet lol
logifail 8y ago
>> "AI" (read: machine learning) [is] delivering enormous value to nearly every business on the planet That statement appears to contain two fairly bold claims - could you share sources?
- adrianN 8y ago
  Every big website uses some kind of machine learning to prevent fraud. Banks do the same. Data mining is used everywhere to improve customer experience. Data mining is used in industrial applications for preventative maintenance.
- overlords 8y ago
  Not OP, but of course even 0.1% improvements are worth millions to search engines, social network feeds, financial forecasters, and self driving car companies. Also worth thousands or millions to manufacturing processes, and to small businesses, which might be using ecommerce optimization systems through providers.

dontreact 8y ago

I don't think adversarial examples give any evidence of relevant problems with these models because they occur on a very specific subset of images that can only be discovered using detailed knowledge of how these networks process images. For all we know, humans have similar problems on some obscure subset of images, but we can't find human's adversarial examples because we don't have detailed knowledge of how the brain processes images.

adrianN 8y ago
I think adversarial examples for humans are called "optical illusions".
- ccvannorman 8y ago
  There is a categorical difference between "a [specifically designed] image that can be construed as a duck or a rabbit" and "a human can regularly mis-categorize random pictures of ducks as rabbits if a weird filter is overlayed". The first is well-known and fun and trite -- the second is unheard of and probably impossible for humans, yet provably possible for trained computers.
  - fons 8y ago
    The point outlined is that we don't know enough about how we identify objects to discard a simple adversarial attack; probably not a filter-based but maybe something else.
  - cameldrv 8y ago
    It's called camoflauge. The natural world is full of adversarial examples.
  - dontreact 8y ago
    "probably impossible for humans" Based on what?
  - fenomas 8y ago
    I'd imagine GP was referring to "humans perceive straight lines to be curved when certain shapes are overlayed", or "humans perceive shapes of the same color to be different colors when filters are applied" sorts of optical illusions. There are plenty of those, and I personally I think they're probably analogous to how adversarial filters fool AI classifiers.

mannykannot 8y ago

I don't think it is just a matter of computational power: I have been quite surprised by how effective word embedding, for example, has been in abetting language translation (note that I am not claiming that it has solved the problem; I am too familiar with the problems of idiomatic Vietnamese-English translation.) Of course, if you had different intuitions (or more knowledge) than me, you might not be so impressed. Having said that, I agree that the projections seem highly optimistic, but maybe I will be surprised again.

est 8y ago

> how susceptible they are to adversarial attacks. Adding small amounts of noise causes misclassification in images, and some papers even explore the inevitability of adversarial examples What if, in a distant future, computers turn out to be the correct one, humans's perception are biased?

red75prime 8y ago
In what sense an image of a rabbit perturbed by adversarial noise can be _correctly_ recognized as a duck? There can be a general consensus that the image looks like a duck at most. And if humans see a rabbit and AIs see a duck there just won't be consensus.

Dirlewanger 8y ago

>I'm convinced the self-driving cars are still a ways off as well. Technology-wise, absolutely they are. The problem is that in actuality, they aren't. Companies will continue to push as hard as they can for as wide of a launch as they can, while governments (and any kind of sorely-needed oversight) will be ages behind.