4 ms·

There has been a massive explosion in deep learning in the past few years, mainly enabled by the rapid drop in the price of computing (GPUs.) They are breaking

There has been a massive explosion in deep learning in the past few years, mainly enabled by the rapid drop in the price of computing (GPUs.) They are breaking bencmarks at a bunch of different AI tasks from speech recognition to translation to machine vision. I wrote a summary here of the main achievements of 2014: https://www.reddit.com/r/Futurology/comments/2qq993/developments_in_ai_in_2014/ https://www.reddit.com/r/Futurology/comments/2qq993/developm... Stuart Russel recently said "The commercial investment in AI the last five years has exceeded the entire world wide government investment in AI research since it's beginnings in the 1950's."

jacquesm 12y ago
None of these come close to AGI and combining them in any number of constellations and increasing complexity still misses a qualitative step. All those machine learning tricks have been available for the longest time, the two orders of magnitude speed-up that we have received (courtesy of the gaming industry) should be seen as such a qualitative step and yet we are no closer to a general AI than we were before that speed up took place. If anything we've learned how incredibly hard the problem really is and the predictions on how close we are from a decade ago have already slipped significantly.
- hyperbovine 12y ago
  Agreed. Throwing more and more GPUs at the problem in order to eek a couple more percent out on ImageNet is never, ever going to lead to the sort of AI that most people associate with that term. My takeaway from the past 5 years of deep learning is: ANNs can store a ton of patterns. Which makes them incredible useful for certain things, but AI it AIn't.
- hyperpallium 12y ago
  Not in itself, but as a tool, in the manner of better telescopes and microscopes. And people don't invest a lot of thought on approaches that are utterly infeasible, until tech makes them feasible - plus, there's much learning in trial-and-error, experiments, tinkering, which you just can't do if it's too slow. Unfortunately, much of current statistical machine learning does not help its developers to see, unlike a *-scope. It's just a blackbox. See Chomsky vs Norvig http://norvig.com/chomsky.html http://norvig.com/chomsky.html
- Houshalter 12y ago
  This is just the AI effect. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect No matter what accomplishments are made, there will always be someone shouting that it's not really AI or it isn't really progress. It's impossible to argue against. Until we finally pass the very last goal post, and then it's too late.
  - jacquesm 12y ago
    No, it isn't the same thing. Those things that we now group under machine learning and applied statistics are not general artificial intelligence, they are narrow applications of statistical methods but they are not generally applicable intelligence. It's still 'programmer directed' and that's the key difference. As soon as the programmer is no longer required to string the pieces together you have something that has the potential to cross over the barrier. Generalized pattern matching is a tool in the toolbox of an AI but it is not AI by itself and there may be work-arounds to AI which do not require generalized pattern matching (that's an interesting one, requiring a bit of a trick in that you if you could generate a specialized pattern matcher on demand that you don't need a generalized one).
    - Houshalter 12y ago
      For all we know actual neurons are just implementing "simple narrow statistical algorithms", just really big and highly tuned by evolution. Based on what I know about neuroscience I strongly bet that's the case. Define "programmer directed". There are neural networks that can do reinforcement learning and play video games which are very general.
- bane 12y ago
  I'm also kind of concerned at how we're willing to give up understanding the specifics of cognition by just throwing data into differently shaped statistical bags and using whatever model seems to kinda work the best.
  - jacquesm 12y ago
    I think this is in part an admission that actually understanding the specifics of cognition is a lot harder than was originally thought and the wholesale methods yield results.
    - bane 12y ago
      Yeah, the newer methods are getting quick and useful results. But I've heard some good arguments that it's unclear if these techniques are useful towards a general purpose AI or if they're just a distraction and we'll have to go back again and try to actually figure it out for real.
shock-value 12y ago
Deep learning is great and useful and a big step forward, but it's still basically just brute-forced pattern matching. Wonderful for some kinds of data analysis but, without additional architectural breakthroughs, useless for developing the kinds of AI everyone dreams about -- systems that can take creative action in a world (whether the real world or some virtually constructed one) and evaluate the effects and utility (for some interesting utility function) of said actions.
- frutiger 12y ago
  > but it's still basically just brute-forced pattern matching Do you have any reason to believe the human brain is any different? We just have more neurons.
  - shock-value 12y ago
    We have more neurons and, in particular, they are arranged in architecturally interesting ways, rather than just feedforward networks as in most current strategies for deep learning. There is of course work being done on recurrent networks, but it's basically in its infancy. And what is being done so far is still nowhere near as complex as the human brain, which clearly has many distinct areas with distinct but of course interrelated functions.
    - maroonblazer 12y ago
      I'm not an AI researcher so this is probably a naive question: Do we necessariliy need to mimic the architecture of the human brain to achieve AGI? As someone (Chalmers?) once said about the problem of consciousness: we didn't need to replicate the flapping of wings or the locomotion of sea creatures before taking to the air or underwater. Might it not also be the case with AI that there's some fundamental principle we've yet to discover that just so happens to have expression in the substrate of 1200cc's of fatty tissue, but could possess the same fidelity (and greater) in silicon and looks nothing like the architecture of the human brain?
      - 12y ago

jacquesm 12y ago

None of these come close to AGI and combining them in any number of constellations and increasing complexity still misses a qualitative step. All those machine learning tricks have been available for the longest time, the two orders of magnitude speed-up that we have received (courtesy of the gaming industry) should be seen as such a qualitative step and yet we are no closer to a general AI than we were before that speed up took place. If anything we've learned how incredibly hard the problem really is and the predictions on how close we are from a decade ago have already slipped significantly.

hyperbovine 12y ago
Agreed. Throwing more and more GPUs at the problem in order to eek a couple more percent out on ImageNet is never, ever going to lead to the sort of AI that most people associate with that term. My takeaway from the past 5 years of deep learning is: ANNs can store a ton of patterns. Which makes them incredible useful for certain things, but AI it AIn't.
hyperpallium 12y ago
Not in itself, but as a tool, in the manner of better telescopes and microscopes. And people don't invest a lot of thought on approaches that are utterly infeasible, until tech makes them feasible - plus, there's much learning in trial-and-error, experiments, tinkering, which you just can't do if it's too slow. Unfortunately, much of current statistical machine learning does not help its developers to see, unlike a *-scope. It's just a blackbox. See Chomsky vs Norvig http://norvig.com/chomsky.html http://norvig.com/chomsky.html
Houshalter 12y ago
This is just the AI effect. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect No matter what accomplishments are made, there will always be someone shouting that it's not really AI or it isn't really progress. It's impossible to argue against. Until we finally pass the very last goal post, and then it's too late.
- jacquesm 12y ago
  No, it isn't the same thing. Those things that we now group under machine learning and applied statistics are not general artificial intelligence, they are narrow applications of statistical methods but they are not generally applicable intelligence. It's still 'programmer directed' and that's the key difference. As soon as the programmer is no longer required to string the pieces together you have something that has the potential to cross over the barrier. Generalized pattern matching is a tool in the toolbox of an AI but it is not AI by itself and there may be work-arounds to AI which do not require generalized pattern matching (that's an interesting one, requiring a bit of a trick in that you if you could generate a specialized pattern matcher on demand that you don't need a generalized one).
  - Houshalter 12y ago
    For all we know actual neurons are just implementing "simple narrow statistical algorithms", just really big and highly tuned by evolution. Based on what I know about neuroscience I strongly bet that's the case. Define "programmer directed". There are neural networks that can do reinforcement learning and play video games which are very general.
bane 12y ago
I'm also kind of concerned at how we're willing to give up understanding the specifics of cognition by just throwing data into differently shaped statistical bags and using whatever model seems to kinda work the best.
- jacquesm 12y ago
  I think this is in part an admission that actually understanding the specifics of cognition is a lot harder than was originally thought and the wholesale methods yield results.
  - bane 12y ago
    Yeah, the newer methods are getting quick and useful results. But I've heard some good arguments that it's unclear if these techniques are useful towards a general purpose AI or if they're just a distraction and we'll have to go back again and try to actually figure it out for real.

shock-value 12y ago

Deep learning is great and useful and a big step forward, but it's still basically just brute-forced pattern matching. Wonderful for some kinds of data analysis but, without additional architectural breakthroughs, useless for developing the kinds of AI everyone dreams about -- systems that can take creative action in a world (whether the real world or some virtually constructed one) and evaluate the effects and utility (for some interesting utility function) of said actions.

frutiger 12y ago
> but it's still basically just brute-forced pattern matching Do you have any reason to believe the human brain is any different? We just have more neurons.
- shock-value 12y ago
  We have more neurons and, in particular, they are arranged in architecturally interesting ways, rather than just feedforward networks as in most current strategies for deep learning. There is of course work being done on recurrent networks, but it's basically in its infancy. And what is being done so far is still nowhere near as complex as the human brain, which clearly has many distinct areas with distinct but of course interrelated functions.
  - maroonblazer 12y ago
    I'm not an AI researcher so this is probably a naive question: Do we necessariliy need to mimic the architecture of the human brain to achieve AGI? As someone (Chalmers?) once said about the problem of consciousness: we didn't need to replicate the flapping of wings or the locomotion of sea creatures before taking to the air or underwater. Might it not also be the case with AI that there's some fundamental principle we've yet to discover that just so happens to have expression in the substrate of 1200cc's of fatty tissue, but could possess the same fidelity (and greater) in silicon and looks nothing like the architecture of the human brain?
    - 12y ago