5 ms·

> What this underscores, if anything, is the accelerating pace of technological growth, for better or for worse. While growth may be accelerating, this is simp

> What this underscores, if anything, is the accelerating pace of technological growth, for better or for worse. While growth may be accelerating, this is simply the result of one big paradigm shift in deep learning/NNs. Once we've learned to milk it for all its worth, we'll have to wait for the next epiphany.

hmate9 11y ago
Milking neural networks out completely is pretty much AI as depicted in the movies. If we can milk it completely there probably isn't a need for the next epiphany.
- swombat 11y ago
  Or, at the very least, the next epiphany need not be human-designed. Just train a neural network in the art of creating AI paradigms and implementations that can do general purpose AI. Once that's "milked", the era of human technological evolution is finished.
  - foota 11y ago
    That sounds like a bad idea.
    - swombat 11y ago
      It's the core idea of AI, the primary reason why it is suspected that developing strong AI will inevitably lead to the end of the human era of evolution.
    - deleted 11y ago
      [deleted]
  - YeGoblynQueenne 11y ago
    I don't want to be mean, but that's like saying you'll train a magic neural net with the mystical flavour of unicorn tears and then the era of making rainbows out of them will be finished. Or something. I mean, come on- "the art of creating AI paradigms"? What is that even? You're going to find data on this, where, and train on it, how, exactly? Sorry to take this out on you but the level of hand-waving and magical thinking is reaching critical mass lately, and it's starting to obscure the significance of the AlphaGo achievement. Edit: not to mention, the crazy hype surrounding ANNs in the popular press (not least because it's the subject of SF stories, like someone notes above) risks killing nascent ideas and technologies that may well have the potential to be the next big breakthrough. If we end up to the point where everyone thinks all our AI problems are solved, if we just throw a few more neural layers to them, then we're in trouble. Hint: because they're not.
    - swombat 11y ago
      I totally see your point and my purpose is definitely not to be alarmist and sound the alarm that skynet is about to come out of AlphaGo or some other equivalent neural net. But I think the opposite attitude is also false. As others have pointed out, we don't really know how the brain works. Neural nets represent one of our best attempts to model brains. Whether or not it's good enough to create real intelligence is completely unknown. Maybe it is, maybe it's not. Intelligence appears to be an emergent property and we don't know the circumstances under which it emerges. It could come out of a neural network. Or maybe it could not. The only way we'll find out is by trying to make it happen. Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must. This is Hacker News, not a mass newspaper, so I think we can take the more nuanced and complex view here.
      - TheOtherHobbes 11y ago
        We don't really know how AI works either. NNs (for example) do stuff, and sometimes it's hard to see why. >Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must. Not really. Right now it's taking the position that there is no practical path that anyone can imagine from a go-bot, which is working in a very restricted problem space, to a magical self-improving AI-squared god-bot, which would be working in a problem space with a completely unknown shape, boundaries, and inner properties. Meta-AI isn't even a thing yet. There are some obvious things that could be tried - like trying to evolve a god-bot out of a gigantic pre-Cambrian soup of micro-bots where each bot is a variation on one of the many possible AI implementations - but at the moment basic AI is too resource intensive to make those kinds of experiments a possibility. And there's no guarantee anything we can think of today will work.
      - YeGoblynQueenne 11y ago
        >> Neural nets represent one of our best attempts to model brains. See now that's one of the misconceptions. ANNs are not modelled on the brain, not anymore and not ever since the poor single-layer Perceptron which itself was modelled after an early model of neuronal activation. What ANNs really are is algorithms for optimising systems of functions. And that includes things like Support Vector Machines and Radial Basis Function networks that don't even fit in the usual multi-layer network diagram particularly well. It's unfortunate that this sort of language and imagery is still used abundantly, by people who should know better no less, but I guess "it's an artificial brain" sounds more magical than "it's function optimisation". You shouldn't let it mislead you though. >> Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must. I don't agree. It's a subject that's informed by a solid understanding of the fundamental concepts - function optimisation, again. There's uncertainty because there's theoretical limits that are hard to test, frex the fact that multi-layer perceptrons with three neural layers can learn any function given a sufficient number of inputs, or on the opposite side, that non-finite languages are _not_ learnable in the limit (not ANN-specific but limiting what any algorithm can learn) etc. But the arguments on either side are, well, arguments. Nobody is being "blind". People defend their ideas, is all.
- _yosefk 11y ago
  You're basically saying that there's no task (including passing the Turing test, programming web apps, etc.) which requires intelligence and is best tackled with either something else than a neural network or with NN combined with something else. I think it's a pretty bold statement which is really hard to back up by anything but a hunch.
  - deleted 11y ago
    [deleted]
  - swombat 11y ago
    Our current assertion is that neural networks basically replicate the brain's function, so our current understanding of this paradigm is that "milking neural networks" is going to match or exceed human general purpose intelligence. I believe hmate9 is correct. If this paradigm is exploited to the full, unless we've missed something fundamental about how the brain works, we don't need to bother ourselves with inventing the next paradigm (of which there will no doubt be many), because one of the results of the current paradigm will be either an AGI (Artificial General Intelligence) that runs faster and better than human intelligence, or, more likely, an ASI (Artificial Super Intelligence). Either of those is more capable than we are for the purpose of inventing the next paradigm.
    - argonaut 11y ago
      No deep learning researcher believes neural networks "basically replicate" the brain's function. Neural nets do a ton of things brains don't do (nobody believes the brain is doing stochastic gradient descent on a million data points in mini-batches). Brains also do a billion things that neural nets don't do. I've never even taken a neuroscience class, and I can think of the following: synaptic gaps, neurotransmitters, the concept of time, theta oscillations, all or nothing action potentials, Schwann cells. You have missed something fundamental about how the brain works. Namely, neuroscientists don't really know how it works. Neuroscientists do not fully understand how neurons in our brain learn. According to Andrew Ng (https://www.quora.com/What-does-Andrew-Ng-think-about-Deep-Learning https://www.quora.com/What-does-Andrew-Ng-think-about-Deep-L...): "Because we fundamentally don't know how the brain works, attempts to blindly replicate what little we know in a computer also has not resulted in particularly useful AI systems. Instead, the most effective deep learning work today has made its progress by drawing from CS and engineering principles and at most a touch of biological inspiration, rather than try to blindly copy biology. Concretely, if you hear someone say "The brain does X. My system also does X. Thus we're on a path to building the brain," my advice is to run away!"
HaikuEU 11y ago
Could you care to explain what was that big paradigm shift ?
- bla2 11y ago
  This four-part series explains it well: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/ http://www.andreykurenkov.com/writing/a-brief-history-of-neu...
JulianMorrison 11y ago
But that's what technological growth is. A series of epiphanies, building on what came before.
- msbarnett 11y ago
  Yes, but an epiphany is not evidence of an accelerating rate of epiphanies, nor evidence that such epiphanies will continue apace into the future.
  - JulianMorrison 11y ago
    You can look at the past for that, although obviously it doesn't predict the future. But it ought to be a priori obvious, at least, that the more you know (as a species), the more surface area of knowledge you have to synthesize into an extending step beyond the known.
    - ska 11y ago
      You could look at the past, but that isn't what the claim did. In fact looking at the rate of change in applications over an "epiphany" period is probably the least useful estimate of progress & rate of change in progress.

hmate9 11y ago

Milking neural networks out completely is pretty much AI as depicted in the movies. If we can milk it completely there probably isn't a need for the next epiphany.

swombat 11y ago
Or, at the very least, the next epiphany need not be human-designed. Just train a neural network in the art of creating AI paradigms and implementations that can do general purpose AI. Once that's "milked", the era of human technological evolution is finished.
- foota 11y ago
  That sounds like a bad idea.
  - swombat 11y ago
    It's the core idea of AI, the primary reason why it is suspected that developing strong AI will inevitably lead to the end of the human era of evolution.
  - deleted 11y ago
    [deleted]
- YeGoblynQueenne 11y ago
  I don't want to be mean, but that's like saying you'll train a magic neural net with the mystical flavour of unicorn tears and then the era of making rainbows out of them will be finished. Or something. I mean, come on- "the art of creating AI paradigms"? What is that even? You're going to find data on this, where, and train on it, how, exactly? Sorry to take this out on you but the level of hand-waving and magical thinking is reaching critical mass lately, and it's starting to obscure the significance of the AlphaGo achievement. Edit: not to mention, the crazy hype surrounding ANNs in the popular press (not least because it's the subject of SF stories, like someone notes above) risks killing nascent ideas and technologies that may well have the potential to be the next big breakthrough. If we end up to the point where everyone thinks all our AI problems are solved, if we just throw a few more neural layers to them, then we're in trouble. Hint: because they're not.
  - swombat 11y ago
    I totally see your point and my purpose is definitely not to be alarmist and sound the alarm that skynet is about to come out of AlphaGo or some other equivalent neural net. But I think the opposite attitude is also false. As others have pointed out, we don't really know how the brain works. Neural nets represent one of our best attempts to model brains. Whether or not it's good enough to create real intelligence is completely unknown. Maybe it is, maybe it's not. Intelligence appears to be an emergent property and we don't know the circumstances under which it emerges. It could come out of a neural network. Or maybe it could not. The only way we'll find out is by trying to make it happen. Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must. This is Hacker News, not a mass newspaper, so I think we can take the more nuanced and complex view here.
    - TheOtherHobbes 11y ago
      We don't really know how AI works either. NNs (for example) do stuff, and sometimes it's hard to see why. >Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must. Not really. Right now it's taking the position that there is no practical path that anyone can imagine from a go-bot, which is working in a very restricted problem space, to a magical self-improving AI-squared god-bot, which would be working in a problem space with a completely unknown shape, boundaries, and inner properties. Meta-AI isn't even a thing yet. There are some obvious things that could be tried - like trying to evolve a god-bot out of a gigantic pre-Cambrian soup of micro-bots where each bot is a variation on one of the many possible AI implementations - but at the moment basic AI is too resource intensive to make those kinds of experiments a possibility. And there's no guarantee anything we can think of today will work.
    - YeGoblynQueenne 11y ago
      >> Neural nets represent one of our best attempts to model brains. See now that's one of the misconceptions. ANNs are not modelled on the brain, not anymore and not ever since the poor single-layer Perceptron which itself was modelled after an early model of neuronal activation. What ANNs really are is algorithms for optimising systems of functions. And that includes things like Support Vector Machines and Radial Basis Function networks that don't even fit in the usual multi-layer network diagram particularly well. It's unfortunate that this sort of language and imagery is still used abundantly, by people who should know better no less, but I guess "it's an artificial brain" sounds more magical than "it's function optimisation". You shouldn't let it mislead you though. >> Taking a position that neural networks cannot ever result in strong AI is as blind as taking a position that they must. I don't agree. It's a subject that's informed by a solid understanding of the fundamental concepts - function optimisation, again. There's uncertainty because there's theoretical limits that are hard to test, frex the fact that multi-layer perceptrons with three neural layers can learn any function given a sufficient number of inputs, or on the opposite side, that non-finite languages are _not_ learnable in the limit (not ANN-specific but limiting what any algorithm can learn) etc. But the arguments on either side are, well, arguments. Nobody is being "blind". People defend their ideas, is all.
_yosefk 11y ago
You're basically saying that there's no task (including passing the Turing test, programming web apps, etc.) which requires intelligence and is best tackled with either something else than a neural network or with NN combined with something else. I think it's a pretty bold statement which is really hard to back up by anything but a hunch.
- deleted 11y ago
  [deleted]
- swombat 11y ago
  Our current assertion is that neural networks basically replicate the brain's function, so our current understanding of this paradigm is that "milking neural networks" is going to match or exceed human general purpose intelligence. I believe hmate9 is correct. If this paradigm is exploited to the full, unless we've missed something fundamental about how the brain works, we don't need to bother ourselves with inventing the next paradigm (of which there will no doubt be many), because one of the results of the current paradigm will be either an AGI (Artificial General Intelligence) that runs faster and better than human intelligence, or, more likely, an ASI (Artificial Super Intelligence). Either of those is more capable than we are for the purpose of inventing the next paradigm.
  - argonaut 11y ago
    No deep learning researcher believes neural networks "basically replicate" the brain's function. Neural nets do a ton of things brains don't do (nobody believes the brain is doing stochastic gradient descent on a million data points in mini-batches). Brains also do a billion things that neural nets don't do. I've never even taken a neuroscience class, and I can think of the following: synaptic gaps, neurotransmitters, the concept of time, theta oscillations, all or nothing action potentials, Schwann cells. You have missed something fundamental about how the brain works. Namely, neuroscientists don't really know how it works. Neuroscientists do not fully understand how neurons in our brain learn. According to Andrew Ng (https://www.quora.com/What-does-Andrew-Ng-think-about-Deep-Learning https://www.quora.com/What-does-Andrew-Ng-think-about-Deep-L...): "Because we fundamentally don't know how the brain works, attempts to blindly replicate what little we know in a computer also has not resulted in particularly useful AI systems. Instead, the most effective deep learning work today has made its progress by drawing from CS and engineering principles and at most a touch of biological inspiration, rather than try to blindly copy biology. Concretely, if you hear someone say "The brain does X. My system also does X. Thus we're on a path to building the brain," my advice is to run away!"

HaikuEU 11y ago

Could you care to explain what was that big paradigm shift ?

bla2 11y ago
This four-part series explains it well: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/ http://www.andreykurenkov.com/writing/a-brief-history-of-neu...

JulianMorrison 11y ago

But that's what technological growth is. A series of epiphanies, building on what came before.

msbarnett 11y ago
Yes, but an epiphany is not evidence of an accelerating rate of epiphanies, nor evidence that such epiphanies will continue apace into the future.
- JulianMorrison 11y ago
  You can look at the past for that, although obviously it doesn't predict the future. But it ought to be a priori obvious, at least, that the more you know (as a species), the more surface area of knowledge you have to synthesize into an extending step beyond the known.
  - ska 11y ago
    You could look at the past, but that isn't what the claim did. In fact looking at the rate of change in applications over an "epiphany" period is probably the least useful estimate of progress & rate of change in progress.