4 ms·

> deep learning must be supplemented by other techniques if we are to reach artificial general intelligence I don't think anyone major ever disputed that. Hav

> deep learning must be supplemented by other techniques if we are to reach artificial general intelligence I don't think anyone major ever disputed that. Having said that, thousand times yes to the author's concerns. Deep learning is AI's cryptocurrency in terms of being overhyped, although its main proponents are not to blame for that.

bitL 9y ago
Why do you think it's overhyped? It pushed state-of-art results in quite a few difficult domains by quite high margin; it's deservedly praised. Or you don't like that you can't really understand what is going on inside, despite it using simple math and primitive non-linear optimization, making it "conceptually" inferior and not as "tasty" to other ML methods where we can actually prove something?
- vadimberman 9y ago
  Because today, most lay people equate deep learning with AI. The black box part is also an issue which both the author of the paper and literally everyone else, including Google's Peter Norvig, is concerned about. But that's not related to the hype part.
- yorwba 9y ago
  Deep learning is especially overhyped by articles that describe it as "just like the human brain". It's a useful technique that lets you skip lots of feature engineering by just letting the classifier learn its own features, but it is also really easy to project abilities onto neural networks that they don't really have. Deep learning is not magic, for every network architecture that beats the state of the art, there are a hundred very similar ones that completely fail, or run too slow, or don't fit into GPU memory ... and the only way we know to get improvements is to fiddle with the hyperparameters until everything works out.
Ar-Curunir 9y ago
Deep learning has had much greater impact within AI than cryptocurrencies have had in cryptography/systems/security
- vadimberman 9y ago
  I should have added, "what cryptocurrencies are to fintech". Is it better :) ?
myaso 9y ago
To get AI you have an interface problem you need to solve since reality is a bitch to parse; this is just the start, what about the formulation of goals? How would you even make that in code? You would need to start with a certain kind of data that was extracted by the sensors but how will you ever know you have the correct set? You have two problems to solve simultaneously getting the right data and building the right system on top of it -- this core data when done in reality is not the same core data that would make this possible in a simulation and the system on top while maybe having the same fundamental form is also very likely different. Makings intelligent agents in a simulation is not the same as making intelligent agents arbitrarily useful in the real world, granted a simulation is a good means to explore techniques -- in a simulation I control the sensory apparatus of the agent perfectly. So I think the lesson is you either build systems from which it is easy to stream data and configure programmatically or deal with the fact that this is likely impossible without simulating a universe to high fidelity. Deep Learning is extremely useful as it is, not the same as cryptocurrencies which are useful but this usefulness will never be properly realized due to human nature -- right now their primary purpose seems to be to satiate greed and humans with poor impulse control while promising a lot of outlandish things.
- vadimberman 9y ago
  You hit the nail on the head. Oren Etzioni has been shouting about the knowledge representation for years (full disclosure: this is something I'm focused on, too).
goatlover 9y ago
Why do we want to reach AGI? There are around 7.5 billion biological general intelligences, and we already know how to make more of them. Is it to save companies labor costs and increase their profit margins? I'm not sure that's ultimately in the interest of most people, or even the long term health of the economy. Seriously, why do we want to replace human intelligence? I get augmenting it with narrow forms of automation, but AGI is a different animal.
riku_iki 9y ago
> I don't think anyone major ever disputed that. Is it that obvious though? Any good evidence that human brain on the planet scale cluster can't be modelled with DL?

bitL 9y ago

Why do you think it's overhyped? It pushed state-of-art results in quite a few difficult domains by quite high margin; it's deservedly praised. Or you don't like that you can't really understand what is going on inside, despite it using simple math and primitive non-linear optimization, making it "conceptually" inferior and not as "tasty" to other ML methods where we can actually prove something?

vadimberman 9y ago
Because today, most lay people equate deep learning with AI. The black box part is also an issue which both the author of the paper and literally everyone else, including Google's Peter Norvig, is concerned about. But that's not related to the hype part.
yorwba 9y ago
Deep learning is especially overhyped by articles that describe it as "just like the human brain". It's a useful technique that lets you skip lots of feature engineering by just letting the classifier learn its own features, but it is also really easy to project abilities onto neural networks that they don't really have. Deep learning is not magic, for every network architecture that beats the state of the art, there are a hundred very similar ones that completely fail, or run too slow, or don't fit into GPU memory ... and the only way we know to get improvements is to fiddle with the hyperparameters until everything works out.

Ar-Curunir 9y ago

Deep learning has had much greater impact within AI than cryptocurrencies have had in cryptography/systems/security

vadimberman 9y ago
I should have added, "what cryptocurrencies are to fintech". Is it better :) ?

myaso 9y ago

To get AI you have an interface problem you need to solve since reality is a bitch to parse; this is just the start, what about the formulation of goals? How would you even make that in code? You would need to start with a certain kind of data that was extracted by the sensors but how will you ever know you have the correct set? You have two problems to solve simultaneously getting the right data and building the right system on top of it -- this core data when done in reality is not the same core data that would make this possible in a simulation and the system on top while maybe having the same fundamental form is also very likely different. Makings intelligent agents in a simulation is not the same as making intelligent agents arbitrarily useful in the real world, granted a simulation is a good means to explore techniques -- in a simulation I control the sensory apparatus of the agent perfectly. So I think the lesson is you either build systems from which it is easy to stream data and configure programmatically or deal with the fact that this is likely impossible without simulating a universe to high fidelity. Deep Learning is extremely useful as it is, not the same as cryptocurrencies which are useful but this usefulness will never be properly realized due to human nature -- right now their primary purpose seems to be to satiate greed and humans with poor impulse control while promising a lot of outlandish things.

vadimberman 9y ago
You hit the nail on the head. Oren Etzioni has been shouting about the knowledge representation for years (full disclosure: this is something I'm focused on, too).

goatlover 9y ago

Why do we want to reach AGI? There are around 7.5 billion biological general intelligences, and we already know how to make more of them. Is it to save companies labor costs and increase their profit margins? I'm not sure that's ultimately in the interest of most people, or even the long term health of the economy. Seriously, why do we want to replace human intelligence? I get augmenting it with narrow forms of automation, but AGI is a different animal.

riku_iki 9y ago

> I don't think anyone major ever disputed that. Is it that obvious though? Any good evidence that human brain on the planet scale cluster can't be modelled with DL?