5 ms·

There's also the AI adage that you don't get to the moon by building successively larger ladders... all this deep learning stuff is great and will unlock amazin

by arise 4y ago

There's also the AI adage that you don't get to the moon by building successively larger ladders... all this deep learning stuff is great and will unlock amazing value (esp if we fund open data+models), but we run the risk of exhausting consumer and government patience if we keep promising the world with this single family of techniques.

TremendousJudge 4y ago
Not sure if the analogy holds. Human brains work on the same principle as lizard brais, no change in strategy was needed to go from lizard intelligence to human intelligence. I'm not saying that DL holds this promise though.
- russdill 4y ago
  - Zamiel_Snawley 4y ago
    Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community.[1] [1] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- deleted 4y ago
  [deleted]
- eternalban 4y ago
  https://en.wikipedia.org/wiki/Triune_brain https://en.wikipedia.org/wiki/Triune_brain There are distinct architectures and our brains have additional architectural features that lizards' lack. https://en.wikipedia.org/wiki/Basal_ganglia https://en.wikipedia.org/wiki/Basal_ganglia https://en.wikipedia.org/wiki/Limbic_system https://en.wikipedia.org/wiki/Limbic_system https://en.wikipedia.org/wiki/Neocortex https://en.wikipedia.org/wiki/Neocortex
  - TremendousJudge 4y ago
    Yes, but unlike a ladder and a rocket, they are not fundamentally different. This is why I don't think the analogy holds. Evolution iterated over the same model, never starting again from scratch.
obblekk 4y ago
That's interesting and something I hadn't heard before. Any pointers to further reading on why deep learning might be a dead end?
- burnished 4y ago
  They didn't say it was a dead end. I think their observation is rooted in gradient descent and how it improves but gets stuck in local maximas, when that solution might never be good enough. Hence the ladder. No matter how good you get at ladder making it with take you to the moon, but it will get you closer the entire time you improve at it.
- PeterisP 4y ago
  For a contrasting viewpoint, there's always Sutton's Bitter Lesson - http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html
- vba616 4y ago
  How about a simple explanation of why it seems palpably absurd to some people that it will not be a dead end? Consider a teratoma or lab-grown organ. Is grafting or engineering another attached sensory mechanism going to bring it closer to being an ordinary organism? I think whether capabilities are super- or sub- human is a red herring. Even a really primitive organism is still taking all of its computational capabilities and outputting, implicitly, decisions in one context that is its perceived reality. A collection of computation and perception modules does not do this, without something else. I don't think developing "something else" is obviously impossible or would require magic. But I'm not sure anyone sane would want to create it when it inherently creates unlimited risk of running amok. This is what the LessWrong people are afraid of, aren't they?
- danaris 4y ago
  I think it's important to distinguish between something that's a "dead end" in that it will take you nowhere interesting, and something that's a "dead end" in that it will, after a certain point, cease advancing. The former would imply that there is no point using deep learning and other similar techniques at all, and is the common implication when people say something is a "dead end". The latter is what I believe the current generation of machine learning to be: I do not believe it will lead to AGI, and I am skeptical that it can do a great deal more than what it has already done (it can continue to refine the types of things it already does, and I expect it to do so, but I don't think it will open up new categories of things it can do many more times). But despite that, it does do some very cool things now, and as they are refined, I think they can be commercially successful and generally beneficial tools.

by arise 4y ago

TremendousJudge 4y ago

Not sure if the analogy holds. Human brains work on the same principle as lizard brais, no change in strategy was needed to go from lizard intelligence to human intelligence. I'm not saying that DL holds this promise though.

russdill 4y ago
- Zamiel_Snawley 4y ago
  Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community.[1] [1] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
deleted 4y ago
[deleted]
eternalban 4y ago
https://en.wikipedia.org/wiki/Triune_brain https://en.wikipedia.org/wiki/Triune_brain There are distinct architectures and our brains have additional architectural features that lizards' lack. https://en.wikipedia.org/wiki/Basal_ganglia https://en.wikipedia.org/wiki/Basal_ganglia https://en.wikipedia.org/wiki/Limbic_system https://en.wikipedia.org/wiki/Limbic_system https://en.wikipedia.org/wiki/Neocortex https://en.wikipedia.org/wiki/Neocortex
- TremendousJudge 4y ago
  Yes, but unlike a ladder and a rocket, they are not fundamentally different. This is why I don't think the analogy holds. Evolution iterated over the same model, never starting again from scratch.

obblekk 4y ago

That's interesting and something I hadn't heard before. Any pointers to further reading on why deep learning might be a dead end?

burnished 4y ago
They didn't say it was a dead end. I think their observation is rooted in gradient descent and how it improves but gets stuck in local maximas, when that solution might never be good enough. Hence the ladder. No matter how good you get at ladder making it with take you to the moon, but it will get you closer the entire time you improve at it.
PeterisP 4y ago
For a contrasting viewpoint, there's always Sutton's Bitter Lesson - http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html
vba616 4y ago
How about a simple explanation of why it seems palpably absurd to some people that it will not be a dead end? Consider a teratoma or lab-grown organ. Is grafting or engineering another attached sensory mechanism going to bring it closer to being an ordinary organism? I think whether capabilities are super- or sub- human is a red herring. Even a really primitive organism is still taking all of its computational capabilities and outputting, implicitly, decisions in one context that is its perceived reality. A collection of computation and perception modules does not do this, without something else. I don't think developing "something else" is obviously impossible or would require magic. But I'm not sure anyone sane would want to create it when it inherently creates unlimited risk of running amok. This is what the LessWrong people are afraid of, aren't they?
danaris 4y ago
I think it's important to distinguish between something that's a "dead end" in that it will take you nowhere interesting, and something that's a "dead end" in that it will, after a certain point, cease advancing. The former would imply that there is no point using deep learning and other similar techniques at all, and is the common implication when people say something is a "dead end". The latter is what I believe the current generation of machine learning to be: I do not believe it will lead to AGI, and I am skeptical that it can do a great deal more than what it has already done (it can continue to refine the types of things it already does, and I expect it to do so, but I don't think it will open up new categories of things it can do many more times). But despite that, it does do some very cool things now, and as they are refined, I think they can be commercially successful and generally beneficial tools.