5 ms·

I'm relatively unfamiliar with people who are actually arguing that deep learning is all that is needed for AGI - Andrew Ng can say what he wants, but do many o

by twtw 8y ago

I'm relatively unfamiliar with people who are actually arguing that deep learning is all that is needed for AGI - Andrew Ng can say what he wants, but do many of the researchers and engineers working with deep learning systems actually think this way? I tend to think of deep learning as adding additional sensory capabilities to machines. In this sense, is a network misclassifying a school bus as a snowplow significantly different from a GPS sensor generating garbage position data due to multipath? All existing sensors are noisy and can get confused, and yet no one says that GPS isn't useful because it can be so completely wrong in urban canyons. Of course, no one says GPS is going to become sentient - this brings us back to my first question above.

AndrewKemendo 8y ago
You're unfamiliar with them because there aren't any. At least none that I have seen which have any kind of history of research or applications. See my discussion with Gary here: https://twitter.com/AndrewKemendo/status/1068908423956312071 https://twitter.com/AndrewKemendo/status/1068908423956312071 Ng wasn't even really saying it in a way that is explicitly making the case - at least not enough to cause this kind of fracas. It's a bit unfortunate really, and I'm not sure what's driving the animosity or aggressiveness around this issue. While I agree with Marcus, I don't think anyone actually disagrees - largely because nobody is arguing what he is trying to address. Marcus has built a strawman and is tilting at Windmills a bit.
- woodandsteel 8y ago
  I have a question. Marcus is saying that to get AGI, or at least get a lot closer to it, we need to combine ML with symbolic manipulation. Do you agree or disagree?
  - AndrewKemendo 8y ago
    His position is actually too poorly designed to really evaluate. However I think it's a reasonable argument to make that a General Intelligence may use some form of a symbolic system. I'd even accept the claim that General Intelligence would require some form of expert systems to cross the threshold. So in that sense, combining DL with Expert Systems, other symbolic methods or approaches like Bayesian Causal analysis is probably required.
Izkata 8y ago
This is one of the reasons I really liked Terminator: The Sarah Connor Chronicles. A large chunk of the first season was making similar points as you, explaining that Skynet wasn't one system, it was dozens all working together. For example, one episode focused on a traffic camera system becoming it's "eyes" into the world. One of the biggest missing pieces in my mind has always been, what drives it? What does this theoretical AI want/need, and how does it know to maximize on those values?
evrydayhustling 8y ago
> do many of the researchers and engineers working with deep learning systems actually think this way? No, this has been a straw-man argument, made repeatedly in popular press for public recognition, for years. Even the NYT article to which Marcus responded in 2012 [1] does not make the claims to which it responds! Here is the sentence in Marcus' New Yorker post where it transitions to attacking a claim nobody made (hint - look at where the quote ends): > While the Times reports that “advances in an artificial intelligence technology that can recognize patterns offer the possibility of machines that perform human activities like seeing, listening and thinking,” deep learning takes us, at best, only a small step toward the creation of truly intelligent machines. As a rule, the academic deep learning community avoids discussions and claims about what constitutes a "truly intelligent machine", instead focusing on results in specific applications which can be objectively measured. I'm sure you can find both academics and public press that claim deep learning is a panacea. But it's disingenuous punching up to continually imply that leaders in the field are slow to acknowledge its limits. [1] https://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html https://www.nytimes.com/2012/11/24/science/scientists-see-ad...
- AndrewKemendo 8y ago
  Very eloquently said. A trend in the DL/ML research community is the strong and vocal distancing from discussing or approaching the problem of General Intelligence or anything which might look like sci-fi AI. You can see this with aggressive railing against silly robots like Sophia and others, which have gotten press. I think this is warranted, and understandable as the field is so scared of another AI winter. On the other hand it leaves the field without a real progress vector. Especially given that broadly, ML research isn't scientific as such. In other words, the vast majority of "research" is not around hypothesis testing and using computing to test questions about intelligence. Rather it's demonstrating increments of improvement within very narrow computing tasks. I make that point not to diminish the work or results, as they've obviously been astounding. Rather, to point out that unlike basic or social sciences research it's not attempting to surface fundamental truths (theories, laws etc...) about some natural or emergent phenomena, but rather continuously benchmarking on narrow tasks - often without prior reference. Now that ILSVRC is gone, I'm wondering what kind of benchmarking is going to emerge as the new vector for the field.
cgearhart 8y ago
Yes and no. I've spoken to many folks who believe that deep learning == ai, or rather that the rest of "AI" (especially older stuff) is obviated by end-to-end optimization of an appropriate network. However, I don't know anyone working in AI who thinks that deep learning today is sufficient for AGI. I _suspect_ that many quietly believe that continued advances in deep learning will produce AGI.
Eridrus 8y ago
I think Greg Brockman from OpenAI has given talks where he argues that we may be able to get AGI by just scaling up existing algorithms. Their work on DoTA 2 is basically on this track of just scaling up well known algos.

by twtw 8y ago

AndrewKemendo 8y ago

You're unfamiliar with them because there aren't any. At least none that I have seen which have any kind of history of research or applications. See my discussion with Gary here: https://twitter.com/AndrewKemendo/status/1068908423956312071 https://twitter.com/AndrewKemendo/status/1068908423956312071 Ng wasn't even really saying it in a way that is explicitly making the case - at least not enough to cause this kind of fracas. It's a bit unfortunate really, and I'm not sure what's driving the animosity or aggressiveness around this issue. While I agree with Marcus, I don't think anyone actually disagrees - largely because nobody is arguing what he is trying to address. Marcus has built a strawman and is tilting at Windmills a bit.

woodandsteel 8y ago
I have a question. Marcus is saying that to get AGI, or at least get a lot closer to it, we need to combine ML with symbolic manipulation. Do you agree or disagree?
- AndrewKemendo 8y ago
  His position is actually too poorly designed to really evaluate. However I think it's a reasonable argument to make that a General Intelligence may use some form of a symbolic system. I'd even accept the claim that General Intelligence would require some form of expert systems to cross the threshold. So in that sense, combining DL with Expert Systems, other symbolic methods or approaches like Bayesian Causal analysis is probably required.

Izkata 8y ago

This is one of the reasons I really liked Terminator: The Sarah Connor Chronicles. A large chunk of the first season was making similar points as you, explaining that Skynet wasn't one system, it was dozens all working together. For example, one episode focused on a traffic camera system becoming it's "eyes" into the world. One of the biggest missing pieces in my mind has always been, what drives it? What does this theoretical AI want/need, and how does it know to maximize on those values?

evrydayhustling 8y ago

> do many of the researchers and engineers working with deep learning systems actually think this way? No, this has been a straw-man argument, made repeatedly in popular press for public recognition, for years. Even the NYT article to which Marcus responded in 2012 [1] does not make the claims to which it responds! Here is the sentence in Marcus' New Yorker post where it transitions to attacking a claim nobody made (hint - look at where the quote ends): > While the Times reports that “advances in an artificial intelligence technology that can recognize patterns offer the possibility of machines that perform human activities like seeing, listening and thinking,” deep learning takes us, at best, only a small step toward the creation of truly intelligent machines. As a rule, the academic deep learning community avoids discussions and claims about what constitutes a "truly intelligent machine", instead focusing on results in specific applications which can be objectively measured. I'm sure you can find both academics and public press that claim deep learning is a panacea. But it's disingenuous punching up to continually imply that leaders in the field are slow to acknowledge its limits. [1] https://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html https://www.nytimes.com/2012/11/24/science/scientists-see-ad...

AndrewKemendo 8y ago
Very eloquently said. A trend in the DL/ML research community is the strong and vocal distancing from discussing or approaching the problem of General Intelligence or anything which might look like sci-fi AI. You can see this with aggressive railing against silly robots like Sophia and others, which have gotten press. I think this is warranted, and understandable as the field is so scared of another AI winter. On the other hand it leaves the field without a real progress vector. Especially given that broadly, ML research isn't scientific as such. In other words, the vast majority of "research" is not around hypothesis testing and using computing to test questions about intelligence. Rather it's demonstrating increments of improvement within very narrow computing tasks. I make that point not to diminish the work or results, as they've obviously been astounding. Rather, to point out that unlike basic or social sciences research it's not attempting to surface fundamental truths (theories, laws etc...) about some natural or emergent phenomena, but rather continuously benchmarking on narrow tasks - often without prior reference. Now that ILSVRC is gone, I'm wondering what kind of benchmarking is going to emerge as the new vector for the field.

cgearhart 8y ago

Yes and no. I've spoken to many folks who believe that deep learning == ai, or rather that the rest of "AI" (especially older stuff) is obviated by end-to-end optimization of an appropriate network. However, I don't know anyone working in AI who thinks that deep learning today is sufficient for AGI. I _suspect_ that many quietly believe that continued advances in deep learning will produce AGI.

Eridrus 8y ago

I think Greg Brockman from OpenAI has given talks where he argues that we may be able to get AGI by just scaling up existing algorithms. Their work on DoTA 2 is basically on this track of just scaling up well known algos.