4 ms·

While I haven't read those particular papers yet, one common pattern in the ML literature seems to be: 1) Identify a commonly seen problem with deep learning a

While I haven't read those particular papers yet, one common pattern in the ML literature seems to be: 1) Identify a commonly seen problem with deep learning architectures, whether that's large data volumes, lack of transfer learning, etc. 2) Invent a solution to the problem. 3) Test that the solution works on toy examples, like MNIST, simple block worlds, simulated data, etc. 4) Hint that the technique, now proven to work, will naturally be extended to real data sets very soon, so we should consider the problem basically solved now. Hooray! 5) Return to step #1. If anyone applies the technique to real data sets, they find, of course, that it doesn't generalize well and works only on toy examples. This is simply another form of what happened in the 60s and 70s, when many expected that SHRDLU and ELIZA would rapidly be extended to human-like, general-purpose intelligences, with just a bit of tweaking and a bit more computing power. Of course, that never happened. We still don't have that today, and when we do, I'm sure the architecture will look very very different from 1970s AI (or modern chatbots, for that matter, which are mostly built the same way as ELIZA). I don't mean to be too cynical. Like I said, I haven't read those particular papers yet, so I can't fairly pass judgement on them. I'm just saying that historically, saying problem X "has been addressed" by Y doesn't always mean very much. See also eg. the classic paper "Artificial Intelligence Meets Natural Stupidity": https://dl.acm.org/citation.cfm?id=1045340 https://dl.acm.org/citation.cfm?id=1045340. EDIT: To be clear, I'm not saying that people shouldn't explore new architectures, test new ideas, or write up papers about them, even if they haven't been proven to work yet. That's part of what research is. The problem comes when there's an expectation that an idea about how to solve the problem means that the problem is close to being solved. Most ideas don't work very well and have to be abandoned later. Eg., as one example, this is the Neural Turing Machine paper from a few years back: https://arxiv.org/pdf/1410.5401.pdf https://arxiv.org/pdf/1410.5401.pdf It's a cool idea. I'm glad someone tried it out. But the paper was widely advertised in the mainstream press as being successful, even though it was not tested on "hard" data sets, and (to the best of my knowledge) it still hasn't several years later. That creates unrealistic expectations.

trextrex 8y ago
While I don't completely disagree with you, how would you propose researchers go about the problem? If anything, machine learning is applied to real world problems these days more than it ever was. For better or worse, AGI is a hard problem, that's going to take a long time to solve. And we're not going to solve it without exploring what works and what doesn't.
- skywhopper 8y ago
  I think the mere fact that the OP feels the need to state that (paraphrasing) "possibly additional techniques besides deep learning will be likely necessary to reach AGI" reveals just how deeply the hype has infected the research community. This overblown self-delusion infects reporting on self-driving cars, automatic translation, facial recognition, content generation, and any number of other tasks that have reached the sort-of-works-but-not-really point with deep learning methods. But however rapid recent progress has been, these things won't be "solved" anytime soon, and we keep falling into the trap of believing the hype based on toy results. It'll be better for the researchers, investors, and society to be a little more skeptical of the claim that "computers can solve everything, we're 80% of the way there, just give us more time and money, and don't try to solve the problems any other way while you wait!"
  - trextrex 8y ago
    Agreed. The hype surrounding machine learning is quite disproportionate to what's actually going on. But it's always been that way with machine learning -- maybe because it captures the public's imagination like few other fields do. And there are definitely researchers, top ones no less, who play along with the hype. Very likely to secure more funding, and more attention for themselves and the field. Which has turned out to be quite an effective strategy, if you think about it. The other upside of this hype is that it ends up attracting a lot of really smart people to work on this field, because of the money involved. So each hype cycle leads to greater progress. The crash afterwards might slow things down a bit, particularly in the private sector. But the quantum of government funding available changes much more slowly, and could well last until the next hype cycle starts.
    - YeGoblynQueenne 8y ago
      >> The other upside of this hype is that it ends up attracting a lot of really smart people to work on this field, because of the money involved. The hype certainly attracts people who are "smart" in the sense that they know how to profit from it, but that doesn't mean they can actually do useful research. The result is, like the other poster says, a huge number of papers that claim to have solved really hard problems, which of course remain far from solved; in other words, so much useless noise. It's what you can expect when you see everyone and their little sister jumping on a bandwagon when the money starts pouring in. Greed is great for making money, but not so much for making progress.
goolulusaurs 8y ago
The Neural Turing Machine has been expanded upon considerably since that paper, and inspired a number of other memory augmented neural network architectures, including: https://www.gwern.net/docs/rl/2016-graves.pdf https://www.gwern.net/docs/rl/2016-graves.pdf , https://arxiv.org/abs/1605.06065 https://arxiv.org/abs/1605.06065 and https://arxiv.org/abs/1803.10760 https://arxiv.org/abs/1803.10760 . If you don't want to read a bunch of papers this video also has a good discussion of that last paper, describing how a memory system based on the NTM has been applied to reinforcement learning to achieve very impressive results that seem to me to be a very significant step towards human-like general purpose intelligence: https://www.youtube.com/watch?v=9z3_tJAu7MQ https://www.youtube.com/watch?v=9z3_tJAu7MQ
- sterlind 8y ago
  Exciting to see a video referencing the MERLIN paper! I hope Deepmind decides to release the code. The paper outlines the architecture well, but reproducing RL results is very tricky. With so many interlinked neural networks in the closed loop, it'll be slow going to isolate failures due to bugs from unfortunate hyperparameter selection or starting seed.. I'd still like to try, though!
  - goolulusaurs 8y ago
    Yeah, it is super interesting. I've been gradually working on reproducing it too, mostly as a way to challenge myself and to try to keep on top of some of the cool RL research that has been coming out lately, but I've still got a bit left to do. I started out by working on World Models (https://worldmodels.github.io/ https://worldmodels.github.io/) since it is conceptually similar, but without the memory system and the components are more isolated and easier to test. It has been a lot of fun though, and all the background reading has been very educational! In the Merlin paper I do appreciate how thorough the description of the architecture is, especially compared to some of the earlier deep RL papers. I am hoping since its just a preprint we may get code released when/if it gets officially published, although maybe its not too likely given their history.
vinn124 8y ago
> 3) Test that the solution works on toy examples, like MNIST, simple block worlds, simulated data, etc. youre right: mnist, imagenet, etc are toy examples that do not extend into the real world. but the point of reproducible research is to experiment on agreed upon, existing benchmarks.

trextrex 8y ago

While I don't completely disagree with you, how would you propose researchers go about the problem? If anything, machine learning is applied to real world problems these days more than it ever was. For better or worse, AGI is a hard problem, that's going to take a long time to solve. And we're not going to solve it without exploring what works and what doesn't.

skywhopper 8y ago
I think the mere fact that the OP feels the need to state that (paraphrasing) "possibly additional techniques besides deep learning will be likely necessary to reach AGI" reveals just how deeply the hype has infected the research community. This overblown self-delusion infects reporting on self-driving cars, automatic translation, facial recognition, content generation, and any number of other tasks that have reached the sort-of-works-but-not-really point with deep learning methods. But however rapid recent progress has been, these things won't be "solved" anytime soon, and we keep falling into the trap of believing the hype based on toy results. It'll be better for the researchers, investors, and society to be a little more skeptical of the claim that "computers can solve everything, we're 80% of the way there, just give us more time and money, and don't try to solve the problems any other way while you wait!"
- trextrex 8y ago
  Agreed. The hype surrounding machine learning is quite disproportionate to what's actually going on. But it's always been that way with machine learning -- maybe because it captures the public's imagination like few other fields do. And there are definitely researchers, top ones no less, who play along with the hype. Very likely to secure more funding, and more attention for themselves and the field. Which has turned out to be quite an effective strategy, if you think about it. The other upside of this hype is that it ends up attracting a lot of really smart people to work on this field, because of the money involved. So each hype cycle leads to greater progress. The crash afterwards might slow things down a bit, particularly in the private sector. But the quantum of government funding available changes much more slowly, and could well last until the next hype cycle starts.
  - YeGoblynQueenne 8y ago
    >> The other upside of this hype is that it ends up attracting a lot of really smart people to work on this field, because of the money involved. The hype certainly attracts people who are "smart" in the sense that they know how to profit from it, but that doesn't mean they can actually do useful research. The result is, like the other poster says, a huge number of papers that claim to have solved really hard problems, which of course remain far from solved; in other words, so much useless noise. It's what you can expect when you see everyone and their little sister jumping on a bandwagon when the money starts pouring in. Greed is great for making money, but not so much for making progress.

goolulusaurs 8y ago

The Neural Turing Machine has been expanded upon considerably since that paper, and inspired a number of other memory augmented neural network architectures, including: https://www.gwern.net/docs/rl/2016-graves.pdf https://www.gwern.net/docs/rl/2016-graves.pdf , https://arxiv.org/abs/1605.06065 https://arxiv.org/abs/1605.06065 and https://arxiv.org/abs/1803.10760 https://arxiv.org/abs/1803.10760 . If you don't want to read a bunch of papers this video also has a good discussion of that last paper, describing how a memory system based on the NTM has been applied to reinforcement learning to achieve very impressive results that seem to me to be a very significant step towards human-like general purpose intelligence: https://www.youtube.com/watch?v=9z3_tJAu7MQ https://www.youtube.com/watch?v=9z3_tJAu7MQ

sterlind 8y ago
Exciting to see a video referencing the MERLIN paper! I hope Deepmind decides to release the code. The paper outlines the architecture well, but reproducing RL results is very tricky. With so many interlinked neural networks in the closed loop, it'll be slow going to isolate failures due to bugs from unfortunate hyperparameter selection or starting seed.. I'd still like to try, though!
- goolulusaurs 8y ago
  Yeah, it is super interesting. I've been gradually working on reproducing it too, mostly as a way to challenge myself and to try to keep on top of some of the cool RL research that has been coming out lately, but I've still got a bit left to do. I started out by working on World Models (https://worldmodels.github.io/ https://worldmodels.github.io/) since it is conceptually similar, but without the memory system and the components are more isolated and easier to test. It has been a lot of fun though, and all the background reading has been very educational! In the Merlin paper I do appreciate how thorough the description of the architecture is, especially compared to some of the earlier deep RL papers. I am hoping since its just a preprint we may get code released when/if it gets officially published, although maybe its not too likely given their history.

vinn124 8y ago

> 3) Test that the solution works on toy examples, like MNIST, simple block worlds, simulated data, etc. youre right: mnist, imagenet, etc are toy examples that do not extend into the real world. but the point of reproducible research is to experiment on agreed upon, existing benchmarks.