4 ms·

Thanks for explaining your take but this sounds very reductive. When it comes down to it every problem is solved by tuning for that problem specifically. While

Thanks for explaining your take but this sounds very reductive. When it comes down to it every problem is solved by tuning for that problem specifically. While Never Give Up (NGU) is not fundamentally new, it is an important step in computers learning. You need to be able to generalize solutions to problems where you don't have contextual information. Imagine if you were a caveman and asked to operate an iPhone. You're not stupid if you don't know how, but if I tell you, "never give up", and put you in a room for 5 years I'd expect some results from a sentient being. This is an important process too. We would be much closer to good AI if it can figure things out by itself instead of being constantly fed "clean" data.

WnZ39p0Dgydaz1 7y ago
There's a difference between research and engineering. Is this system impressive? Definitely! It's a complex engineering effort, and highly tuned to solve a specific problem - beat the Atari benchmark. Does it teach us anything fundamentally new? No, it still has horrible sample complexity and does not generalize to anything outside of Atari unless you completely re-tune it. And I don't mean re-train. I mean changing the architecture and assumptions. That's different from projects such as e.g. AlphaZero or MuZero. IMO this would've been more appropriate to be published as an open-source system so that it can be applied and tuned to other problems as opposed to a research paper. As research, nobody outside of DeepMind can ever reproduce this. You are completely changing the topic with this: > While Never Give Up (NGU) is not fundamentally new, it is an important step in computers learning. You need to be able to generalize solutions to problems where you don't have contextual information We're not even talking about NGU, we're talking about the paper linked in this post. This specific paper proposes little new in that regard. It just engineers a system to do this specifically for Atari games by taking a previous paper and changing some parameters. Neat, but it's not some kind of breakthrough.
- visarga 7y ago
  > Does it teach us anything fundamentally new? No, it still has horrible sample complexity and does not generalize to anything outside of Atari unless you completely re tune it. I thought it was interesting that they let the agent learn the exploration/exploitation tradeoff, also combining memory and intrinsic motivation. Another contribution of this paper would be that they showed all these tricks can build on each other, thus, are complementary. Humans brains are also a bag of tricks fine-tuned to the goal and requirements of making more humans.
AndrewKemendo 7y ago
I think you're both right but are pointing at different ways to approach the intelligence problem. This is kind of the Connectionist vs Symbolic debate. The fundamental question is, is a representational (contextual) bootstrap required in the long run for a contained computational system to perform at human level across a large number of domains? This isn't a solved problem. So yes, AI would be better if it could "figure things out by itself" however humans don't "figure things out by themselves" they come pre-wired with a lot out of the box and a lot of help cleaning the data (parents, teachers, literal labels etc...)
- silvaring 7y ago
  Wouldn't the 'pre wiring' in this instance be the code and hardware that the algorithm was running on?

WnZ39p0Dgydaz1 7y ago

There's a difference between research and engineering. Is this system impressive? Definitely! It's a complex engineering effort, and highly tuned to solve a specific problem - beat the Atari benchmark. Does it teach us anything fundamentally new? No, it still has horrible sample complexity and does not generalize to anything outside of Atari unless you completely re-tune it. And I don't mean re-train. I mean changing the architecture and assumptions. That's different from projects such as e.g. AlphaZero or MuZero. IMO this would've been more appropriate to be published as an open-source system so that it can be applied and tuned to other problems as opposed to a research paper. As research, nobody outside of DeepMind can ever reproduce this. You are completely changing the topic with this: > While Never Give Up (NGU) is not fundamentally new, it is an important step in computers learning. You need to be able to generalize solutions to problems where you don't have contextual information We're not even talking about NGU, we're talking about the paper linked in this post. This specific paper proposes little new in that regard. It just engineers a system to do this specifically for Atari games by taking a previous paper and changing some parameters. Neat, but it's not some kind of breakthrough.

visarga 7y ago
> Does it teach us anything fundamentally new? No, it still has horrible sample complexity and does not generalize to anything outside of Atari unless you completely re tune it. I thought it was interesting that they let the agent learn the exploration/exploitation tradeoff, also combining memory and intrinsic motivation. Another contribution of this paper would be that they showed all these tricks can build on each other, thus, are complementary. Humans brains are also a bag of tricks fine-tuned to the goal and requirements of making more humans.

AndrewKemendo 7y ago

I think you're both right but are pointing at different ways to approach the intelligence problem. This is kind of the Connectionist vs Symbolic debate. The fundamental question is, is a representational (contextual) bootstrap required in the long run for a contained computational system to perform at human level across a large number of domains? This isn't a solved problem. So yes, AI would be better if it could "figure things out by itself" however humans don't "figure things out by themselves" they come pre-wired with a lot out of the box and a lot of help cleaning the data (parents, teachers, literal labels etc...)

silvaring 7y ago
Wouldn't the 'pre wiring' in this instance be the code and hardware that the algorithm was running on?