3 ms·

I think if you watch a child play anything, even a video game, you'll see that when it's not immediately obvious what to do, a human will just 'mess around' and

by aab0 10y ago

I think if you watch a child play anything, even a video game, you'll see that when it's not immediately obvious what to do, a human will just 'mess around' and 'try stuff out' to 'just to see what happens'. As the paper says, the idea of novelty bonuses in reinforcement learning draws directly from discussions of curiosity and play and intrinsic motivation in animals and humans.

chongli 10y ago
Initially, yes, but then as a human learns more about the game they begin to make plans at a much higher level than just the basic mechanics of moving around. In the video, the machine continues to die in very basic ways even after a hundred million iterations. A human who had played the game even a tiny fraction of that many times would be expected to complete an entire play-through flawlessly. And this still doesn't address my original question. A human who was able to master Montezuma's Revenge would have a dramatic advantage in learning and mastering the game Zelda compared to somebody who had played neither game before. What experience, if any, could this machine be expected to bring to Zelda, assuming no modification by the researchers?
- gwern 10y ago
  You would get some boost if you use a NN factored to have game-specific module feeding into a universal game-playing NN: https://arxiv.org/abs/1511.06342 https://arxiv.org/abs/1511.06342 " Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning", Parisotto et al 2015 > The ability to act in multiple environments and transfer previous knowledge to new situations can be considered a critical aspect of any intelligent agent. Towards this goal, we define a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains. This method, termed "Actor-Mimic", exploits the use of deep reinforcement learning and model compression techniques to train a single policy network that learns how to act in a set of distinct tasks by using the guidance of several expert teachers. We then show that the representations learnt by the deep policy network are capable of generalizing to new tasks with no prior expert guidance, speeding up learning in novel environments. Although our method can in general be applied to a wide range of problems, we use Atari games as a testing environment to demonstrate these methods.

by aab0 10y ago

chongli 10y ago

Initially, yes, but then as a human learns more about the game they begin to make plans at a much higher level than just the basic mechanics of moving around. In the video, the machine continues to die in very basic ways even after a hundred million iterations. A human who had played the game even a tiny fraction of that many times would be expected to complete an entire play-through flawlessly. And this still doesn't address my original question. A human who was able to master Montezuma's Revenge would have a dramatic advantage in learning and mastering the game Zelda compared to somebody who had played neither game before. What experience, if any, could this machine be expected to bring to Zelda, assuming no modification by the researchers?

gwern 10y ago
You would get some boost if you use a NN factored to have game-specific module feeding into a universal game-playing NN: https://arxiv.org/abs/1511.06342 https://arxiv.org/abs/1511.06342 " Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning", Parisotto et al 2015 > The ability to act in multiple environments and transfer previous knowledge to new situations can be considered a critical aspect of any intelligent agent. Towards this goal, we define a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains. This method, termed "Actor-Mimic", exploits the use of deep reinforcement learning and model compression techniques to train a single policy network that learns how to act in a set of distinct tasks by using the guidance of several expert teachers. We then show that the representations learnt by the deep policy network are capable of generalizing to new tasks with no prior expert guidance, speeding up learning in novel environments. Although our method can in general be applied to a wide range of problems, we use Atari games as a testing environment to demonstrate these methods.