4 ms·

I love the different yet similar 3D environments for RL agent learning. OpenAI has a similar environment they call OpenAI Gym[1] to train agents, of which RoboS

I love the different yet similar 3D environments for RL agent learning. OpenAI has a similar environment they call OpenAI Gym[1] to train agents, of which RoboSchool is the closest environmental[2] analogy to DMLab-30. From a cursory glance, DMLab-30 levels are based on id's code, while RoboSchool uses a full-blown physics engine (Bullet[3]). Pretty exciting times! [1] https://gym.openai.com/envs/ https://gym.openai.com/envs/ [2] https://blog.openai.com/roboschool/ https://blog.openai.com/roboschool/ [3] https://pybullet.org/wordpress/ https://pybullet.org/wordpress/

visarga 9y ago
These game environments are the sandbox where AI agents need to play in order to learn and evolve. Just like human children need to play, RL agents have the same need. There is a fundamental difference between learning statistical patterns from a fixed dataset (supervised learning) and learning by interactivity and exploration (reinforcement learning), where the agent can create and test hypothesis, thus gaining causal inference powers. A simulator is an unlimited, dynamic dataset, so much more than, say, ImageNET or even Wikipedia. The ability of an agent to create its own predictions about the evolution of the environment is essential in planning and reasoning. That's how we get to be so smart - we have a mental simulator we can apply to any situation to check what the outcome would be. When agents learn to do the same, they too can become smart. AlphaGo for example was doing that - planning ahead (MCTS) and that's how it beat us at Go. A simple neural net without planning wouldn't have beaten the best humans. I think simulation is going to be the next ingredient we add to AI to make it learn and reason like humans. It's the missing ingredient we need, that combined with RL will lead to the next breakthrough.

visarga 9y ago

These game environments are the sandbox where AI agents need to play in order to learn and evolve. Just like human children need to play, RL agents have the same need. There is a fundamental difference between learning statistical patterns from a fixed dataset (supervised learning) and learning by interactivity and exploration (reinforcement learning), where the agent can create and test hypothesis, thus gaining causal inference powers. A simulator is an unlimited, dynamic dataset, so much more than, say, ImageNET or even Wikipedia. The ability of an agent to create its own predictions about the evolution of the environment is essential in planning and reasoning. That's how we get to be so smart - we have a mental simulator we can apply to any situation to check what the outcome would be. When agents learn to do the same, they too can become smart. AlphaGo for example was doing that - planning ahead (MCTS) and that's how it beat us at Go. A simple neural net without planning wouldn't have beaten the best humans. I think simulation is going to be the next ingredient we add to AI to make it learn and reason like humans. It's the missing ingredient we need, that combined with RL will lead to the next breakthrough.