4 ms·

It is also time to move on to real world environments. Games are cool, but the network is essentially learning some thousand lines of C++. Reinforcement learnin

It is also time to move on to real world environments. Games are cool, but the network is essentially learning some thousand lines of C++. Reinforcement learning needs to prove itself in real-world tasks.

kevinflo 8y ago
I'm a layman, but to my knowledge navigating a digital environment and a real one are the same minus some steps of the process. A self-driving car recreates a digital reality via sensors with as little delay and fidelity loss as possible, then navigates a digital car within that rapidly constructed virtual reality. It then signals back to the real car to navigate the real car exactly how it would navigate the digital one given its immediate digital environment. Since many of the hard problems lie with the navigation in the digital space, removing the sensors-to-digital part of it by training with video games still nets a lot of valuable learnings that can be brought right back into real-world applications. Also, learning in a fully digital environment allows this part of the learning to be done without the spatial/time constraints of reality.
- buboard 8y ago
  games are simulations of idealized processes using generally a small number of equations. a sufficient neural network should be able to learn them but the real world has orders of magnitude higher complexity. Something similar happens with robots trained with physics engines.
deburo 8y ago
One step at a time sounds like a better strategy. Watch this and learn a bit more about why they're doing Starcraft AI. The Deepmind guys first appear around 32:00. https://www.youtube.com/watch?v=cUTMhmVh1qs https://www.youtube.com/watch?v=cUTMhmVh1qs
setr 8y ago
The problem with real world tasks is you can’t really train on them.. you train on a simulation (because its much, much faster/cheaper to run [and debug], thus faster to train) and deploy/test in the real world. So simulation is always a part of it, regardless of target execution environment. In which case, games make a lot of sense, since they’re already complex simulations (thus useful for learning how to train AI’s, and what parts of the simulation are worth having), make good PR, can be tested against quickly, and in the case of competitive games, have a set of “experts” that already study what the human-powered collective AI learned over time, and thus it comes with a set of known expectations of how the AI might fail, and what we expect it to learn. So tldr; simulation vs real world is a red herring. It’s the same learning process, just different execution environments.

kevinflo 8y ago

I'm a layman, but to my knowledge navigating a digital environment and a real one are the same minus some steps of the process. A self-driving car recreates a digital reality via sensors with as little delay and fidelity loss as possible, then navigates a digital car within that rapidly constructed virtual reality. It then signals back to the real car to navigate the real car exactly how it would navigate the digital one given its immediate digital environment. Since many of the hard problems lie with the navigation in the digital space, removing the sensors-to-digital part of it by training with video games still nets a lot of valuable learnings that can be brought right back into real-world applications. Also, learning in a fully digital environment allows this part of the learning to be done without the spatial/time constraints of reality.

buboard 8y ago
games are simulations of idealized processes using generally a small number of equations. a sufficient neural network should be able to learn them but the real world has orders of magnitude higher complexity. Something similar happens with robots trained with physics engines.

deburo 8y ago

One step at a time sounds like a better strategy. Watch this and learn a bit more about why they're doing Starcraft AI. The Deepmind guys first appear around 32:00. https://www.youtube.com/watch?v=cUTMhmVh1qs https://www.youtube.com/watch?v=cUTMhmVh1qs

setr 8y ago

The problem with real world tasks is you can’t really train on them.. you train on a simulation (because its much, much faster/cheaper to run [and debug], thus faster to train) and deploy/test in the real world. So simulation is always a part of it, regardless of target execution environment. In which case, games make a lot of sense, since they’re already complex simulations (thus useful for learning how to train AI’s, and what parts of the simulation are worth having), make good PR, can be tested against quickly, and in the case of competitive games, have a set of “experts” that already study what the human-powered collective AI learned over time, and thus it comes with a set of known expectations of how the AI might fail, and what we expect it to learn. So tldr; simulation vs real world is a red herring. It’s the same learning process, just different execution environments.