3 ms·
RL agents (and humans) don't actually need perfect internal simulation of the environment. Instead we need ability to quickly recover from deviations, when the
by visarga 9y ago
RL agents (and humans) don't actually need perfect internal simulation of the environment. Instead we need ability to quickly recover from deviations, when the model diverges from reality, like self driving cars. We are using the world itself as a "simulator".
The problem with AI agents doing the same is that it's expensive. We'd need cheap, fast, indestructible robots and environments to do that. Humans benefited form millions of years of evolution, and previous species took billions of years to evolve up to our species. That's a lot of "high resolution environment time" which would be impossible to recreate in silico as of now.
Then there's the problem of sample complexity - humans need less examples than AI agents to learn, and that's because we have prior knowledge about the world encoded in our genetic code by evolution, while RL agents don't. We have to bake in some pretty clever prior knowledge in RL agents if we want them to be efficient.
So AI's can make due with imperfect internal simulation, but they need perfect external simulation of the same complexity with the real world in order to become smart like us.