5 ms·

I don't really understand the intuition on why this helps RL. The original game has a lot more detail, why can't it be used directly?

• FeepingCreature 2y ago
  In the real world, you can't just boot up a copy of reality to play out strategies. You need an internal model.
  • tourmalinetaco 2y ago
    So, effectively, these video game models are proof-of-concepts to say “we can make models with extremely accurate predictions using minimal resources”?
    • vbezhenar 2y ago
      May be I'm wrong but my understanding is that you can film some area using, say, dashcams and then generate this kind of neuro model. Then you can train robot to walk in this area with this neuro-model. It can perform billions of training sessions without touching physical world. Alternatively you can somehow perform 3D scan of area, recreate its 3D model and use, say, game engine to simulate, but that probably requires more effort and not necessarily better.
      • usrusr 2y ago
        And the leg motions we sometimes see in sleeping dogs suggest that this is very much a way how having dreams is useful!
    • usrusr 2y ago
      Not sure where you see the "minimal resources" here? But I'd just counter all questions about "why" with the blanket response of "for understanding natural intelligence": the way biology innovates is that it throws everything against the wall and not pick the one thing that sticks as the winner and focus on that mechanism, it keeps the sticky bits and also everything else as long as their cost isn't prohibitive. Symbolic modeling ("this is an object that can fall down"), prediction chains based on visual similarity patterns (this), hardwired reflexes (we tend to not trust anything that looks and moves like a spider or snake) and who knows what else, it's all there, it all runs in parallel, invited or not, and they all influence each other in subtle and less subtle ways. The interaction is not engineered, it's more like crosstalk that's allowed to happen and has more upside than downside, or else evolution would have preferred variations of the setup that have less of the kind of crosstalk in question. But in our quest to understand us, it's super exciting to see candidates for processes that perhaps play some role in our minds, in isolation, no matter if that role is big or small.
    • empath75 2y ago
      Yes, this is exactly right. What they need is a giant dataset of agent data and audio and video from real world locomotion.
• jampekka 2y ago
  It is used as a predictive model of the environment for model-based RL. I.e. agents can predict consequences of their actions.
  • ThouYS 2y ago
    Oh, I see. I was somehow under the impression that the simulation was the game the RL agent learns to play (which kinda seemed nonsensical).
• visarga 2y ago
  It can use the game directly but if you try this with real life robots, then it is better to do neural simulation before performing an action that could result in injury or damage. We don't need to fall with our cars off the road many times to learn to drive on the road because we can imagine the consequences. Same thing here.