5 ms·
It's a really cool phenomena, it also means we have to make our simulations better. These sorts of RL algorithms are so good at finding "exploits" in the physic
by sgillen 8y ago
It's a really cool phenomena, it also means we have to make our simulations better. These sorts of RL algorithms are so good at finding "exploits" in the physics engine they are in that help them "cheat" sometimes, compared to what the researcher wanted.
- TaylorAlexander 8y agoI agree, but FWIW in the linked work they did randomize some of the engine parameters during training to avoid fitting too much to a specific set of assumptions. Certainly though more accurate simulations, as long as they were still fast to compute, would be very useful!
- sgillen 8y agoyeah, I think a big reason why they needed to do randomization in the first place is because of inaccuracies in the simulation compared to the real world. And the extra randomization required almost two orders of magnitude more training time!
- jashmenn 8y agootoh, maybe finding "exploits" is all we ever do. I don't grip with my thumb and little finger because it hurts the back of my hand, but that isn't a problem for Dactyl.
- sgillen 8y agoSure but we want to find exploits in the physical universe not in the approximation that we call a physics engine.
- sitkack 8y agoThis is always true, by using the real universe as the simulation environment, the exploits are applicable.