3 ms·
It has to be fast. Reinforcement learning takes 1000s of hours of game play for each experiment, and you want to do multiple experiments per day on a small clus
by tlb 9y ago
It has to be fast. Reinforcement learning takes 1000s of hours of game play for each experiment, and you want to do multiple experiments per day on a small cluster. The Atari games in OpenAI Gym run about 200x real-time per core, and the SNES games maybe 20x. Aim for something in that range. 1x is useless for research.
OpenAI Gym has a stable API for several video games. If you copy the API, researchers will be able to compare algorithms directly across games which is valuable.
- shawn 9y agoWhat does Dota 2 get? That's probably the metric to aim for, since OpenAI uses it themselves, and it's a modern 3D game. They had to do a few hacks to get it working: https://blog.openai.com/more-on-dota-2/? https://blog.openai.com/more-on-dota-2/?! The first step in the project was figuring out how to run Dota 2 in the cloud on a physical GPU. The game gave an obscure error message on GPU cloud instances. But when starting it on Greg’s personal GPU desktop (which is the desktop brought onstage during the show), we noticed that Dota booted when the monitor was plugged in, but gave the same error message when unplugged. So we configured our cloud GPU instances to pretend there was a physical monitor attached. Dota didn’t support custom dedicated servers at the time, meaning that running scalably and without a GPU was possible only with very slow software rendering. We then created a shim to stub out most OpenGL calls, except the ones needed to boot. Still, Dota uses simplified collision detection. It would be interesting to know how fast an actual physics simulation could run in headless mode.