3 ms·

Show HN: A batched, concurrent wrapper for OpenAI gym library

12 points by MutatedFlood 7y ago

Hello HN, new user here, so please let me know if I break some rules. Currently I've been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it. I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested. For more details, please visit: https://github.com/Chimpan-Z/agymc Feedback really appreciated! Have a good day everyone!

mooneater 7y ago
Is this different than VecEnv in baselines? https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py https://github.com/openai/baselines/blob/master/baselines/co...
- MutatedFlood 7y ago
  Thanks for the feedback. OpenAI said they used multiple processes (it seems that they used multiprocessing module from your link). Processes is handled by OS and, it depends, sometimes will only switch to another process on long CPU stalls. I used an event loop to manage that, which supposedly should be fixed those long stalls. Also, this version runs on a single CPU while OpenAI's probably won't, given a multicore CPU. My initial thought was that extending using multiprocessing is much easier than wrapping it using an event loop, hence I only implemented the event loop part.
  - mooneater 7y ago
    Thanks!
wookayin 7y ago
When simulation is CPU-heavy, it will never scale. Though I admit that event-based concurrency might work better in many cases (like where communication overhead is greater than computation), but in some environments you would need real parallelism using many CPU cores. This library also lacks benchmark.
- MutatedFlood 7y ago
  You're right. I plan to add multiprocessing support such that there would be numCPU processes, each running one event loop. Which should scale properly. What kinds of benchmark do you suggest? I'll do my best to provide one. Thanks for the feedback.

12 points by MutatedFlood 7y ago

mooneater 7y ago

Is this different than VecEnv in baselines? https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py https://github.com/openai/baselines/blob/master/baselines/co...

MutatedFlood 7y ago
Thanks for the feedback. OpenAI said they used multiple processes (it seems that they used multiprocessing module from your link). Processes is handled by OS and, it depends, sometimes will only switch to another process on long CPU stalls. I used an event loop to manage that, which supposedly should be fixed those long stalls. Also, this version runs on a single CPU while OpenAI's probably won't, given a multicore CPU. My initial thought was that extending using multiprocessing is much easier than wrapping it using an event loop, hence I only implemented the event loop part.
- mooneater 7y ago
  Thanks!

wookayin 7y ago

When simulation is CPU-heavy, it will never scale. Though I admit that event-based concurrency might work better in many cases (like where communication overhead is greater than computation), but in some environments you would need real parallelism using many CPU cores. This library also lacks benchmark.

MutatedFlood 7y ago
You're right. I plan to add multiprocessing support such that there would be numCPU processes, each running one event loop. Which should scale properly. What kinds of benchmark do you suggest? I'll do my best to provide one. Thanks for the feedback.