3 ms·
I tried to learn deep Q-learning recently using OpenAI gym. I looked at the "leader boards" and tried to learn from their code, but I wasn't getting results nea
by Buttons840 7y ago
I tried to learn deep Q-learning recently using OpenAI gym. I looked at the "leader boards" and tried to learn from their code, but I wasn't getting results nearly as good as the "leaders".
I eventually checked out the leaders code and ran it myself, but removed their carefully selected random seeds, and found that the supposed leading solutions often failed to converge at all without their magically selected random seed.
I left the experience believing deep RL is still very unreliable.
- yd218 7y agoYou may be interested in our new Behaviour Suite for Reinforcement Learning project at https://github.com/deepmind/bsuite https://github.com/deepmind/bsuite. This is an attempt at defining a set of minimal but meaningful benchmark RL experiments. In each experiment, we typically evaluate over a range of environment parameters and compute an aggregate score. See https://arxiv.org/abs/1908.03568 https://arxiv.org/abs/1908.03568 for more details. We also include a simple starter RL codebase, and examples of evaluating agents defined in other RL frameworks.
- Buttons840 7y agoI am interested, thanks.
- maxc01 7y agoactually the seed is also a hyper-parameter
- thrax 7y agoI will contrast with my anecdote.. I build a simulation of 10x10 cityblocks with a population of 30 pedestrians.. I made a reward function based on distance to a random "target point" with penalty for walking in the street vs sidewalk.. and penalty for walking into walls.. the inputs were a 360 degree raycast of 16 samples.. and "distance to target" and the outputs were awsd keyboard inputs. I left it running overnight and by morning, bots were pretty efficiently walking around the city to their random targets via sidewalks. It felt like magic. I could have coded the behavior directly but the learned version seemed somewhat noisier and more organic. This was done a couple years ago using the a JavaScript deepq learning library. It feels like a squishier version of a-star or something.
- basman 7y agoThe raycast probably disambiguated the state pretty well, such that it essentially had to memorize a few hundred actions, so that it did end up basically doing a sort of asynchronous distributed Dijkstra's algorithm.