3 ms·

> Perhaps the best lesson is this: you just need it to work. On your machine. For now. It doesn’t need to be perfect, just good enough to let you do what you ac

by lmm 2y ago

> Perhaps the best lesson is this: you just need it to work. On your machine. For now. It doesn’t need to be perfect, just good enough to let you do what you actually want to do. And that's how you end up with the kind of trail of flakiness that brought them so much trouble. Disappearing tutorials and deprecated plugins are a result of this kind of attitude. A lot of this honestly sounds like "skill issue" - they're trying to do a cascade of things that is multiple layers of unsupported, and a lot of the problems are ones they've made for themselves (e.g. why not set it up on a local machine first, get your setup steps working there, and then put it on your headless machine, rather than having to deal with Xvnc at the same time as everything else? Why not use a working computer and OS instead of buying a new one?). But also this is what doing real work, especially research, tends to look like. The stuff that is hard is rarely the deep mathematical parts, it's the ops rough edges.

ehnto 2y ago
You might be getting a downvote for trivializing the issues I think, but there is definitely something to be said for stacking the deck against yourself when it comes to debugging. I focus a lot on reducing iteration time to keep devs efficient, and the largest part of that is making sure debugging can happen locally, so you can test your changes or step through the code with a debugger. If your setup doesn't allow that because you have inimitable third party APIs, enterprise data or hosting restrictions for example, you will see a significant cost in development speed, and tricky problems become massive delays. Also it's unpleasant to work so hamstrung by the tools, and unhappy devs leave.
- The_Colonel 2y ago
  > I focus a lot on reducing iteration time to keep devs efficient, and the largest part of that is making sure debugging can happen locally, so you can test your changes or step through the code with a debugger. I think people don't realize how big of a deal this is. I hate that not being able to run and debug your project locally (without crazy mocking) is becoming the new normal these days.
gwern 2y ago
This is especially true when it comes to reinforcement learning: what people should take away from the bloodstained guides written by past RL programmers, it is one thing at a time: https://andyljones.com/posts/rl-debugging.html https://andyljones.com/posts/rl-debugging.html https://clemenswinter.com/2021/03/24/my-reinforcement-learning-learnings/ https://clemenswinter.com/2021/03/24/my-reinforcement-learni... https://blog.nelhage.com/post/systems-that-defy-understanding/ https://blog.nelhage.com/post/systems-that-defy-understandin... Never make multiple changes. Test every case where you have a known answer or oracle, and start with the simplest possible case - because it'll be broken anyway. Log the heck out of everything. Make sure you can visualize the agent's state and inputs. Start with the simplest possible agents like a hardwired sequence of actions and only gradually ramp up the complexity etc. Know your domain very well, so you don't have surprises like the Nethack DRL dev recently discovering a bizarre performance regression was caused by the phase of the moon during testing runs vs training data. etc Jumping in to a complex proprietary video game stack where this purpose is totally unsupported and wanting to apply fancy DRL is a recipe for suffering, and for Heisenbugs, and for interactions, and papercuts.

by lmm 2y ago

ehnto 2y ago

You might be getting a downvote for trivializing the issues I think, but there is definitely something to be said for stacking the deck against yourself when it comes to debugging. I focus a lot on reducing iteration time to keep devs efficient, and the largest part of that is making sure debugging can happen locally, so you can test your changes or step through the code with a debugger. If your setup doesn't allow that because you have inimitable third party APIs, enterprise data or hosting restrictions for example, you will see a significant cost in development speed, and tricky problems become massive delays. Also it's unpleasant to work so hamstrung by the tools, and unhappy devs leave.

The_Colonel 2y ago
> I focus a lot on reducing iteration time to keep devs efficient, and the largest part of that is making sure debugging can happen locally, so you can test your changes or step through the code with a debugger. I think people don't realize how big of a deal this is. I hate that not being able to run and debug your project locally (without crazy mocking) is becoming the new normal these days.

gwern 2y ago

This is especially true when it comes to reinforcement learning: what people should take away from the bloodstained guides written by past RL programmers, it is one thing at a time: https://andyljones.com/posts/rl-debugging.html https://andyljones.com/posts/rl-debugging.html https://clemenswinter.com/2021/03/24/my-reinforcement-learning-learnings/ https://clemenswinter.com/2021/03/24/my-reinforcement-learni... https://blog.nelhage.com/post/systems-that-defy-understanding/ https://blog.nelhage.com/post/systems-that-defy-understandin... Never make multiple changes. Test every case where you have a known answer or oracle, and start with the simplest possible case - because it'll be broken anyway. Log the heck out of everything. Make sure you can visualize the agent's state and inputs. Start with the simplest possible agents like a hardwired sequence of actions and only gradually ramp up the complexity etc. Know your domain very well, so you don't have surprises like the Nethack DRL dev recently discovering a bizarre performance regression was caused by the phase of the moon during testing runs vs training data. etc Jumping in to a complex proprietary video game stack where this purpose is totally unsupported and wanting to apply fancy DRL is a recipe for suffering, and for Heisenbugs, and for interactions, and papercuts.