I think hot code reloading can be very effective. This extremely reduces the feedback loop, which can be very important for productivity. Unfortunately, I rare

I think hot code reloading can be very effective. This extremely reduces the feedback loop, which can be very important for productivity. Unfortunately, I rarely was able to make this work in my setups, except long time ago when I used Visual Basic. Then I worked with C++, mostly game development using SDL. The last couple of years I wrote Python, mostly for machine learning with Theano and then TensorFlow. The startup time of an initial prototype might still be fast enough (couple of seconds) that you don't care too much and you are still productive. But as complexity adds up, and you add other slowdowns like slow NFS or whatever, this can reach minutes, and this already is annoying. But most development I do would probably not really allow for hot reloading, or I don't really know a good way. This is anyway not really common practice for Python, and whatever you try in that direction will probably be unstable and/or non-Pythonic. And then, you have deep learning research itself, where your feedback loop is at minimum one day, but often more like a week. I remember that I saw screencast of notch developing on some games, where he intensively used hot code reloading on some functions which defined the behavior or look of some game entities. This looked like an extremely productive and powerful tool. Just within half a second or so, he saw the result of his code change. He just played a lot around until the behavior was nice.