9 ms·

Reloadr – Hot code reloading tool for Python

175 points by albertzeyer 6y ago

albertzeyer 6y ago
I stumbled upon this because there seems to be a FOSDEM talk which talks about it: https://fosdem.org/2021/schedule/event/python_reloading/ https://fosdem.org/2021/schedule/event/python_reloading/ (https://news.ycombinator.com/item?id=26034851 https://news.ycombinator.com/item?id=26034851) I wonder if this is used seriously by someone. I always thought that this has a potential to greatly improve the development productivity. I remember I saw live coding videos from Notch (Minecraft) on some gamejam where he intensively used hot code reloading for Java. About 90% of his development time, he had the game running, and tuned the logic of some game elements live, and immediately saw the result. I know that Erlang also has such a feature, such that you don't need to shut down your production system, and that you can update it live while it keeps running.
- nerpderp82 6y ago
  This man is a genius.
jorshman 6y ago
What are some simple use cases for this?
- conjectures 6y ago
  The use case is making it take less time to build stuff I imagine.
- brunocvcunha 6y ago
  Not a user of Reloadr, but hot code reloading is a big thing on my Java development process. There are servers and use cases that take a long time to start or setup, and having the possibility to debug -> edit code -> re-execute a specific method using the same context goes a long way. For production usage, maybe that wouldn't be considered safe, but there are companies that provide services with this purpose (e.g. JRebel).
  - nerpderp82 6y ago
    JRebel literally saved our ** years ago. Compile, restart cycles would have killed us, with JRebel we were able to stay in the zone and iterate way way faster. The difference between rage quitting and going home at 430.
- fouric 6y ago
  Well, the main benefit is going to be for complex things where you don't want to restart a large program in order to test a small change, but in terms of simple things, programs with event loops (like games or interactive tools) are a good candidate.
- agumonkey 6y ago
  shorter iteration times when exploring things live
- magicalhippo 6y ago
  I imagine it can be very useful if you want to debug and fix a problem in a contained area of the code where it can take a significant effort to get the program in a state which can trigger the bug, but once in that state it's easy to trigger repeatedly. Say you have to load a lot of data, but once loaded you can trigger the issue by just hitting a button. With hot code reloading you could incrementally add debugging code and try your fix, without having to restart the program and wait for the data to reload each time.
- the_jeremy 6y ago
  I wrote a (worse) version of this at my first job, because loading and initial manipulation of the data I was doing analysis on took tens of minutes, and I kept wanting to make changes to what I was doing. Saved me a couple hours.
Waterluvian 6y ago
Having a hot reloaded set up for both my django (It's built in) environment and Web app environment is such a force multiplier. It's not about simply saving time doing it manually. It fundamentally changes my dev-test/experiment workflow. I love being able to tab to a server file, make a change, tab to a client file, make another change, and instantly test it in a browser.
- speg 6y ago
  Have you tried PHP? :) I jest, but really I feel like we've made things so complicated it just gets in our way. Our Javascript pipeline is huge, complicated, and requires you to sit down and wrap your head around it every time you want to modify it. If I get to greenfield another project, it's going to be so bloody simple.
  - Waterluvian 6y ago
    I'm writing applications for people to manage fleets of thousands of robots in real-time from the other side of the planet. There's no room for "bloody simple" =D They don't resemble traditional websites/CRUD apps a whole lot. Much closer to Sim City + Factorio. When it comes to making your standard issue website/webapp, I think some of them could stand to be simpler.
  - folkrav 6y ago
    > If I get to greenfield another project, it's going to be so bloody simple. It's always simple when it starts up. It's after months/years of iterating without having a decent battle plan that it becomes complicated.
    - OJFord 6y ago
      I agree things grow to be complicated, and that's somewhat unavoidable (accepting usual constraints), but they don't have to start out complex. I think that's the same point GP is making.
      - 6y ago
briggers 6y ago
I used to love doing this with Clojure, it’s an awesome way to increase productivity by a good chunk. It’s less about saving the time to re-run something, and more about removing conceptual overhead (I think).
hikarudo 6y ago
Any pointers on hot code reloading for C++?
- namibj 6y ago
  Check out the Julia interop with C++. In general, though, it's going to be hard.
- maccard 6y ago
  Live++ [0] is an excellent tool, I use it daily. [0] https://liveplusplus.tech/ https://liveplusplus.tech/
- wozer 6y ago
  I think Visual C++ supports it for a subset of the language: https://docs.microsoft.com/de-de/visualstudio/debugger/supported-code-changes-cpp?view=vs-2019 https://docs.microsoft.com/de-de/visualstudio/debugger/suppo...
- jheruty 6y ago
  There is https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledCPlusPlus https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledC... The idea is to use virtual functions, and recompile new classes into a shared library. The shared lib is linked into the running program, and old instances of the classes are deleted and replaced by new instances constructed from the shared library. I am working on a library called hscpp using the same (stolen) idea here: https://github.com/jheruty/hscpp https://github.com/jheruty/hscpp Proof of concept demo: https://m.youtube.com/watch?v=pjGngeKgni8 https://m.youtube.com/watch?v=pjGngeKgni8 hscpp is still very alpha, and I’m sure I’ll find lots of bugs as I work on a “real” demo. In contrast, Runtime Compiled C++ is quite mature and is used in real game projects. Note that this approach very much limits your architecture. For example, you won’t be able to use statics, as the newly compiled shared libraries won’t see them. It’s a finicky thing, worth it to me, but not something you can just plop in to an established project.
fouric 6y ago
I'm slightly confused by this wording in the docs: "This allows it to reload code that is followed by blocking instructions such as the infinite loops you can find in the examples." The examples just have loops where the reloaded function is called in a loop. Is this just updating the definition of the function between calls?
ericol 6y ago
Few years back, I did something similar to this with PHP. I was always marveled at the capacities of xDebug, and never stopped using it since I found it (Some 10+ years), and honestly, coupling it with PHPStorm is another level. But still, hitting reload on the browser, or running a script from the CLI gets tiring after the 20th time; so I made this little class where you would call a method, and the method will reload the script where the code was located continuously until I managed to get it to do what I wanted. It was very Johnny Mnemonic, but it worked.
fabioz 6y ago
Ha, I'm just working on integrating the `pydev.debugger` code reloading into `debugpy` (so that the code reloading from the PyDev debugger will be available for Python-VSCode debugger users too). The in-memory reloading (which may be seen at: https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_bundle/pydevd_reload.py https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_b...) unfortunately doesn't work all the time (fixing the references at runtime in Python is difficult), but it works well enough for the cases where you have a simple function and change it (that file does explain some of the issues that the reloading faces). One major issue is that after doing the reload, you need to get out of the function and then back into it to see the difference -- so, if you're stopped in the function in a breakpoint, it's not possible to simply drop the function and restart it as Python itself doesn't support that... maybe that's a good idea for a PEP ;) Anyways, when it works, it's pretty nice in that you don't have to restart your whole debug session to see changes (the other approach, which webservers use in general is just restart the whole process, so, for instance, Django has an initial process just monitoring for changes and a 2nd process which is actually serving requests which is restarted when something changes -- not as good if you have a heavy startup or are working with something where you have to click through multiple things to get to the state which you want).
jC6fhrfHRLM9b3 6y ago
Here's an alternative way which does not require changing your production code. https://odwyer.software/blog/supercharged-python-testing-workflow https://odwyer.software/blog/supercharged-python-testing-wor...
deleted 6y ago
[deleted]
gamesbrainiac 6y ago
Made a short video on it -> https://twitter.com/gamesbrainiac/status/1357697930258448384?s=20 https://twitter.com/gamesbrainiac/status/1357697930258448384... It's pretty darn neat. I'm going to play around with it a little more over the weekend.
junke 6y ago
Lisp - hot code reloading for Lisp (just kidding)
sitkack 6y ago
For reloading from within the repl from importlib import reload as r Then in the ipython shell you can type r(module) to reload the entire thing. So I'll often have the editor in one terminal window, and ipython in the other one, save the code and switch back the repl, and type r(t) import mymodule as t Now tie in some ipdb [1] and you have a pretty sweet setup. Also depending on your serving framework it might also have hot reloading as well, tornado, flask, etc. [1] https://github.com/gotcha/ipdb https://github.com/gotcha/ipdb [2] https://www.tornadoweb.org/en/stable/autoreload.html https://www.tornadoweb.org/en/stable/autoreload.html https://werkzeug.palletsprojects.com/en/0.14.x/serving/ https://werkzeug.palletsprojects.com/en/0.14.x/serving/
- rcfox 6y ago
  Why not just use ipython's built-in autoreload? https://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html https://ipython.readthedocs.io/en/stable/config/extensions/a...
  - sitkack 6y ago
    This is pretty neat.
neves 6y ago
I use IPython that already have this feature. Just type in a prompt: %load_ext autoreload %autoreload 2 now every run automatically reload your modules.
satya71 6y ago
Last commit in Jan 2020. Is this project still active? Also ideally, not needing to decorate would be nice.
- breuleux 6y ago
  For what it's worth, I'm working on my own reloader which does not require decorating anything: https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged (still a WIP, but it works well enough).
  - satya71 6y ago
    Thanks for the good work. I’ll give it a try.
breuleux 6y ago
That's a bit awkward, I'm currently working on a package[1] that does more or less the same thing in more or less the same way (I think), except you don't need to decorate anything. Just run `python -m jurigged script.py` or `jurigged script.py` instead of `python script.py` and it will watch everything in the current working directory. There's still some bugs I have to iron out, though, it should be better in a month or so. Still, you can pip install it if you want. [1] https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged
- ArchOversight 6y ago
  Have you looked at `hupper` from the Pylons Project? The author posted here: https://news.ycombinator.com/item?id=26039366 https://news.ycombinator.com/item?id=26039366
  - breuleux 6y ago
    Seems fairly different. My tool does not restart the process, it hot patches functions and methods whenever they are modified in the source, like the OP.
  - tbt43p 6y ago
    why is such feature of reloading needed in the first place? I am a beginner thanks
    - vermilingua 6y ago
      Accellerates the dev process by removing a whole bunch of steps: terminating the running program, relaunching the running program, repeating any steps to get back to the code in question. While these things only take maybe a few seconds each, thats a few seconds that you’ll perform maybe hundreds of times. Reducing context switches is always a good thing.
    - okso 6y ago
      albertzeyer's example with Notch explains the issue pretty well: When working on a video game, restarting the game every time you are tweaking the behaviour of an entity takes a lot of time. Hot code reloading while keeping the state of the game makes development much easier. On the other hand, when the entire state is stored outside of the Python app (eg: a web app using a database), restarting the entire process and reloading the page is fast enough and more reliable.
sllabres 6y ago
Cool feature. Visual Studio seems to have "edit and continue" which I only knew from VB, but seems to be available in C# and C++ too. Not the same, but it shortens the development cycle in a similar fashion. The feature in Python reminds me on hot code loading in Erlang which I found quite interesting when i first read (never used) about the feature. I wondered which other languages provides (or didn't) an equivalent solution. Back then it felt a bit scary, with not much more than RCS as VCS around :) But I think Erlang implemented it for increased up-time, not for increased development cycle? To keep the up-time high, today even parts or entire operating systems are patched 'live' (ksplice, kpatch, kgraft, AIX live update) (https://ksplice.oracle.com/ https://ksplice.oracle.com/) (https://www.ibm.com/support/knowledgecenter/ssw_aix_72/install/live_update_install.html https://www.ibm.com/support/knowledgecenter/ssw_aix_72/insta...). Here too to prevent downtime and technologically completely different from what is presented here.
- sgh1 6y ago
  I've never had any success at all with VS's edit and continue, even after tinkering some project settings to try to get it to work better. To be fair, this was a few years ago now, maybe worth another look?
- Too 6y ago
  Edit and continue in C# is like magic. Paired with ability to move the execution-pointer or unwind the stack and rerun the function you can write a whole algorithm without ever stopping the debugger. Oops - something went wrong, just change the code, reassign the variable in the watch-window and drag up the execution pointer to previous line. Really powerful! If only more environments had similar tools. Once you've worked with this other tools feel rather primitive.
bambam3000 6y ago
does this patch existing class instances, if you modify a method for example? I cobbled together something similar a few years back using the reload stuff from ipython and a script I think GuidoVR wrote that I stumbled on. I like to debug on the cli (pdb++), I made it autodetect changed modules and reload those whenever i entered the 'reload' command. For me having to decorate things first would be a no-no. My helper worked really nicely for some projects but broke with others. Well done on getting something out there.
mmerickel 6y ago
My project, hupper [1], isn't hot reloading but it's worth mentioning in this thread alongside other code reloaders. It is a general purpose tool to monitor python code and reload the process when code changes. You can use it with any script that you're working on via `hupper -m foo` and your `foo.py` is then reloaded/re-executed everytime you change it. It's used under the hood in other tools to implement things like `pserve --reload` in Pyramid and is used in Warehouse (the codebase for PyPI) to ease development workflows. It's an improvement on some previous attempts at the solution that would crash when reloading invalid code. Hupper does not have this problem due to its multi process design. [1] https://docs.pylonsproject.org/projects/hupper/en/latest/ https://docs.pylonsproject.org/projects/hupper/en/latest/
williamsmj 6y ago
You can get this behavior in ipython and any other jupyter tool (notebook, lab) without adding a python dependency by putting this: %load_ext autoreload %autoreload 2 in ~/.ipython/profile_default/startup/autoreload.ipy
teajunky 6y ago
I am developing my own reloader in spirit of jrebel and how hotswapping works in javascript bundlers. While investigating the topic I found a lot of projects: - https://github.com/yiblet/inquest https://github.com/yiblet/inquest - https://github.com/mpolney/hotpatch https://github.com/mpolney/hotpatch - https://pypi.org/project/patchy/ https://pypi.org/project/patchy/ - https://pypi.org/project/plone.reload/ https://pypi.org/project/plone.reload/ - http://svn.python.org/projects/sandbox/trunk/xreload/ http://svn.python.org/projects/sandbox/trunk/xreload/ - https://pypi.org/project/hot_reload/ https://pypi.org/project/hot_reload/ - https://pypi.org/project/Reloadr/ https://pypi.org/project/Reloadr/ - https://pypi.org/project/hotswap/ https://pypi.org/project/hotswap/ - https://pypi.org/project/reloading/ https://pypi.org/project/reloading/ - https://pypi.org/project/hotreload/ https://pypi.org/project/hotreload/ - https://pypi.org/project/livereload/ https://pypi.org/project/livereload/ - https://pypi.org/project/reimport/ https://pypi.org/project/reimport/ - https://pypi.org/project/livecoding/ https://pypi.org/project/livecoding/ - https://pypi.org/project/pylivecoding/ https://pypi.org/project/pylivecoding/ - https://pypi.org/project/nameko-hot-reload/ https://pypi.org/project/nameko-hot-reload/ - https://github.com/albertz/PythonHotswap https://github.com/albertz/PythonHotswap - https://github.com/mikr/pyqtboiler/blob/master/baseapp/utils/hotswap.py https://github.com/mikr/pyqtboiler/blob/master/baseapp/utils... - https://github.com/narfdotpl/hottie https://github.com/narfdotpl/hottie with decorator - https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f16318a4ceca0ac1c3688697/IPython/extensions/autoreload.py https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f163... - https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged I thing that xreload from Guido and the code inside ipython both provide the best hot-patching. But both projects miss the feature to watch for changed files.
julvo 6y ago
Wrote a similar thing [1]. It can also load loop bodies from source before each iteration. Can be helpful for long running loops like when downloading or training deep learning models. [1] https://github.com/julvo/reloading https://github.com/julvo/reloading
huijzer 6y ago
In Julia, this can be done with even less efforts on the user-side by loading Revise (https://github.com/timholy/Revise.jl https://github.com/timholy/Revise.jl) before any other package you load.
okso 6y ago
Author here, thank you for all this feedback . It's great to see I am not the only one interested in this way of coding. I will have a look at all the alternatives your posted about, let's get hot code reloading in Python awesome! Join the conversation on FOSDEM's Matrix channel tomorrow to discuss the subject further: #python-python_reloading:fosdem.org

175 points by albertzeyer 6y ago

I stumbled upon this because there seems to be a FOSDEM talk which talks about it: https://fosdem.org/2021/schedule/event/python_reloading/ https://fosdem.org/2021/schedule/event/python_reloading/ (https://news.ycombinator.com/item?id=26034851 https://news.ycombinator.com/item?id=26034851) I wonder if this is used seriously by someone. I always thought that this has a potential to greatly improve the development productivity. I remember I saw live coding videos from Notch (Minecraft) on some gamejam where he intensively used hot code reloading for Java. About 90% of his development time, he had the game running, and tuned the logic of some game elements live, and immediately saw the result. I know that Erlang also has such a feature, such that you don't need to shut down your production system, and that you can update it live while it keeps running.

nerpderp82 6y ago
This man is a genius.

jorshman 6y ago

What are some simple use cases for this?

conjectures 6y ago
The use case is making it take less time to build stuff I imagine.
brunocvcunha 6y ago
Not a user of Reloadr, but hot code reloading is a big thing on my Java development process. There are servers and use cases that take a long time to start or setup, and having the possibility to debug -> edit code -> re-execute a specific method using the same context goes a long way. For production usage, maybe that wouldn't be considered safe, but there are companies that provide services with this purpose (e.g. JRebel).
- nerpderp82 6y ago
  JRebel literally saved our ** years ago. Compile, restart cycles would have killed us, with JRebel we were able to stay in the zone and iterate way way faster. The difference between rage quitting and going home at 430.
fouric 6y ago
Well, the main benefit is going to be for complex things where you don't want to restart a large program in order to test a small change, but in terms of simple things, programs with event loops (like games or interactive tools) are a good candidate.
agumonkey 6y ago
shorter iteration times when exploring things live
magicalhippo 6y ago
I imagine it can be very useful if you want to debug and fix a problem in a contained area of the code where it can take a significant effort to get the program in a state which can trigger the bug, but once in that state it's easy to trigger repeatedly. Say you have to load a lot of data, but once loaded you can trigger the issue by just hitting a button. With hot code reloading you could incrementally add debugging code and try your fix, without having to restart the program and wait for the data to reload each time.
the_jeremy 6y ago
I wrote a (worse) version of this at my first job, because loading and initial manipulation of the data I was doing analysis on took tens of minutes, and I kept wanting to make changes to what I was doing. Saved me a couple hours.

Waterluvian 6y ago

Having a hot reloaded set up for both my django (It's built in) environment and Web app environment is such a force multiplier. It's not about simply saving time doing it manually. It fundamentally changes my dev-test/experiment workflow. I love being able to tab to a server file, make a change, tab to a client file, make another change, and instantly test it in a browser.

speg 6y ago
Have you tried PHP? :) I jest, but really I feel like we've made things so complicated it just gets in our way. Our Javascript pipeline is huge, complicated, and requires you to sit down and wrap your head around it every time you want to modify it. If I get to greenfield another project, it's going to be so bloody simple.
- Waterluvian 6y ago
  I'm writing applications for people to manage fleets of thousands of robots in real-time from the other side of the planet. There's no room for "bloody simple" =D They don't resemble traditional websites/CRUD apps a whole lot. Much closer to Sim City + Factorio. When it comes to making your standard issue website/webapp, I think some of them could stand to be simpler.
- folkrav 6y ago
  > If I get to greenfield another project, it's going to be so bloody simple. It's always simple when it starts up. It's after months/years of iterating without having a decent battle plan that it becomes complicated.
  - OJFord 6y ago
    I agree things grow to be complicated, and that's somewhat unavoidable (accepting usual constraints), but they don't have to start out complex. I think that's the same point GP is making.
    - 6y ago

briggers 6y ago

I used to love doing this with Clojure, it’s an awesome way to increase productivity by a good chunk. It’s less about saving the time to re-run something, and more about removing conceptual overhead (I think).

hikarudo 6y ago

Any pointers on hot code reloading for C++?

namibj 6y ago
Check out the Julia interop with C++. In general, though, it's going to be hard.
maccard 6y ago
Live++ [0] is an excellent tool, I use it daily. [0] https://liveplusplus.tech/ https://liveplusplus.tech/
wozer 6y ago
I think Visual C++ supports it for a subset of the language: https://docs.microsoft.com/de-de/visualstudio/debugger/supported-code-changes-cpp?view=vs-2019 https://docs.microsoft.com/de-de/visualstudio/debugger/suppo...
jheruty 6y ago
There is https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledCPlusPlus https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledC... The idea is to use virtual functions, and recompile new classes into a shared library. The shared lib is linked into the running program, and old instances of the classes are deleted and replaced by new instances constructed from the shared library. I am working on a library called hscpp using the same (stolen) idea here: https://github.com/jheruty/hscpp https://github.com/jheruty/hscpp Proof of concept demo: https://m.youtube.com/watch?v=pjGngeKgni8 https://m.youtube.com/watch?v=pjGngeKgni8 hscpp is still very alpha, and I’m sure I’ll find lots of bugs as I work on a “real” demo. In contrast, Runtime Compiled C++ is quite mature and is used in real game projects. Note that this approach very much limits your architecture. For example, you won’t be able to use statics, as the newly compiled shared libraries won’t see them. It’s a finicky thing, worth it to me, but not something you can just plop in to an established project.

fouric 6y ago

I'm slightly confused by this wording in the docs: "This allows it to reload code that is followed by blocking instructions such as the infinite loops you can find in the examples." The examples just have loops where the reloaded function is called in a loop. Is this just updating the definition of the function between calls?

ericol 6y ago

Few years back, I did something similar to this with PHP. I was always marveled at the capacities of xDebug, and never stopped using it since I found it (Some 10+ years), and honestly, coupling it with PHPStorm is another level. But still, hitting reload on the browser, or running a script from the CLI gets tiring after the 20th time; so I made this little class where you would call a method, and the method will reload the script where the code was located continuously until I managed to get it to do what I wanted. It was very Johnny Mnemonic, but it worked.

fabioz 6y ago

Ha, I'm just working on integrating the `pydev.debugger` code reloading into `debugpy` (so that the code reloading from the PyDev debugger will be available for Python-VSCode debugger users too). The in-memory reloading (which may be seen at: https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_bundle/pydevd_reload.py https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_b...) unfortunately doesn't work all the time (fixing the references at runtime in Python is difficult), but it works well enough for the cases where you have a simple function and change it (that file does explain some of the issues that the reloading faces). One major issue is that after doing the reload, you need to get out of the function and then back into it to see the difference -- so, if you're stopped in the function in a breakpoint, it's not possible to simply drop the function and restart it as Python itself doesn't support that... maybe that's a good idea for a PEP ;) Anyways, when it works, it's pretty nice in that you don't have to restart your whole debug session to see changes (the other approach, which webservers use in general is just restart the whole process, so, for instance, Django has an initial process just monitoring for changes and a 2nd process which is actually serving requests which is restarted when something changes -- not as good if you have a heavy startup or are working with something where you have to click through multiple things to get to the state which you want).

jC6fhrfHRLM9b3 6y ago

Here's an alternative way which does not require changing your production code. https://odwyer.software/blog/supercharged-python-testing-workflow https://odwyer.software/blog/supercharged-python-testing-wor...

deleted 6y ago

[deleted]

gamesbrainiac 6y ago

Made a short video on it -> https://twitter.com/gamesbrainiac/status/1357697930258448384?s=20 https://twitter.com/gamesbrainiac/status/1357697930258448384... It's pretty darn neat. I'm going to play around with it a little more over the weekend.

junke 6y ago

Lisp - hot code reloading for Lisp (just kidding)

sitkack 6y ago

For reloading from within the repl from importlib import reload as r Then in the ipython shell you can type r(module) to reload the entire thing. So I'll often have the editor in one terminal window, and ipython in the other one, save the code and switch back the repl, and type r(t) import mymodule as t Now tie in some ipdb [1] and you have a pretty sweet setup. Also depending on your serving framework it might also have hot reloading as well, tornado, flask, etc. [1] https://github.com/gotcha/ipdb https://github.com/gotcha/ipdb [2] https://www.tornadoweb.org/en/stable/autoreload.html https://www.tornadoweb.org/en/stable/autoreload.html https://werkzeug.palletsprojects.com/en/0.14.x/serving/ https://werkzeug.palletsprojects.com/en/0.14.x/serving/

rcfox 6y ago
Why not just use ipython's built-in autoreload? https://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html https://ipython.readthedocs.io/en/stable/config/extensions/a...
- sitkack 6y ago
  This is pretty neat.

neves 6y ago

I use IPython that already have this feature. Just type in a prompt: %load_ext autoreload %autoreload 2 now every run automatically reload your modules.

satya71 6y ago

Last commit in Jan 2020. Is this project still active? Also ideally, not needing to decorate would be nice.

breuleux 6y ago
For what it's worth, I'm working on my own reloader which does not require decorating anything: https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged (still a WIP, but it works well enough).
- satya71 6y ago
  Thanks for the good work. I’ll give it a try.

breuleux 6y ago

That's a bit awkward, I'm currently working on a package[1] that does more or less the same thing in more or less the same way (I think), except you don't need to decorate anything. Just run `python -m jurigged script.py` or `jurigged script.py` instead of `python script.py` and it will watch everything in the current working directory. There's still some bugs I have to iron out, though, it should be better in a month or so. Still, you can pip install it if you want. [1] https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged

ArchOversight 6y ago
Have you looked at `hupper` from the Pylons Project? The author posted here: https://news.ycombinator.com/item?id=26039366 https://news.ycombinator.com/item?id=26039366
- breuleux 6y ago
  Seems fairly different. My tool does not restart the process, it hot patches functions and methods whenever they are modified in the source, like the OP.
- tbt43p 6y ago
  why is such feature of reloading needed in the first place? I am a beginner thanks
  - vermilingua 6y ago
    Accellerates the dev process by removing a whole bunch of steps: terminating the running program, relaunching the running program, repeating any steps to get back to the code in question. While these things only take maybe a few seconds each, thats a few seconds that you’ll perform maybe hundreds of times. Reducing context switches is always a good thing.
  - okso 6y ago
    albertzeyer's example with Notch explains the issue pretty well: When working on a video game, restarting the game every time you are tweaking the behaviour of an entity takes a lot of time. Hot code reloading while keeping the state of the game makes development much easier. On the other hand, when the entire state is stored outside of the Python app (eg: a web app using a database), restarting the entire process and reloading the page is fast enough and more reliable.

sllabres 6y ago

Cool feature. Visual Studio seems to have "edit and continue" which I only knew from VB, but seems to be available in C# and C++ too. Not the same, but it shortens the development cycle in a similar fashion. The feature in Python reminds me on hot code loading in Erlang which I found quite interesting when i first read (never used) about the feature. I wondered which other languages provides (or didn't) an equivalent solution. Back then it felt a bit scary, with not much more than RCS as VCS around :) But I think Erlang implemented it for increased up-time, not for increased development cycle? To keep the up-time high, today even parts or entire operating systems are patched 'live' (ksplice, kpatch, kgraft, AIX live update) (https://ksplice.oracle.com/ https://ksplice.oracle.com/) (https://www.ibm.com/support/knowledgecenter/ssw_aix_72/install/live_update_install.html https://www.ibm.com/support/knowledgecenter/ssw_aix_72/insta...). Here too to prevent downtime and technologically completely different from what is presented here.

sgh1 6y ago
I've never had any success at all with VS's edit and continue, even after tinkering some project settings to try to get it to work better. To be fair, this was a few years ago now, maybe worth another look?
Too 6y ago
Edit and continue in C# is like magic. Paired with ability to move the execution-pointer or unwind the stack and rerun the function you can write a whole algorithm without ever stopping the debugger. Oops - something went wrong, just change the code, reassign the variable in the watch-window and drag up the execution pointer to previous line. Really powerful! If only more environments had similar tools. Once you've worked with this other tools feel rather primitive.

bambam3000 6y ago

does this patch existing class instances, if you modify a method for example? I cobbled together something similar a few years back using the reload stuff from ipython and a script I think GuidoVR wrote that I stumbled on. I like to debug on the cli (pdb++), I made it autodetect changed modules and reload those whenever i entered the 'reload' command. For me having to decorate things first would be a no-no. My helper worked really nicely for some projects but broke with others. Well done on getting something out there.

mmerickel 6y ago

My project, hupper [1], isn't hot reloading but it's worth mentioning in this thread alongside other code reloaders. It is a general purpose tool to monitor python code and reload the process when code changes. You can use it with any script that you're working on via `hupper -m foo` and your `foo.py` is then reloaded/re-executed everytime you change it. It's used under the hood in other tools to implement things like `pserve --reload` in Pyramid and is used in Warehouse (the codebase for PyPI) to ease development workflows. It's an improvement on some previous attempts at the solution that would crash when reloading invalid code. Hupper does not have this problem due to its multi process design. [1] https://docs.pylonsproject.org/projects/hupper/en/latest/ https://docs.pylonsproject.org/projects/hupper/en/latest/

williamsmj 6y ago

You can get this behavior in ipython and any other jupyter tool (notebook, lab) without adding a python dependency by putting this: %load_ext autoreload %autoreload 2 in ~/.ipython/profile_default/startup/autoreload.ipy

teajunky 6y ago

I am developing my own reloader in spirit of jrebel and how hotswapping works in javascript bundlers. While investigating the topic I found a lot of projects: - https://github.com/yiblet/inquest https://github.com/yiblet/inquest - https://github.com/mpolney/hotpatch https://github.com/mpolney/hotpatch - https://pypi.org/project/patchy/ https://pypi.org/project/patchy/ - https://pypi.org/project/plone.reload/ https://pypi.org/project/plone.reload/ - http://svn.python.org/projects/sandbox/trunk/xreload/ http://svn.python.org/projects/sandbox/trunk/xreload/ - https://pypi.org/project/hot_reload/ https://pypi.org/project/hot_reload/ - https://pypi.org/project/Reloadr/ https://pypi.org/project/Reloadr/ - https://pypi.org/project/hotswap/ https://pypi.org/project/hotswap/ - https://pypi.org/project/reloading/ https://pypi.org/project/reloading/ - https://pypi.org/project/hotreload/ https://pypi.org/project/hotreload/ - https://pypi.org/project/livereload/ https://pypi.org/project/livereload/ - https://pypi.org/project/reimport/ https://pypi.org/project/reimport/ - https://pypi.org/project/livecoding/ https://pypi.org/project/livecoding/ - https://pypi.org/project/pylivecoding/ https://pypi.org/project/pylivecoding/ - https://pypi.org/project/nameko-hot-reload/ https://pypi.org/project/nameko-hot-reload/ - https://github.com/albertz/PythonHotswap https://github.com/albertz/PythonHotswap - https://github.com/mikr/pyqtboiler/blob/master/baseapp/utils/hotswap.py https://github.com/mikr/pyqtboiler/blob/master/baseapp/utils... - https://github.com/narfdotpl/hottie https://github.com/narfdotpl/hottie with decorator - https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f16318a4ceca0ac1c3688697/IPython/extensions/autoreload.py https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f163... - https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged I thing that xreload from Guido and the code inside ipython both provide the best hot-patching. But both projects miss the feature to watch for changed files.

julvo 6y ago

Wrote a similar thing [1]. It can also load loop bodies from source before each iteration. Can be helpful for long running loops like when downloading or training deep learning models. [1] https://github.com/julvo/reloading https://github.com/julvo/reloading

huijzer 6y ago

In Julia, this can be done with even less efforts on the user-side by loading Revise (https://github.com/timholy/Revise.jl https://github.com/timholy/Revise.jl) before any other package you load.

okso 6y ago

Author here, thank you for all this feedback . It's great to see I am not the only one interested in this way of coding. I will have a look at all the alternatives your posted about, let's get hot code reloading in Python awesome! Join the conversation on FOSDEM's Matrix channel tomorrow to discuss the subject further: #python-python_reloading:fosdem.org