12 ms·

Hot-swapping Python 3 code

120 points by metahost 6y ago

pronoyc 6y ago
How is this different from using Python Watchdog? https://pythonhosted.org/watchdog/ https://pythonhosted.org/watchdog/
- erikschoster 6y ago
  Looks like it starts a thread that repeatedly hashes the content of the target script to poll for changes. Watchdog uses OS features like inotify and FSEvents and only uses polling and comparing hashes as a fallback.
sillysaurusx 6y ago
Does this "just work"? How does it handle classes? I've been writing my python code in a style that can be hotswapped. Roughly, I use global functions and pass around structs, like C. The function calls are made with `api.foo(data)` instead of `data.foo()`. Then I can just replace `api.foo`'s definition with whatever I want after attaching with a debugger. Example: https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d6cfdc3192aba0f2b4/train_multi.py#L475-L499 https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d... I assume there's a much better way to do this, but the class system seems to make it hard to swap things out. Doesn't "isinstance" break if you redefine the class? EDIT: Oh, this isn't actual hot-reloading for python. Darn. I'm really hoping someone will tackle that... Cool project though! EDIT2: Actually, this project does seem to reload the modules: https://github.com/say4n/hotreload/blob/0cf3a0b466466f99f8c7ccdb546fcef4553617f9/hotreload/reloader.py#L47-L48 https://github.com/say4n/hotreload/blob/0cf3a0b466466f99f8c7... ... so the original question stands. By the way, you may want to use `import traceback; traceback.print_exc()` to log exceptions. I'm not sure how that plays with python's `logger` class though.
- oefrha 6y ago
  importlib.reload does not "just work". https://docs.python.org/3/library/importlib.html#importlib.reload https://docs.python.org/3/library/importlib.html#importlib.r... documents what is done and not done. > Doesn't "isinstance" break if you redefine the class? With importlib.reload, yes. You might be interested in IPython's autoreload implementation, which does not break isinstance: https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f16318a4ceca0ac1c3688697/IPython/extensions/autoreload.py#L253-L417 https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f163... However, you could still end up with semi-broken instances, for instance if you update class A: pass to class A: def __init__(self): self.attr = "something" def __str__(self): return self.attr > By the way, you may want to use `import traceback; traceback.print_exc()` to log exceptions. I'm not sure how that plays with python's `logger` class though. PSL logging has builtin integration with exception tracebacks. See Logger.exception [1] or the exc_info keyword param to Logger.debug/info/warning/error/critical/log [2]. [1] https://docs.python.org/3/library/logging.html#logging.Logger.exception https://docs.python.org/3/library/logging.html#logging.Logge... [2] https://docs.python.org/3/library/logging.html#logging.Logger.debug https://docs.python.org/3/library/logging.html#logging.Logge...
  - aidos 6y ago
    I couldn’t live without ipython autoreload. You need to know where the limitations are that require you to create your instances again. Things like sqlalchemy model changes run into issues because of the meta class magic (I think, not looked too deeply). I feel like this library is a bit misrenamed. It seems to be more like “autorun”. Edit: here’s the ipython config I use to turn it on https://github.com/aidos/dotfiles/blob/master/_config/ipython/profile_default/ipython_config.py https://github.com/aidos/dotfiles/blob/master/_config/ipytho...
- viraptor 6y ago
  Using logger.exception("message") would also log the trace by default.
oefrha 6y ago
For comparison, here’s Werkzeug’s reloader implementation: https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_reloader.py https://github.com/pallets/werkzeug/blob/master/src/werkzeug... (when you flask run or call app.run with reloader turned on, it’s handled by this module.)
- sillysaurusx 6y ago
  Thanks! That `_iter_module_files()` function is handy. Unfortunately, it looks like it simply restarts Python each time the code changes: https://github.com/pallets/werkzeug/blob/75215972967c1c00c15b8b5c3c661c2278423360/src/werkzeug/_reloader.py#L156 https://github.com/pallets/werkzeug/blob/75215972967c1c00c15... which isn't quite the same thing as reloading it.
  - the_mitsuhiko 6y ago
    It used to do hot reloading back in the days but unfortunately that doesn’t really work in Python because of all the import side effects. It’s better to restart everything. Werkzeug keeps the socket open so you don’t observe the reload from the outside.
  - oefrha 6y ago
    Actual reloading is just hairy and you can end up with semi-broken instances among other things. Restarting the interpreter is just cleaner, and it's totally acceptable for a dev server. (Okay looks like Armin himself replied too, obviously take his words above mine.) I pointed to IPython's autoreload in another reply which is more powerful than importlib.reload.
  - raziel2p 6y ago
    Why is that unfortunate?
sfgweilr4f 6y ago
Not keen on hashing the file every second but I understand why... I'd rather it used filesystem modification events and responded accordingly. But that might more more complex than what you intend. But for simple scripts being live-coded that need to be quickly "up" and hot-swapped on rapid modification this looks fine. I could always increase the delay to a minute or hour or more if the changes were only occasionally changed.
hathym 6y ago
having an infinite loop reading the file and calculating its sha checksum every 1s is not ideal. I'd suggest using the OS premitives to get notifications when the file is touched (inotify on linux and ReadDirectoryChangesW on windows), or simply use the watchdog lib
deleted 6y ago
[deleted]
BiteCode_dev 6y ago
Because of the nature of Python, using importlib.reload() is going to lead to terrible bugs at some point. Use importlib.reload() only in the shell, for convenience. If you want autoreload for dev (e.g: like django dev server), better restart the entire process. There is a 7 years old project that can help you with that called watchdog: https://pythonhosted.org/watchdog/ It's a library that uses the fastest way available (e.g: inotify,kqueue...) to react to file changes (you can filter by types such as create, delete, write, etc) and triggers whatever code you want, passing you the metadata. If you just want something very simple, you don't have to code, you can install watchdog with its optional watchmedo dependancy: pip install watchdog[watchmedo] It will give you the watchmedo command, that lets you do something like this: watchmedo shell-command --patterns="*.py" --recursive \ --command='python your_script_to_rerun.py' .
- stock_toaster 6y ago
  entr[1] is another good option. [1]: http://eradman.com/entrproject/ http://eradman.com/entrproject/
  - BiteCode_dev 6y ago
    Yes but unfortunatly, you have to compile it for many systems yourself. And if you want to use it from Python (not from the command line), you need binding. On the other hand, it's probably more performant than watchdog if you have a lot of files and need to fallback on manual watching. Plus, if you are not a Python person, you may not want to use pip.
- X-Istence 6y ago
  If you are working on code that runs a web server or some other service that is generally a long running application, take a look at hupper: https://docs.pylonsproject.org/projects/hupper/en/latest/ https://docs.pylonsproject.org/projects/hupper/en/latest/ It is built specifically to restart things like servers as it was implemented for the Pylons Project (Pyramid web framework).
  - 6y ago
orf 6y ago
Hot reloading is next to impossible to do safely in the general case in Python, and this library is really basic and doesn’t really do anything special. The only case where it does work is if you maintain a single reference to the module you wish to reload, _and_ the target module doesn’t load any C extensions, _and_ it doesn’t create state outside its module (very easy to do accidentally). That’s why every autoreloader implementation restarts the whole process when any change is made, because how would you even detect that it was safe to hot reload something in the general case let alone actually hot reload it safely? If you need to add an autoreloader to your project use Hupper[1]. If you’re weird like me and you’re interested in autoreloaders, check out the Django[2] and Flask[3] implementations or my talk[4] for more details. 1. https://pypi.org/project/hupper/ https://pypi.org/project/hupper/ 2. https://github.com/django/django/blob/master/django/utils/autoreload.py https://github.com/django/django/blob/master/django/utils/au... 3. https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_reloader.py https://github.com/pallets/werkzeug/blob/master/src/werkzeug... 4. https://youtu.be/IghyoR6ld60 https://youtu.be/IghyoR6ld60
- lunixbochs 6y ago
  I got it working well in a large project but I basically designed auto reloading into deep parts of the framework. It's not something you can safely bolt on, and I'm honestly not sure I recommend the sanity impact of trying to build it properly (e.g. ran into a bunch of memory leaking even once I had it working safely). But there are some kinds of programs you can't really restart on every change. The biggest caveat of these kinds of single file reloaders is they completely fall apart when you import between your modules. For example, if `a` does `from b import obj` and `b` does `from c import obj`, and you modify `c.py`, `a` and `b` will both have a stale reference to obj (which might be a primitive or even interned type like a string or an int, so it's not like you can mutate it as a solution).
  - orf 6y ago
    I updated my comment to say “impossible in the general case” which is more accurate, but I’d be interested in knowing more about that project - how big was it, and how did handle some of the tricky bits? Like, module A creates a class/function that is consumed by module B and C. If you change module A, how did you know to also reload B and C (recursively)?
david_draco 6y ago
You can do something like this on Linux: inotifywait -m myfile.py -e 'CLOSE_WRITE' | while read line; do python3 myfile.py; done It will be more responsive than a busy while-sleep loop.
jbverschoor 6y ago
Sorry, but that's not really hot-reloading. It's only reloading a file. Why do we only get hot-code-replacement and edit-code-and-continue-debugging from microsoft and sun(oracle)?
- julvo 6y ago
  Have a look at https://github.com/julvo/reloading https://github.com/julvo/reloading, it's a little library I wrote that can give you edit-code-and-continue-debugging in Python loops. A more general solution for this sort of development in Python would be neat though.
  - jbverschoor 6y ago
    So this should be posted instead of the OP
underdeserver 6y ago
For those using Bazel, there's ibazel: https://github.com/bazelbuild/bazel-watcher https://github.com/bazelbuild/bazel-watcher It's great, I use it all the time, especially with unit tests.
brodul 6y ago
As other have pointed out, the best way is to restart the process. If you are looking for hot reloading you should look at programming languages that run in BEAM like Erlang and Elixir[1]. There are some gotchas with hot swapping, but it's designed and used in production for many years [2]. If somebody knows other languages/systems that do let me know. 1. https://elixir-lang.org/ https://elixir-lang.org/ 2. https://stackoverflow.com/questions/37368376/how-does-erlang-hot-code-swapping-work-in-the-middle-of-activity https://stackoverflow.com/questions/37368376/how-does-erlang...
- dig1 6y ago
  > If somebody knows other languages/systems that do let me know. Clojure can do this via namespace live reload. ClojureScript is already doing this inside figwheel [1] environment. CommonLisp had this since dawn of time. Kawa [2] can do it, but because it aggressively optimize the code, you need to be careful. Racket can do it as well [3]. [1] https://github.com/bhauman/lein-figwheel https://github.com/bhauman/lein-figwheel [2] https://www.gnu.org/software/kawa/ https://www.gnu.org/software/kawa/ [3] https://github.com/tonyg/racket-reloadable/tree/master#readme https://github.com/tonyg/racket-reloadable/tree/master#readm... EDIT: added Kawa and Racket.
  - dmux 6y ago
    Tcl as well.
  - em-bee 6y ago
    i believe smalltalk can do this as well. there are also lesser known languages such as pike. although pike doesn't do hotswapping of actual objects but only allows the hotswapping of the compiled classes used to instantiate new objects. this works well enough on webservers where new objects are created for every request, and only live as long as the request is processed. there is a platform called open-steam that emulates hotswapping by using proxy objects. when i reference an object, i actually get a proxy, that then references the real object. when i update the code, then the proxy gets updated to the new real object, and for me it looks like the object i am referencing has been hotswapped.
  - 6y ago
redis_mlc 6y ago
You can use netfilter/iptables on linux to switch code behind a listening port. I'd rather do that than a busy-wait utility, etc.
- dividuum 6y ago
  How so? I assume adding -j DROP temporarily while the code restarts? Or is there any other trick?
pmontra 6y ago
By the way, we had this discussion about Erlang's hot reloading five years ago. https://news.ycombinator.com/item?id=10669131 https://news.ycombinator.com/item?id=10669131 Erlang runs on a VM designed for hot reloading so the comparison is somewhat unfair to Python which was not designed for that.
xiaodai 6y ago
This is like Revise.jl in Julia
adenozine 6y ago
Wait, this is actually just reloading a file upon filesystem changes? That's not hot-swapping at all. How grimy.
glic3rinu 6y ago
When I add autoreload functionality to my programs I just spawn a thread that checks for last modified date and then re-executes the program with os.execlp(sys.argv[0], *sys.argv) I found my solution superior in several ways: - I can force an autoreload just by saving a file (no file changes needed to force an md5 diff) - exec() doesn't suffer from the side effects of not restarting the python interpreter, clean start every time :) - it is also quite portable and doesn't require extra dependencies like inotify/fswatch/etc. </ul>
ed25519FUUU 6y ago
Looks like it uses hashlib to check the file for changes, which is a nice way to make it cross-platform compatible. For those who run on linux, you can use the inotify tools to set up a watch on a file (or directory or recursive directories). inotify uses kernel events and won't require re-reading and re-hashing the file.
hacknat 6y ago
Don’t hot swap. Just don’t do it. 99.99% of us should never need to do it. Learn sound network topology and load balancer practices instead. Embedded? Your hardware documentation will tell you how to offload this problem to it. There’s no earthly reason for most of us to be learning this.
raymondh 6y ago
This seems like an anti-pattern to me.
ranman 6y ago
Why was this upvoted? This does nothing with “hot swapping”...

120 points by metahost 6y ago

pronoyc 6y ago

How is this different from using Python Watchdog? https://pythonhosted.org/watchdog/ https://pythonhosted.org/watchdog/

erikschoster 6y ago
Looks like it starts a thread that repeatedly hashes the content of the target script to poll for changes. Watchdog uses OS features like inotify and FSEvents and only uses polling and comparing hashes as a fallback.

sillysaurusx 6y ago

Does this "just work"? How does it handle classes? I've been writing my python code in a style that can be hotswapped. Roughly, I use global functions and pass around structs, like C. The function calls are made with `api.foo(data)` instead of `data.foo()`. Then I can just replace `api.foo`'s definition with whatever I want after attaching with a debugger. Example: https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d6cfdc3192aba0f2b4/train_multi.py#L475-L499 https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d... I assume there's a much better way to do this, but the class system seems to make it hard to swap things out. Doesn't "isinstance" break if you redefine the class? EDIT: Oh, this isn't actual hot-reloading for python. Darn. I'm really hoping someone will tackle that... Cool project though! EDIT2: Actually, this project does seem to reload the modules: https://github.com/say4n/hotreload/blob/0cf3a0b466466f99f8c7ccdb546fcef4553617f9/hotreload/reloader.py#L47-L48 https://github.com/say4n/hotreload/blob/0cf3a0b466466f99f8c7... ... so the original question stands. By the way, you may want to use `import traceback; traceback.print_exc()` to log exceptions. I'm not sure how that plays with python's `logger` class though.

oefrha 6y ago
importlib.reload does not "just work". https://docs.python.org/3/library/importlib.html#importlib.reload https://docs.python.org/3/library/importlib.html#importlib.r... documents what is done and not done. > Doesn't "isinstance" break if you redefine the class? With importlib.reload, yes. You might be interested in IPython's autoreload implementation, which does not break isinstance: https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f16318a4ceca0ac1c3688697/IPython/extensions/autoreload.py#L253-L417 https://github.com/ipython/ipython/blob/f8c9ea7db42d9830f163... However, you could still end up with semi-broken instances, for instance if you update class A: pass to class A: def __init__(self): self.attr = "something" def __str__(self): return self.attr > By the way, you may want to use `import traceback; traceback.print_exc()` to log exceptions. I'm not sure how that plays with python's `logger` class though. PSL logging has builtin integration with exception tracebacks. See Logger.exception [1] or the exc_info keyword param to Logger.debug/info/warning/error/critical/log [2]. [1] https://docs.python.org/3/library/logging.html#logging.Logger.exception https://docs.python.org/3/library/logging.html#logging.Logge... [2] https://docs.python.org/3/library/logging.html#logging.Logger.debug https://docs.python.org/3/library/logging.html#logging.Logge...
- aidos 6y ago
  I couldn’t live without ipython autoreload. You need to know where the limitations are that require you to create your instances again. Things like sqlalchemy model changes run into issues because of the meta class magic (I think, not looked too deeply). I feel like this library is a bit misrenamed. It seems to be more like “autorun”. Edit: here’s the ipython config I use to turn it on https://github.com/aidos/dotfiles/blob/master/_config/ipython/profile_default/ipython_config.py https://github.com/aidos/dotfiles/blob/master/_config/ipytho...
viraptor 6y ago
Using logger.exception("message") would also log the trace by default.

oefrha 6y ago

For comparison, here’s Werkzeug’s reloader implementation: https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_reloader.py https://github.com/pallets/werkzeug/blob/master/src/werkzeug... (when you flask run or call app.run with reloader turned on, it’s handled by this module.)

sillysaurusx 6y ago
Thanks! That `_iter_module_files()` function is handy. Unfortunately, it looks like it simply restarts Python each time the code changes: https://github.com/pallets/werkzeug/blob/75215972967c1c00c15b8b5c3c661c2278423360/src/werkzeug/_reloader.py#L156 https://github.com/pallets/werkzeug/blob/75215972967c1c00c15... which isn't quite the same thing as reloading it.
- the_mitsuhiko 6y ago
  It used to do hot reloading back in the days but unfortunately that doesn’t really work in Python because of all the import side effects. It’s better to restart everything. Werkzeug keeps the socket open so you don’t observe the reload from the outside.
- oefrha 6y ago
  Actual reloading is just hairy and you can end up with semi-broken instances among other things. Restarting the interpreter is just cleaner, and it's totally acceptable for a dev server. (Okay looks like Armin himself replied too, obviously take his words above mine.) I pointed to IPython's autoreload in another reply which is more powerful than importlib.reload.
- raziel2p 6y ago
  Why is that unfortunate?

sfgweilr4f 6y ago

Not keen on hashing the file every second but I understand why... I'd rather it used filesystem modification events and responded accordingly. But that might more more complex than what you intend. But for simple scripts being live-coded that need to be quickly "up" and hot-swapped on rapid modification this looks fine. I could always increase the delay to a minute or hour or more if the changes were only occasionally changed.

hathym 6y ago

having an infinite loop reading the file and calculating its sha checksum every 1s is not ideal. I'd suggest using the OS premitives to get notifications when the file is touched (inotify on linux and ReadDirectoryChangesW on windows), or simply use the watchdog lib

deleted 6y ago

[deleted]

BiteCode_dev 6y ago

Because of the nature of Python, using importlib.reload() is going to lead to terrible bugs at some point. Use importlib.reload() only in the shell, for convenience. If you want autoreload for dev (e.g: like django dev server), better restart the entire process. There is a 7 years old project that can help you with that called watchdog: https://pythonhosted.org/watchdog/ It's a library that uses the fastest way available (e.g: inotify,kqueue...) to react to file changes (you can filter by types such as create, delete, write, etc) and triggers whatever code you want, passing you the metadata. If you just want something very simple, you don't have to code, you can install watchdog with its optional watchmedo dependancy: pip install watchdog[watchmedo] It will give you the watchmedo command, that lets you do something like this: watchmedo shell-command --patterns="*.py" --recursive \ --command='python your_script_to_rerun.py' .

stock_toaster 6y ago
entr[1] is another good option. [1]: http://eradman.com/entrproject/ http://eradman.com/entrproject/
- BiteCode_dev 6y ago
  Yes but unfortunatly, you have to compile it for many systems yourself. And if you want to use it from Python (not from the command line), you need binding. On the other hand, it's probably more performant than watchdog if you have a lot of files and need to fallback on manual watching. Plus, if you are not a Python person, you may not want to use pip.
X-Istence 6y ago
If you are working on code that runs a web server or some other service that is generally a long running application, take a look at hupper: https://docs.pylonsproject.org/projects/hupper/en/latest/ https://docs.pylonsproject.org/projects/hupper/en/latest/ It is built specifically to restart things like servers as it was implemented for the Pylons Project (Pyramid web framework).
- 6y ago

orf 6y ago

Hot reloading is next to impossible to do safely in the general case in Python, and this library is really basic and doesn’t really do anything special. The only case where it does work is if you maintain a single reference to the module you wish to reload, _and_ the target module doesn’t load any C extensions, _and_ it doesn’t create state outside its module (very easy to do accidentally). That’s why every autoreloader implementation restarts the whole process when any change is made, because how would you even detect that it was safe to hot reload something in the general case let alone actually hot reload it safely? If you need to add an autoreloader to your project use Hupper[1]. If you’re weird like me and you’re interested in autoreloaders, check out the Django[2] and Flask[3] implementations or my talk[4] for more details. 1. https://pypi.org/project/hupper/ https://pypi.org/project/hupper/ 2. https://github.com/django/django/blob/master/django/utils/autoreload.py https://github.com/django/django/blob/master/django/utils/au... 3. https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_reloader.py https://github.com/pallets/werkzeug/blob/master/src/werkzeug... 4. https://youtu.be/IghyoR6ld60 https://youtu.be/IghyoR6ld60

lunixbochs 6y ago
I got it working well in a large project but I basically designed auto reloading into deep parts of the framework. It's not something you can safely bolt on, and I'm honestly not sure I recommend the sanity impact of trying to build it properly (e.g. ran into a bunch of memory leaking even once I had it working safely). But there are some kinds of programs you can't really restart on every change. The biggest caveat of these kinds of single file reloaders is they completely fall apart when you import between your modules. For example, if `a` does `from b import obj` and `b` does `from c import obj`, and you modify `c.py`, `a` and `b` will both have a stale reference to obj (which might be a primitive or even interned type like a string or an int, so it's not like you can mutate it as a solution).
- orf 6y ago
  I updated my comment to say “impossible in the general case” which is more accurate, but I’d be interested in knowing more about that project - how big was it, and how did handle some of the tricky bits? Like, module A creates a class/function that is consumed by module B and C. If you change module A, how did you know to also reload B and C (recursively)?

david_draco 6y ago

You can do something like this on Linux: inotifywait -m myfile.py -e 'CLOSE_WRITE' | while read line; do python3 myfile.py; done It will be more responsive than a busy while-sleep loop.

jbverschoor 6y ago

Sorry, but that's not really hot-reloading. It's only reloading a file. Why do we only get hot-code-replacement and edit-code-and-continue-debugging from microsoft and sun(oracle)?

julvo 6y ago
Have a look at https://github.com/julvo/reloading https://github.com/julvo/reloading, it's a little library I wrote that can give you edit-code-and-continue-debugging in Python loops. A more general solution for this sort of development in Python would be neat though.
- jbverschoor 6y ago
  So this should be posted instead of the OP

underdeserver 6y ago

For those using Bazel, there's ibazel: https://github.com/bazelbuild/bazel-watcher https://github.com/bazelbuild/bazel-watcher It's great, I use it all the time, especially with unit tests.

brodul 6y ago

As other have pointed out, the best way is to restart the process. If you are looking for hot reloading you should look at programming languages that run in BEAM like Erlang and Elixir[1]. There are some gotchas with hot swapping, but it's designed and used in production for many years [2]. If somebody knows other languages/systems that do let me know. 1. https://elixir-lang.org/ https://elixir-lang.org/ 2. https://stackoverflow.com/questions/37368376/how-does-erlang-hot-code-swapping-work-in-the-middle-of-activity https://stackoverflow.com/questions/37368376/how-does-erlang...

dig1 6y ago
> If somebody knows other languages/systems that do let me know. Clojure can do this via namespace live reload. ClojureScript is already doing this inside figwheel [1] environment. CommonLisp had this since dawn of time. Kawa [2] can do it, but because it aggressively optimize the code, you need to be careful. Racket can do it as well [3]. [1] https://github.com/bhauman/lein-figwheel https://github.com/bhauman/lein-figwheel [2] https://www.gnu.org/software/kawa/ https://www.gnu.org/software/kawa/ [3] https://github.com/tonyg/racket-reloadable/tree/master#readme https://github.com/tonyg/racket-reloadable/tree/master#readm... EDIT: added Kawa and Racket.
- dmux 6y ago
  Tcl as well.
- em-bee 6y ago
  i believe smalltalk can do this as well. there are also lesser known languages such as pike. although pike doesn't do hotswapping of actual objects but only allows the hotswapping of the compiled classes used to instantiate new objects. this works well enough on webservers where new objects are created for every request, and only live as long as the request is processed. there is a platform called open-steam that emulates hotswapping by using proxy objects. when i reference an object, i actually get a proxy, that then references the real object. when i update the code, then the proxy gets updated to the new real object, and for me it looks like the object i am referencing has been hotswapped.
- 6y ago

redis_mlc 6y ago

You can use netfilter/iptables on linux to switch code behind a listening port. I'd rather do that than a busy-wait utility, etc.

dividuum 6y ago
How so? I assume adding -j DROP temporarily while the code restarts? Or is there any other trick?

pmontra 6y ago

By the way, we had this discussion about Erlang's hot reloading five years ago. https://news.ycombinator.com/item?id=10669131 https://news.ycombinator.com/item?id=10669131 Erlang runs on a VM designed for hot reloading so the comparison is somewhat unfair to Python which was not designed for that.

xiaodai 6y ago

This is like Revise.jl in Julia

adenozine 6y ago

Wait, this is actually just reloading a file upon filesystem changes? That's not hot-swapping at all. How grimy.

glic3rinu 6y ago

When I add autoreload functionality to my programs I just spawn a thread that checks for last modified date and then re-executes the program with os.execlp(sys.argv[0], *sys.argv) I found my solution superior in several ways: - I can force an autoreload just by saving a file (no file changes needed to force an md5 diff) - exec() doesn't suffer from the side effects of not restarting the python interpreter, clean start every time :) - it is also quite portable and doesn't require extra dependencies like inotify/fswatch/etc. </ul>

ed25519FUUU 6y ago

Looks like it uses hashlib to check the file for changes, which is a nice way to make it cross-platform compatible. For those who run on linux, you can use the inotify tools to set up a watch on a file (or directory or recursive directories). inotify uses kernel events and won't require re-reading and re-hashing the file.

hacknat 6y ago

Don’t hot swap. Just don’t do it. 99.99% of us should never need to do it. Learn sound network topology and load balancer practices instead. Embedded? Your hardware documentation will tell you how to offload this problem to it. There’s no earthly reason for most of us to be learning this.

raymondh 6y ago

This seems like an anti-pattern to me.

ranman 6y ago

Why was this upvoted? This does nothing with “hot swapping”...