4 ms·

I've been using jurigged [0] for hot code reloading which is great. It works by replacing code objects in place at the function level which naturally handles ma

I've been using jurigged [0] for hot code reloading which is great. It works by replacing code objects in place at the function level which naturally handles many cases like applying to existing objects, avoiding loading a module multiple times, etc. As others noted the actual source of this one is not there so it's hard to know what it's doing. The published package contains a binary shared library so I suppose it's written as a CPython extension. [0] https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged

breuleux 4y ago
Jurigged author here, just tried reloadium out of curiosity. I couldn't run it on my M1 Mac (unsupported, it seems), on Linux I was able to, and it seems to work similarly (identifying what needs to be updated and updating these things specifically). That's nice to see. There is one thing jurigged does that this doesn't, which is updating closures, and I also define a protocol for custom libraries to watch changes so that e.g. a multiple dispatch library can update its internal structures when code changes. There may be things reloadium does better, for example it looks like there is some special casing for Flask/etc.? Not sure how that scales. I didn't try the PyCharm stuff. Glad jurigged works for you. Coincidentally, I was working on an update today, and I am going to present jurigged at PyCon this weekend :)
- kwazar90 4y ago
  I have tried jurigged and I like how lightweight it is. It does not seem to reload changes that relate to other parts of code though. For a given code: a = 1 b = a + 2 Changing a = 5 will not update b which will still be equal to 3. The same goes to relations between files. Whether it is a flaw or feature it depends on the situation and codebase I guess. Also it seems like it is not possible to reload currently executing functions (re-entering the function is needed).
  - chii 4y ago
    i wonder how you would make the update to `b` work in this case? If you have a white listed set of pure operations (like the `+` operation), then you can check if the hot-reloaded code has any dependencies which could be re-executed (like the b=a+2 case). You wouldn't want to re-execute code that has a side-effect upon hot reloading imho.
    - kwazar90 4y ago
      Reloadium executes a module in a sandbox environment until it reaches the modules top most frame location (to prevent getting into a loop or other entrypoint and being stuck) and looks for changes. For code that should be executed only once you have to add # reloadium: no_reload (blocks of code can be ignored as well, more details here https://reloadium.io/documentation/decorators https://reloadium.io/documentation/decorators)
  - breuleux 4y ago
    Yeah, we are using different strategies I think. My approach is to not re-run anything that has not changed, so if you change the code of a function in a module, the module's global state will remain untouched, and if you change the code of a decorated function, the decorator will not be re-run (the function inside will be fudged directly). I wouldn't say either approach is intrinsically better, in either case there are situations where it won't do what you want. I think I prefer mine because it is, in a way, a bit more predictable: if e.g. I add a print statement to some function, then the next time I call it, it will print. I know that it won't reset any global state anywhere or re-run anything expensive, regardless of what else there is in the module. Usually, that's what I want. The develoop feature (which I am rewriting right now) sort of does the reloading of currently executing functions, in the sense that you can pick one or more focus functions that are run in a loop and aborted/restarted when the source changes. At some point I looked into changing functions during a breakpoint and continuing in the new code, but I couldn't see any way to do it in pure Python, except maybe through some ctypes voodoo.
- gjvc 4y ago
  jurigged is/was a secret weapon I used in turning round a project which had gone off the rails and over-run and I needed to start from scratch. This process migrates (and transforms) data from one system to another. To do so, it needs to load rarely-changing static data from both the source and the target, and then talks to the source and target to achieve its goals. This loading of static data took ~30 seconds every time. Because I needed to start from scratch, I couldn't afford to be re-starting all the time, and jurigged was key to this. The secret was to use jurigged in co-operation with cmd/cmd2 which kept the process running between the dispatch of actions. During this interval, jurigged is able to jump in and reload, avoiding the need to restart and refetch the static data, and enabling me to edit and re-dispatch in a fluid manner. This approach avoids the mental cost of context-switching and time cost of restarting, leading to a much more flow-like state of mind. Thank you very much! (remember of course that Kent Beck taught us all this in Smalltalk and s-unit)
  - gjvc 4y ago
    this project has now been successfully delivered out of the jaws of failure. Thank you to everyone who has worked on jurigged and cmd2!

breuleux 4y ago

Jurigged author here, just tried reloadium out of curiosity. I couldn't run it on my M1 Mac (unsupported, it seems), on Linux I was able to, and it seems to work similarly (identifying what needs to be updated and updating these things specifically). That's nice to see. There is one thing jurigged does that this doesn't, which is updating closures, and I also define a protocol for custom libraries to watch changes so that e.g. a multiple dispatch library can update its internal structures when code changes. There may be things reloadium does better, for example it looks like there is some special casing for Flask/etc.? Not sure how that scales. I didn't try the PyCharm stuff. Glad jurigged works for you. Coincidentally, I was working on an update today, and I am going to present jurigged at PyCon this weekend :)

kwazar90 4y ago
I have tried jurigged and I like how lightweight it is. It does not seem to reload changes that relate to other parts of code though. For a given code: a = 1 b = a + 2 Changing a = 5 will not update b which will still be equal to 3. The same goes to relations between files. Whether it is a flaw or feature it depends on the situation and codebase I guess. Also it seems like it is not possible to reload currently executing functions (re-entering the function is needed).
- chii 4y ago
  i wonder how you would make the update to `b` work in this case? If you have a white listed set of pure operations (like the `+` operation), then you can check if the hot-reloaded code has any dependencies which could be re-executed (like the b=a+2 case). You wouldn't want to re-execute code that has a side-effect upon hot reloading imho.
  - kwazar90 4y ago
    Reloadium executes a module in a sandbox environment until it reaches the modules top most frame location (to prevent getting into a loop or other entrypoint and being stuck) and looks for changes. For code that should be executed only once you have to add # reloadium: no_reload (blocks of code can be ignored as well, more details here https://reloadium.io/documentation/decorators https://reloadium.io/documentation/decorators)
- breuleux 4y ago
  Yeah, we are using different strategies I think. My approach is to not re-run anything that has not changed, so if you change the code of a function in a module, the module's global state will remain untouched, and if you change the code of a decorated function, the decorator will not be re-run (the function inside will be fudged directly). I wouldn't say either approach is intrinsically better, in either case there are situations where it won't do what you want. I think I prefer mine because it is, in a way, a bit more predictable: if e.g. I add a print statement to some function, then the next time I call it, it will print. I know that it won't reset any global state anywhere or re-run anything expensive, regardless of what else there is in the module. Usually, that's what I want. The develoop feature (which I am rewriting right now) sort of does the reloading of currently executing functions, in the sense that you can pick one or more focus functions that are run in a loop and aborted/restarted when the source changes. At some point I looked into changing functions during a breakpoint and continuing in the new code, but I couldn't see any way to do it in pure Python, except maybe through some ctypes voodoo.
gjvc 4y ago
jurigged is/was a secret weapon I used in turning round a project which had gone off the rails and over-run and I needed to start from scratch. This process migrates (and transforms) data from one system to another. To do so, it needs to load rarely-changing static data from both the source and the target, and then talks to the source and target to achieve its goals. This loading of static data took ~30 seconds every time. Because I needed to start from scratch, I couldn't afford to be re-starting all the time, and jurigged was key to this. The secret was to use jurigged in co-operation with cmd/cmd2 which kept the process running between the dispatch of actions. During this interval, jurigged is able to jump in and reload, avoiding the need to restart and refetch the static data, and enabling me to edit and re-dispatch in a fluid manner. This approach avoids the mental cost of context-switching and time cost of restarting, leading to a much more flow-like state of mind. Thank you very much! (remember of course that Kent Beck taught us all this in Smalltalk and s-unit)
- gjvc 4y ago
  this project has now been successfully delivered out of the jaws of failure. Thank you to everyone who has worked on jurigged and cmd2!