5 ms·

Yeah, I've been using this trick to implement a hot reload library, to only update the specific functions that are changed without re-executing modules: https:/

Yeah, I've been using this trick to implement a hot reload library, to only update the specific functions that are changed without re-executing modules: https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged I also use it in a multiple dispatch library (https://github.com/breuleux/ovld https://github.com/breuleux/ovld) to replace the entry point by specialized dispatch code in order to cut some overhead. It's fun. Also, why is every damn post these days somehow framed in an AI context? It's exhausting.

diggan 1y ago
> Also, why is every damn post these days somehow framed in an AI context? It's exhausting. Every 5/10-year segment of my life has somehow had one or two "This is the future"-hypes running concurrently with my life. Previously it was X, now it's Y. And most of the times, everything else is somehow connected to this currently hyped subject, no matter if it's related or not. The only thing I've found to be helpful is thinking about and changing my perspective and framing about it. I read some article like this which is just tangentially related to AI, but the meat is about something else. So mentally I just ignore the other parts, and frame it in some other way in my head. Suddenly people can write their articles with attachments to the hyped subject but I don't mind, I'm reading it for other purposes and get other takeaways that are still helpful. A tiny Jedi mind-trick for avoiding that exhaustion :)
- Twirrim 1y ago
  AI, block chain, rust, go, serverless, nosql, ruby on rails..... The list goes on and on :-) Some of it gets really annoying on the business side, because companies like Gartner jump on the trends, and they have enough influence that businesses have to pay attention. When serverless was a thing, every cloud provider effectively had to add serverless things even if it made zero sense and no customers were asking for it, simply to satisfy Gartner (and their ilk) and be seen as innovating and ahead of the curve. Same thing happened with block chain, and is currently happening with AI.
  - quantum_state 1y ago
    wheels get reinvented again again again and again … this is quite unique to info tech … imagine mathematicians do the same, the world would be in chaos …
    - diggan 1y ago
      > wheels get reinvented again again again and again And yet, it took us (humans) a long time to turn the wheel around and using it for transportation, so maybe we need reinvent even more things, turn them around and such :)
- dkdcio 1y ago
  I also find it useful to keep in mind (often more junior) people are learning new things and expressing their joy, which is a good thing. and most (junior) people learning things in tech right now are doing so in the context of AI, for better or worse (idk if this author is “junior” per se, mostly just agreeing the shift in perspective is helpful to not get burnt out by things like this)
modeless 1y ago
Jurigged is awesome. It works really well and saves me tons of time. Thank you for making it! I do wish there were callbacks I could subscribe to that would notify me whenever my file changed or whenever any code changed, so I could re-run some init. My other feature request would be a way to replace function implementations even when they are currently on the stack, as some other hot reload implementations can. But I certainly understand why this would be difficult.
gjvc 1y ago
I use jurigged in conjunction with cmd2 to make command line driven tools. The combination works well because I can issue a command, watch the output, make a change, hit up-return and see the change just like that. Thank you a bazillion for making it. It works quietly in the background without fuss, and I'm grateful for it every time I use it.
zipy124 1y ago
jurigged is great, love using it for quick GUI prototyping with imgui!
johnfn 1y ago
Jurigged is really cool - thanks for the tool!
Esophagus4 1y ago
> why is every damn post these days somehow framed in an AI context? It's exhausting. It’s even in the real world now - most of my conversations with people in tech end up at AI eventually. It kind of reminds me of the 2010s when non-tech people would ask me about crypto at social events.
frou_dh 1y ago
In some respects that's even nicer to use than a typical editor-integrated live-coding REPL, because one doesn't have to think about what code needs (re)sending from the source to the REPL. Just save the file and it'll be figured out which parts meaningfully changed.
sczi 1y ago
Oh that is really interesting, I was just aware of IPython's autoreload extension, I hadn't found your library. I'm also working on hot reload for python as I'm working on a development environment for python that aims to give it a development experience closer to lisp: https://codeberg.org/sczi/swanky-python/ https://codeberg.org/sczi/swanky-python/ Some minor details. You currently aren't updating functions if their freevars have changed, you can actually do that by using c-api to update __closure__ which is a readonly attribute from python: ctypes.pythonapi.PyFunction_SetClosure.argtypes = [ctypes.py_object, ctypes.py_object] ctypes.pythonapi.PyFunction_SetClosure(old, new.__closure__) Also I think you should update __annotations__, __type_params__, __doc__, and __dict__ attributes for the function. Rather than using gc.get_referrers I just maintain a set for each function containing all the old versions (using weakref so they go away if that old version isn't still referenced by anything). Then when a function updates I don't need to find all references, all references will be to some old version of the function so I just update that set of old functions, and all references will be using the new code. I took this from IPython autoreload. I think it is both more efficient than gc.get_referrers, and more complete as it solves the issue of references "decorated or stashed in some data structure that Jurigged does not understand". The code for that is here: https://codeberg.org/sczi/swanky-python/src/commit/365702a6cc70b312974e8c5c8776514bc93e31ef/src/swanky_python/swank_common.py#L321-L339 https://codeberg.org/sczi/swanky-python/src/commit/365702a6c... hot reload for python is quite tricky to fully get right, I'm still missing plenty parts that I know about and plan on implementing, and surely plenty more that I don't even know. If you or anyone else that's worked on hot reload in python wants to talk about it, I'm happy to, just reach out, my email is visible on codeberg if you're signed in.
- breuleux 1y ago
  Thanks for the tips, I'll try to look into these when I get some time! Didn't know you could modify the closure pointer. I'm not sure what you mean by "maintaining a set of old versions". It's possible I missed something obvious, but the issue here is that I have the code objects (I can snag them from module evaluation using an import hook)... but I do not have the function objects. I never had them in the first place. Take this very silly and very horrible example: def adder(x): def inner(y): return x / y return inner adders = {} def add(x, y): adders.setdefault(x, adder(x)) return adders[x](y) The adders dictionary is dynamically updated with new closures. Each is a distinct function object with a __code__ field. When I update the inner function, I want all of these closures to be updated. Jurigged is able to do it -- it can root them out using get_referrers. I don't see how else to do it. I quickly tested in a Jupyter notebook, and it didn't work: new closures have the new code, but the old ones are not updated.

diggan 1y ago

> Also, why is every damn post these days somehow framed in an AI context? It's exhausting. Every 5/10-year segment of my life has somehow had one or two "This is the future"-hypes running concurrently with my life. Previously it was X, now it's Y. And most of the times, everything else is somehow connected to this currently hyped subject, no matter if it's related or not. The only thing I've found to be helpful is thinking about and changing my perspective and framing about it. I read some article like this which is just tangentially related to AI, but the meat is about something else. So mentally I just ignore the other parts, and frame it in some other way in my head. Suddenly people can write their articles with attachments to the hyped subject but I don't mind, I'm reading it for other purposes and get other takeaways that are still helpful. A tiny Jedi mind-trick for avoiding that exhaustion :)

Twirrim 1y ago
AI, block chain, rust, go, serverless, nosql, ruby on rails..... The list goes on and on :-) Some of it gets really annoying on the business side, because companies like Gartner jump on the trends, and they have enough influence that businesses have to pay attention. When serverless was a thing, every cloud provider effectively had to add serverless things even if it made zero sense and no customers were asking for it, simply to satisfy Gartner (and their ilk) and be seen as innovating and ahead of the curve. Same thing happened with block chain, and is currently happening with AI.
- quantum_state 1y ago
  wheels get reinvented again again again and again … this is quite unique to info tech … imagine mathematicians do the same, the world would be in chaos …
  - diggan 1y ago
    > wheels get reinvented again again again and again And yet, it took us (humans) a long time to turn the wheel around and using it for transportation, so maybe we need reinvent even more things, turn them around and such :)
dkdcio 1y ago
I also find it useful to keep in mind (often more junior) people are learning new things and expressing their joy, which is a good thing. and most (junior) people learning things in tech right now are doing so in the context of AI, for better or worse (idk if this author is “junior” per se, mostly just agreeing the shift in perspective is helpful to not get burnt out by things like this)

modeless 1y ago

Jurigged is awesome. It works really well and saves me tons of time. Thank you for making it! I do wish there were callbacks I could subscribe to that would notify me whenever my file changed or whenever any code changed, so I could re-run some init. My other feature request would be a way to replace function implementations even when they are currently on the stack, as some other hot reload implementations can. But I certainly understand why this would be difficult.

gjvc 1y ago

I use jurigged in conjunction with cmd2 to make command line driven tools. The combination works well because I can issue a command, watch the output, make a change, hit up-return and see the change just like that. Thank you a bazillion for making it. It works quietly in the background without fuss, and I'm grateful for it every time I use it.

zipy124 1y ago

jurigged is great, love using it for quick GUI prototyping with imgui!

johnfn 1y ago

Jurigged is really cool - thanks for the tool!

Esophagus4 1y ago

> why is every damn post these days somehow framed in an AI context? It's exhausting. It’s even in the real world now - most of my conversations with people in tech end up at AI eventually. It kind of reminds me of the 2010s when non-tech people would ask me about crypto at social events.

frou_dh 1y ago

In some respects that's even nicer to use than a typical editor-integrated live-coding REPL, because one doesn't have to think about what code needs (re)sending from the source to the REPL. Just save the file and it'll be figured out which parts meaningfully changed.

sczi 1y ago

Oh that is really interesting, I was just aware of IPython's autoreload extension, I hadn't found your library. I'm also working on hot reload for python as I'm working on a development environment for python that aims to give it a development experience closer to lisp: https://codeberg.org/sczi/swanky-python/ https://codeberg.org/sczi/swanky-python/ Some minor details. You currently aren't updating functions if their freevars have changed, you can actually do that by using c-api to update __closure__ which is a readonly attribute from python: ctypes.pythonapi.PyFunction_SetClosure.argtypes = [ctypes.py_object, ctypes.py_object] ctypes.pythonapi.PyFunction_SetClosure(old, new.__closure__) Also I think you should update __annotations__, __type_params__, __doc__, and __dict__ attributes for the function. Rather than using gc.get_referrers I just maintain a set for each function containing all the old versions (using weakref so they go away if that old version isn't still referenced by anything). Then when a function updates I don't need to find all references, all references will be to some old version of the function so I just update that set of old functions, and all references will be using the new code. I took this from IPython autoreload. I think it is both more efficient than gc.get_referrers, and more complete as it solves the issue of references "decorated or stashed in some data structure that Jurigged does not understand". The code for that is here: https://codeberg.org/sczi/swanky-python/src/commit/365702a6cc70b312974e8c5c8776514bc93e31ef/src/swanky_python/swank_common.py#L321-L339 https://codeberg.org/sczi/swanky-python/src/commit/365702a6c... hot reload for python is quite tricky to fully get right, I'm still missing plenty parts that I know about and plan on implementing, and surely plenty more that I don't even know. If you or anyone else that's worked on hot reload in python wants to talk about it, I'm happy to, just reach out, my email is visible on codeberg if you're signed in.

breuleux 1y ago
Thanks for the tips, I'll try to look into these when I get some time! Didn't know you could modify the closure pointer. I'm not sure what you mean by "maintaining a set of old versions". It's possible I missed something obvious, but the issue here is that I have the code objects (I can snag them from module evaluation using an import hook)... but I do not have the function objects. I never had them in the first place. Take this very silly and very horrible example: def adder(x): def inner(y): return x / y return inner adders = {} def add(x, y): adders.setdefault(x, adder(x)) return adders[x](y) The adders dictionary is dynamically updated with new closures. Each is a distinct function object with a __code__ field. When I update the inner function, I want all of these closures to be updated. Jurigged is able to do it -- it can root them out using get_referrers. I don't see how else to do it. I quickly tested in a Jupyter notebook, and it didn't work: new closures have the new code, but the old ones are not updated.