6 ms·

Dynamically patch a Python function's source code at runtime

157 points by apwheele 1y ago

tovej 1y ago
Ah, self-modifying code, the more things change the more they stay the same. Wasn't SMC one of the LISP-associated AI fields a few decades ago? iirc it's been mostly abandoned due to security issues, but some of it survives in dynamic compilation.
- thrown-0825 1y ago
  this will be the next big breakthrough for agents lmao
rob_c 1y ago
Aka I just learned how to modify python code whilst it's running. Next step, here's how to load modules, resolve a dependency. Handle capabilities and dynamically inject more functionality 'live'. Patching running machine code in memory for compiled objects is the same but you just need to work around the abstraction that is introduced by languages trying to make the whole stack human parseable.
aeonik 1y ago
And wouldn't it be nice if that Python code, instead of a string, was just more python? Then you could use your existing Python code to append, or transform sections of your code! That's what Lisp is! Once you see how cool that is, then you can begin to appreciate why Lisp was the defacto standard for AI programing all the way back in the 1960s!
- almostgotcaught 1y ago
  https://docs.python.org/3/library/ast.html https://docs.python.org/3/library/ast.html
  - diggan 1y ago
    Ah, so in Python, you have "normal code" then you have AST code. Imagine that they were exactly the same, and whenever you're writing "normal code", you're at the same time writing AST code and vice-versa. So whenever you want, you can start using "normal code" for manipulating the "normal code" itself, and hopefully now we have yet another perspective on the same thing, for why Lisps are so awesome :)
    - breuleux 1y ago
      I think there is potential to do something like that with template strings [1] (upcoming feature in Python 3.14). The choice of {} for interpolation isn't ideal because any code with dict and set literals becomes super awkward, but besides that, it could be super interesting for codegen. [1] https://peps.python.org/pep-0750/ https://peps.python.org/pep-0750/
      - diggan 1y ago
        I think what I was getting at, is the power this homoiconicity gives you when combined with macros, and I'm not sure how template strings would help with either. Say you want to have a function that receives the code passed into it, instead of whatever the code evaluated to return, could I somehow achieve that with AST and/or template strings in Python? Say I want `what-code(1 + 1)` to receive `1 + 1` as the argument, not `2`, would either of those things let me do that? At a glance, and without diving deeper, the Ast module could make it so `what-code("1 + 1")` would let the function parse the string and get an Ast, which we could do stuff with, but it pretty much ends at that and is non-ideal for many reasons.
        breuleux 1y ago
        Well... what are you trying to do with "1 + 1"? Lisps are "homoiconic" in the sense that everything is a list, but honestly? When I worked with Scheme in the past, I almost never manipulated code as actual lists. It's not robust. I used match and quote/unquote. What's where I see template strings. So what you could do is something like this: addition = "1 + 1" multiplication = t"{addition} * 7" result = eval(multiplication) # result is 14 Instead of inserting addition as an exact string, producing "1 + 1 * 7" which is 8, it would parse as an AST and insert the addition node as a unit, so that you get the intended result. You could also do an alpha conversion if the injected code declares variables to avoid name clashes. It wouldn't let you do true macros, but it would make codegen easier and less error-prone.
miohtama 1y ago
Also check the history of monkey patching https://en.m.wikipedia.org/wiki/Monkey_patch https://en.m.wikipedia.org/wiki/Monkey_patch
faangguyindia 1y ago
Imagine if you've lisp to make ide and in a closed loop integration with LLM which extends and tests this IDE to achieve the task at hand.
- 1bpp 1y ago
  Would it be much different than a REPL?
dfee 1y ago
Different, but you may find interest in changing function signatures, too: https://github.com/dfee/forge https://github.com/dfee/forge
zzzeek 1y ago
I really couldn't follow the use case. it looked like, they have a chain of method calls with some kind of `mark_circle().encode().properties()`, OK, so, if you want to make those methods do something different, you reach into OOP wisdom from the 1980s and write an appropriate "impl" used by whatever `alt.Chart()` is. Someone explain to me, an old, aging programmer old enough to know UML, why this isn't some we presume very young person who has no idea how to write OOP coming up with some horrible convoluted way to do something routine?
- porridgeraisin 1y ago
  > aging ... UML Oh don't worry... they still cram that down our throats in CS undergrad in one of the courses.... Forget which one. I did my UG from 2020-24 ;)
breuleux 1y ago
Yeah, I've been using this trick to implement a hot reload library, to only update the specific functions that are changed without re-executing modules: https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged I also use it in a multiple dispatch library (https://github.com/breuleux/ovld https://github.com/breuleux/ovld) to replace the entry point by specialized dispatch code in order to cut some overhead. It's fun. Also, why is every damn post these days somehow framed in an AI context? It's exhausting.
- diggan 1y ago
  > Also, why is every damn post these days somehow framed in an AI context? It's exhausting. Every 5/10-year segment of my life has somehow had one or two "This is the future"-hypes running concurrently with my life. Previously it was X, now it's Y. And most of the times, everything else is somehow connected to this currently hyped subject, no matter if it's related or not. The only thing I've found to be helpful is thinking about and changing my perspective and framing about it. I read some article like this which is just tangentially related to AI, but the meat is about something else. So mentally I just ignore the other parts, and frame it in some other way in my head. Suddenly people can write their articles with attachments to the hyped subject but I don't mind, I'm reading it for other purposes and get other takeaways that are still helpful. A tiny Jedi mind-trick for avoiding that exhaustion :)
  - Twirrim 1y ago
    AI, block chain, rust, go, serverless, nosql, ruby on rails..... The list goes on and on :-) Some of it gets really annoying on the business side, because companies like Gartner jump on the trends, and they have enough influence that businesses have to pay attention. When serverless was a thing, every cloud provider effectively had to add serverless things even if it made zero sense and no customers were asking for it, simply to satisfy Gartner (and their ilk) and be seen as innovating and ahead of the curve. Same thing happened with block chain, and is currently happening with AI.
    - quantum_state 1y ago
duttish 1y ago
I used live patching of the function byte code to enforce type safety in python as an experiment. It was quite fun, took about a weekend or so :) not something for production though, due to the performance hit.
jeomon27 1y ago
[dead]
robertclaus 1y ago
Concurrency?
ath3nd 1y ago
There is a reason similar approaches are called 'monkey patching'. Just cause you can do something doesn't mean you should. I send thoughts and prayers for the people debugging programs where this is in place.
woadwarrior01 1y ago
Greenspun's tenth rule, restated for Python instead of C/Fortran. https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
syngrog66 1y ago
Python being riddled with security anti-patterns -- or at least security-unfriendly ergonomics -- is one reason I tried to stop working with it.
edem 1y ago
wait a sec ... i thought you could monkey patch python code. you can only do it by using this technique?
- deleted 1y ago
  [deleted]
zekrioca 1y ago
Other than Lisp, is it possible to do this in other languages?
- OskarS 1y ago
  Several! You're correct that Lisps is the most famous, and there's also languages like Erlang that have this as a core functionality. But it's also used in things like game engines for C/C++. You do have your "updateAndRenderFrame()" function in a dynamic library, having it take a pointer to the full game state as an argument. When you want to reload, you recompile your dynamic library, and the main loop can swap out the implementation, and the game keeps running with the new code. I don't see a reason why you couldn't do this in Rust, though I imagine it's trickier.
  - cenamus 1y ago
    And don't forget smalltalk!
- kbolino 1y ago
  Any language running on the JVM (Java, Kotlin, etc.) and CLR (C#, VB.NET, etc.). As long as you don't compile directly to native code and you aren't running in a locked-down environment where codegen is disabled, you can generate code at runtime and execute it alongside existing code. For example, there's java.lang.reflect.Proxy [1] and System.Reflection.Emit [2]. [1]: https://docs.oracle.com/javase/8/docs/technotes/guides/reflection/proxy.html https://docs.oracle.com/javase/8/docs/technotes/guides/refle... [2]: https://learn.microsoft.com/en-us/dotnet/api/system.reflection.emit?view=net-9.0 https://learn.microsoft.com/en-us/dotnet/api/system.reflecti...
eidorb 1y ago
Another use case is applying small patches to a large function or method. Overriding would mean copy pasting largely similar code. It’s a bit ugly. https://github.com/eidorb/ubank/blob/master/soft_webauthn_patched.py https://github.com/eidorb/ubank/blob/master/soft_webauthn_pa...
lavishlatern 1y ago
I'm not sure what the point of this blogpost was. As far as I can tell, the author discovered eval() but is making it more complicated for no reason? There also isn't any actual patching going on.
j-h-k 1y ago
One of the problem with dynamically loaded code like this is when you raise exceptions in those functions - the traceback will then end with something like `File "<magic>", line 8, in something`, which will at least annoy you when debugging.
mkoubaa 1y ago
Monkey patching python modules is waaaaay more straightforward than this. setattr(mod, name, new_func)
m3047 1y ago
I've contemplated patching Python's asyncio package so that the list of funcs isn't a weakref... but it's not that hard! This is way harder than it needs to be.
pontifier 1y ago
I did this by accident in bash scripts many years ago when I was just getting into linux. I'd be running a script, and editing the script at the same time. It caused some REALLY weird issues before I figured out what was happening. For instance I'd change the text somewhere and it would change in the running program, or the program would get into states it should never be in. I didn't use it constructively, I just avoided editing running programs after that.
tpoacher 1y ago
Neat trick indeed, but I have not yet found a compelling use-case for this that can't be done more sensibly using lambdas. I once inherited a nice library which made heavy use of such compile/exec pairs. The code looked very convoluted and was very bug prone. I replaced all such pairs with lambda wrappers. This was much more readable: my colleagues who were also working on this code were like "oh is that what that was meant to be doing" after this change.
ianbicking 1y ago
The use case is not that obvious to me, but I _think_ it's trying to give the LLM more context to write code by dipping into these code objects. Perhaps telling the LLM what modules are already imported or otherwise available? Creating a debugger for an LLM (not a human) is something I haven't really seen, but seems super useful...?

157 points by apwheele 1y ago

Ah, self-modifying code, the more things change the more they stay the same. Wasn't SMC one of the LISP-associated AI fields a few decades ago? iirc it's been mostly abandoned due to security issues, but some of it survives in dynamic compilation.

thrown-0825 1y ago
this will be the next big breakthrough for agents lmao

rob_c 1y ago

Aka I just learned how to modify python code whilst it's running. Next step, here's how to load modules, resolve a dependency. Handle capabilities and dynamically inject more functionality 'live'. Patching running machine code in memory for compiled objects is the same but you just need to work around the abstraction that is introduced by languages trying to make the whole stack human parseable.

aeonik 1y ago

And wouldn't it be nice if that Python code, instead of a string, was just more python? Then you could use your existing Python code to append, or transform sections of your code! That's what Lisp is! Once you see how cool that is, then you can begin to appreciate why Lisp was the defacto standard for AI programing all the way back in the 1960s!

almostgotcaught 1y ago
https://docs.python.org/3/library/ast.html https://docs.python.org/3/library/ast.html
- diggan 1y ago
  Ah, so in Python, you have "normal code" then you have AST code. Imagine that they were exactly the same, and whenever you're writing "normal code", you're at the same time writing AST code and vice-versa. So whenever you want, you can start using "normal code" for manipulating the "normal code" itself, and hopefully now we have yet another perspective on the same thing, for why Lisps are so awesome :)
  - breuleux 1y ago
    I think there is potential to do something like that with template strings [1] (upcoming feature in Python 3.14). The choice of {} for interpolation isn't ideal because any code with dict and set literals becomes super awkward, but besides that, it could be super interesting for codegen. [1] https://peps.python.org/pep-0750/ https://peps.python.org/pep-0750/
    - diggan 1y ago
      I think what I was getting at, is the power this homoiconicity gives you when combined with macros, and I'm not sure how template strings would help with either. Say you want to have a function that receives the code passed into it, instead of whatever the code evaluated to return, could I somehow achieve that with AST and/or template strings in Python? Say I want `what-code(1 + 1)` to receive `1 + 1` as the argument, not `2`, would either of those things let me do that? At a glance, and without diving deeper, the Ast module could make it so `what-code("1 + 1")` would let the function parse the string and get an Ast, which we could do stuff with, but it pretty much ends at that and is non-ideal for many reasons.
      - breuleux 1y ago
        Well... what are you trying to do with "1 + 1"? Lisps are "homoiconic" in the sense that everything is a list, but honestly? When I worked with Scheme in the past, I almost never manipulated code as actual lists. It's not robust. I used match and quote/unquote. What's where I see template strings. So what you could do is something like this: addition = "1 + 1" multiplication = t"{addition} * 7" result = eval(multiplication) # result is 14 Instead of inserting addition as an exact string, producing "1 + 1 * 7" which is 8, it would parse as an AST and insert the addition node as a unit, so that you get the intended result. You could also do an alpha conversion if the injected code declares variables to avoid name clashes. It wouldn't let you do true macros, but it would make codegen easier and less error-prone.

miohtama 1y ago

Also check the history of monkey patching https://en.m.wikipedia.org/wiki/Monkey_patch https://en.m.wikipedia.org/wiki/Monkey_patch

faangguyindia 1y ago

Imagine if you've lisp to make ide and in a closed loop integration with LLM which extends and tests this IDE to achieve the task at hand.

1bpp 1y ago
Would it be much different than a REPL?

dfee 1y ago

Different, but you may find interest in changing function signatures, too: https://github.com/dfee/forge https://github.com/dfee/forge

zzzeek 1y ago

I really couldn't follow the use case. it looked like, they have a chain of method calls with some kind of `mark_circle().encode().properties()`, OK, so, if you want to make those methods do something different, you reach into OOP wisdom from the 1980s and write an appropriate "impl" used by whatever `alt.Chart()` is. Someone explain to me, an old, aging programmer old enough to know UML, why this isn't some we presume very young person who has no idea how to write OOP coming up with some horrible convoluted way to do something routine?

porridgeraisin 1y ago
> aging ... UML Oh don't worry... they still cram that down our throats in CS undergrad in one of the courses.... Forget which one. I did my UG from 2020-24 ;)

breuleux 1y ago

Yeah, I've been using this trick to implement a hot reload library, to only update the specific functions that are changed without re-executing modules: https://github.com/breuleux/jurigged https://github.com/breuleux/jurigged I also use it in a multiple dispatch library (https://github.com/breuleux/ovld https://github.com/breuleux/ovld) to replace the entry point by specialized dispatch code in order to cut some overhead. It's fun. Also, why is every damn post these days somehow framed in an AI context? It's exhausting.

diggan 1y ago
> Also, why is every damn post these days somehow framed in an AI context? It's exhausting. Every 5/10-year segment of my life has somehow had one or two "This is the future"-hypes running concurrently with my life. Previously it was X, now it's Y. And most of the times, everything else is somehow connected to this currently hyped subject, no matter if it's related or not. The only thing I've found to be helpful is thinking about and changing my perspective and framing about it. I read some article like this which is just tangentially related to AI, but the meat is about something else. So mentally I just ignore the other parts, and frame it in some other way in my head. Suddenly people can write their articles with attachments to the hyped subject but I don't mind, I'm reading it for other purposes and get other takeaways that are still helpful. A tiny Jedi mind-trick for avoiding that exhaustion :)
- Twirrim 1y ago
  AI, block chain, rust, go, serverless, nosql, ruby on rails..... The list goes on and on :-) Some of it gets really annoying on the business side, because companies like Gartner jump on the trends, and they have enough influence that businesses have to pay attention. When serverless was a thing, every cloud provider effectively had to add serverless things even if it made zero sense and no customers were asking for it, simply to satisfy Gartner (and their ilk) and be seen as innovating and ahead of the curve. Same thing happened with block chain, and is currently happening with AI.
  - quantum_state 1y ago

duttish 1y ago

I used live patching of the function byte code to enforce type safety in python as an experiment. It was quite fun, took about a weekend or so :) not something for production though, due to the performance hit.

jeomon27 1y ago

[dead]

robertclaus 1y ago

Concurrency?

ath3nd 1y ago

There is a reason similar approaches are called 'monkey patching'. Just cause you can do something doesn't mean you should. I send thoughts and prayers for the people debugging programs where this is in place.

woadwarrior01 1y ago

Greenspun's tenth rule, restated for Python instead of C/Fortran. https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

syngrog66 1y ago

Python being riddled with security anti-patterns -- or at least security-unfriendly ergonomics -- is one reason I tried to stop working with it.

edem 1y ago

wait a sec ... i thought you could monkey patch python code. you can only do it by using this technique?

deleted 1y ago
[deleted]

zekrioca 1y ago

Other than Lisp, is it possible to do this in other languages?

OskarS 1y ago
Several! You're correct that Lisps is the most famous, and there's also languages like Erlang that have this as a core functionality. But it's also used in things like game engines for C/C++. You do have your "updateAndRenderFrame()" function in a dynamic library, having it take a pointer to the full game state as an argument. When you want to reload, you recompile your dynamic library, and the main loop can swap out the implementation, and the game keeps running with the new code. I don't see a reason why you couldn't do this in Rust, though I imagine it's trickier.
- cenamus 1y ago
  And don't forget smalltalk!
kbolino 1y ago
Any language running on the JVM (Java, Kotlin, etc.) and CLR (C#, VB.NET, etc.). As long as you don't compile directly to native code and you aren't running in a locked-down environment where codegen is disabled, you can generate code at runtime and execute it alongside existing code. For example, there's java.lang.reflect.Proxy [1] and System.Reflection.Emit [2]. [1]: https://docs.oracle.com/javase/8/docs/technotes/guides/reflection/proxy.html https://docs.oracle.com/javase/8/docs/technotes/guides/refle... [2]: https://learn.microsoft.com/en-us/dotnet/api/system.reflection.emit?view=net-9.0 https://learn.microsoft.com/en-us/dotnet/api/system.reflecti...

eidorb 1y ago

Another use case is applying small patches to a large function or method. Overriding would mean copy pasting largely similar code. It’s a bit ugly. https://github.com/eidorb/ubank/blob/master/soft_webauthn_patched.py https://github.com/eidorb/ubank/blob/master/soft_webauthn_pa...

lavishlatern 1y ago

I'm not sure what the point of this blogpost was. As far as I can tell, the author discovered eval() but is making it more complicated for no reason? There also isn't any actual patching going on.

j-h-k 1y ago

One of the problem with dynamically loaded code like this is when you raise exceptions in those functions - the traceback will then end with something like `File "<magic>", line 8, in something`, which will at least annoy you when debugging.

mkoubaa 1y ago

Monkey patching python modules is waaaaay more straightforward than this. setattr(mod, name, new_func)

m3047 1y ago

I've contemplated patching Python's asyncio package so that the list of funcs isn't a weakref... but it's not that hard! This is way harder than it needs to be.

pontifier 1y ago

I did this by accident in bash scripts many years ago when I was just getting into linux. I'd be running a script, and editing the script at the same time. It caused some REALLY weird issues before I figured out what was happening. For instance I'd change the text somewhere and it would change in the running program, or the program would get into states it should never be in. I didn't use it constructively, I just avoided editing running programs after that.

tpoacher 1y ago

Neat trick indeed, but I have not yet found a compelling use-case for this that can't be done more sensibly using lambdas. I once inherited a nice library which made heavy use of such compile/exec pairs. The code looked very convoluted and was very bug prone. I replaced all such pairs with lambda wrappers. This was much more readable: my colleagues who were also working on this code were like "oh is that what that was meant to be doing" after this change.

ianbicking 1y ago

The use case is not that obvious to me, but I _think_ it's trying to give the LLM more context to write code by dipping into these code objects. Perhaps telling the LLM what modules are already imported or otherwise available? Creating a debugger for an LLM (not a human) is something I haven't really seen, but seems super useful...?