3 ms·

Automatically inline Python function calls

46 points by albertzeyer 13y ago

lvh 13y ago
This is pretty cool, albeit pretty hacky ;) For what it's worth, you could also try to make your code work on PyPy, which will do this and much much more :)
- lucian1900 13y ago
  PyPy already inlines as it traces, so it might actually not have any effect, right?
  - easytiger 13y ago
    I think he means instead of using this approach, use pypy
albertzeyer 13y ago
Note: - Currently, it supports only very simple functions (functions with 1 statement). - It is far from trivial to support non-simple functions, or basically any functions which needs its own local variables / its own stack. This is possible via a custom stack object for this purpose, but this all need some extra logic and transformation for the local variable handling. Btw., if you are interested in this, you might also be interested in my project: https://github.com/albertz/PythonHotswap https://github.com/albertz/PythonHotswap It does some similar hacks, i.e. modifies functions, but all on the bytecode level. I.e. no import hook hacks, it uses the code objects and patches them. This makes some things easier and some other things harder.
- masklinn 13y ago
  > basically any functions which needs its own local variables / its own stack. Since cpython's interpreter is stack-based, as long as the function is stack-clean why would it need its own stack? There is a difficulty with local variables though, especially with inlining conflicts (the inlined locals would need to be munged)
  - albertzeyer 13y ago
    On the AST level, you have symbol names, so they become ambiguous. E.g. @inline def f(y): x = y + 1 return x def g(): x = 42 f(0) return x print g() would print 1. When you do it on the bytecode level, many things probably would work, although I don't know whether some things might get confused when there are multiple variables with the same name. Also, accessing `locals()` would (of course) not work.
    - masklinn 13y ago
      Yes but that has nothing to do with the stack, it has to do with namespace conflicts between local variables.
orf 13y ago
Woah, I didn't expect this to get any interest on HNews. This is a major hack that I did during some downtime at work, I think its pretty neat but its entirely useless. I'm glad other people find it interesting!
_ZeD_ 13y ago
if you like to play with the AST, I think you could find useful https://github.com/lihaoyi/macropy https://github.com/lihaoyi/macropy

46 points by albertzeyer 13y ago

lvh 13y ago

This is pretty cool, albeit pretty hacky ;) For what it's worth, you could also try to make your code work on PyPy, which will do this and much much more :)

lucian1900 13y ago
PyPy already inlines as it traces, so it might actually not have any effect, right?
- easytiger 13y ago
  I think he means instead of using this approach, use pypy

albertzeyer 13y ago

Note: - Currently, it supports only very simple functions (functions with 1 statement). - It is far from trivial to support non-simple functions, or basically any functions which needs its own local variables / its own stack. This is possible via a custom stack object for this purpose, but this all need some extra logic and transformation for the local variable handling. Btw., if you are interested in this, you might also be interested in my project: https://github.com/albertz/PythonHotswap https://github.com/albertz/PythonHotswap It does some similar hacks, i.e. modifies functions, but all on the bytecode level. I.e. no import hook hacks, it uses the code objects and patches them. This makes some things easier and some other things harder.

masklinn 13y ago
> basically any functions which needs its own local variables / its own stack. Since cpython's interpreter is stack-based, as long as the function is stack-clean why would it need its own stack? There is a difficulty with local variables though, especially with inlining conflicts (the inlined locals would need to be munged)
- albertzeyer 13y ago
  On the AST level, you have symbol names, so they become ambiguous. E.g. @inline def f(y): x = y + 1 return x def g(): x = 42 f(0) return x print g() would print 1. When you do it on the bytecode level, many things probably would work, although I don't know whether some things might get confused when there are multiple variables with the same name. Also, accessing `locals()` would (of course) not work.
  - masklinn 13y ago
    Yes but that has nothing to do with the stack, it has to do with namespace conflicts between local variables.

orf 13y ago

Woah, I didn't expect this to get any interest on HNews. This is a major hack that I did during some downtime at work, I think its pretty neat but its entirely useless. I'm glad other people find it interesting!

_ZeD_ 13y ago

if you like to play with the AST, I think you could find useful https://github.com/lihaoyi/macropy https://github.com/lihaoyi/macropy