5 ms·

I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Pytho

I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset of that language with C-API compatibility?

CJefferson 4y ago
The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -- Javascript has most of the nasty things Python does, and it's plenty fast on browsers. However, maintaining C-API compatibility means you need to set up lots of data structures exactly how the C API requires, and maintaining and updating those ends up losing you lots of your benefits of JITing. You could, hypothetically, introduce an entirely new API, which allowed for faster dynamic recompiling, but then you'd need to get every package anyone cares about to switch to that.
- rustybolt 4y ago
  Sorry for the ignorance, but what do you mean by C-API here? Normally I'd say you mean the interface you use when you call native machine code from Python, but I don't see how this would slow things down.
  - radicalbyte 4y ago
    The interface to the C language. It is what makes Python fast - you write the code which needs to be fast in optimised C and call into it with Python. The Python code is then basically just the glue.
  - pdpi 4y ago
    That native code still needs to be able to interact with your Python objects somehow. You can't change the API around PyObject without forcing all C libraries to make changes on their side, and that API forces you to expose things a certain way.
- kaba0 4y ago
  That’s why I found GraalVM’s approach intriguing. They provide a high level language API where you can simply write a language interpreter, and it will be able to JIT/AOT compile it down to fast machine code. But the most interesting aspect is that the IR they convert languages to is basically universal, so something like LLVM bitcode can also use the exact same representations. So you can interpret (and later AOT compile as well) LLVM bitcode and python, and this approach will allow cross-language optimizations as well, which were not available at all before. But feel free to add a bit of JS/Java, etc to your code as well!
- vlovich123 4y ago
  I really try hard to understand this argument and I must be missing something and must be super stupid. Don’t languages like JavaScript have this and yet they can still do JIT and the base runtime is still in C++? Java itself has an official way to invoke C programs from Java applications and still has a JIT. And Java also has AOT compilers. Sure. Crossing that FFI boundary is going to be expensive. But there’s lots of techniques to mitigate it or even in the limit eliminate it. if I recall correctly you can JIT a fast call that knows how to invoke the FFI directly without the extra indirection layer. Basically a fancy runtime LTO. I think a huge part of it is CPython’s interest in keeping the core codebase as simple as possible which seems to be the overriding reason for why the global lock still hasn’t been removed (which iirc even Ruby pulled off at some point). Also the reason there’s no JIT afaict and why Pypy got started to prove it is possible to JIT (and frequently sees substantial gains vs cpython). The problem they’ve had is that CPython is a moving target and it’s hard to keep a parallel runtime up to date on a shoestring amount of funding. That’s why you see alternate approaches like numba (JIT’ed Python) which are less of a departure and Cinder (better budget). To me this seems like a CPython project actively hostile to JIT than C data structures meaning you lose some benefit to FFI overhead. Performance is a virtuous cycle too - when there’s enthusiasm about a language you get more and more people paid to make your language fast. For a while companies tried. Google gave up. Facebook only has it as a fork with a public plea for the maintainers of CPython to mainline literally anything. The CPython maintainers feel like the biggest obstacle. No?
  - pjmlp 4y ago
    The difference is that the other languages FFI don't expose internals like CPython does. For example, JNI only exposes handles and you need to convert an handle to a pointer, so the runtime knows for the time being that handle is special and being used by native code. When it is only an opaque handle, lots of optimizations can happen and the native code won't see them.
    - vlovich123 4y ago
      Doesn’t PyPy accomplish it via CPyExt? It sounds like Cinder, Instagram’s version is CPython+JIT (among other things). I haven’t looked at the details so maybe it’s not a sufficient speed up and that’s why all these parallel efforts haven’t been merged? The part I’m missing is how what you said makes it intractable when we have counter examples within and without the language. Sure. Maybe some optimizations aren’t possible. But that’s a world of difference from little to no benefit and impossible. Don’t get me wrong. I’m not passing a value judgement on the maintainers. But the reasons don’t feel technical to me.
klooney 4y ago
This is Pypy and Rpython!
KyeRussell 4y ago
Django makes use of a whole lot of Python’s fancypants stuff. For this reason, for instance, mypy doesn’t do well on Django projects without a purpose-built plugin. But I still take your point.
- LtWorf 4y ago
  pydantic also requires a mypy plugin… But it's just how it was designed. I designed typedload with mypy in mind, so it kinda works (except for some limitations in the type system that don't allow to express some things, as of now).
formerly_proven 4y ago
Most web stuff, but Django especially, relies heavily on dynamic Python magic to make cuter APIs.
hgomersall 4y ago
That would be things like cython (https://cython.org/ https://cython.org/) and rpython (https://rpython.readthedocs.io/en/latest/ https://rpython.readthedocs.io/en/latest/).
LtWorf 4y ago
> Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Yes. FastAPI depends on really slow pydantic (disclaimer: I'm the author of the faster typedload). All those dynamic typechecking modules rely on the dynamic nature of the language. The alternative would be to having to generate code at compile time instead. pydantic is also in the process of being rewritten in rust to be not so slow any longer, and in the process it will become incompatible with anything else than cpython (the normal python runtime). Which in turns means fastapi won't be able to run on anything else (unless they decouple from pydantic… which probably won't be easy).
dagw 4y ago
Can't we have a faster compiler for a subset of Python? Check out Pythran, that is exactly what they've done.
- wageslave99 4y ago
  Thanks! Sounds interesting! https://pythran.readthedocs.io/en/latest/ https://pythran.readthedocs.io/en/latest/
crabbone 4y ago
> Can't we have a faster compiler for a subset of Python? That's exactly how PyPy works.
- thunky 4y ago
  PyPy isn't really a subset of Python. From pypy.org: PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.8 and 3.9
  - acdha 4y ago
    Yes - I think they meant RPython, which the PyPy team developed explicitly as the easy to optimize safer subset of Python.

CJefferson 4y ago

The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -- Javascript has most of the nasty things Python does, and it's plenty fast on browsers. However, maintaining C-API compatibility means you need to set up lots of data structures exactly how the C API requires, and maintaining and updating those ends up losing you lots of your benefits of JITing. You could, hypothetically, introduce an entirely new API, which allowed for faster dynamic recompiling, but then you'd need to get every package anyone cares about to switch to that.

rustybolt 4y ago
Sorry for the ignorance, but what do you mean by C-API here? Normally I'd say you mean the interface you use when you call native machine code from Python, but I don't see how this would slow things down.
- radicalbyte 4y ago
  The interface to the C language. It is what makes Python fast - you write the code which needs to be fast in optimised C and call into it with Python. The Python code is then basically just the glue.
- pdpi 4y ago
  That native code still needs to be able to interact with your Python objects somehow. You can't change the API around PyObject without forcing all C libraries to make changes on their side, and that API forces you to expose things a certain way.
kaba0 4y ago
That’s why I found GraalVM’s approach intriguing. They provide a high level language API where you can simply write a language interpreter, and it will be able to JIT/AOT compile it down to fast machine code. But the most interesting aspect is that the IR they convert languages to is basically universal, so something like LLVM bitcode can also use the exact same representations. So you can interpret (and later AOT compile as well) LLVM bitcode and python, and this approach will allow cross-language optimizations as well, which were not available at all before. But feel free to add a bit of JS/Java, etc to your code as well!
vlovich123 4y ago
I really try hard to understand this argument and I must be missing something and must be super stupid. Don’t languages like JavaScript have this and yet they can still do JIT and the base runtime is still in C++? Java itself has an official way to invoke C programs from Java applications and still has a JIT. And Java also has AOT compilers. Sure. Crossing that FFI boundary is going to be expensive. But there’s lots of techniques to mitigate it or even in the limit eliminate it. if I recall correctly you can JIT a fast call that knows how to invoke the FFI directly without the extra indirection layer. Basically a fancy runtime LTO. I think a huge part of it is CPython’s interest in keeping the core codebase as simple as possible which seems to be the overriding reason for why the global lock still hasn’t been removed (which iirc even Ruby pulled off at some point). Also the reason there’s no JIT afaict and why Pypy got started to prove it is possible to JIT (and frequently sees substantial gains vs cpython). The problem they’ve had is that CPython is a moving target and it’s hard to keep a parallel runtime up to date on a shoestring amount of funding. That’s why you see alternate approaches like numba (JIT’ed Python) which are less of a departure and Cinder (better budget). To me this seems like a CPython project actively hostile to JIT than C data structures meaning you lose some benefit to FFI overhead. Performance is a virtuous cycle too - when there’s enthusiasm about a language you get more and more people paid to make your language fast. For a while companies tried. Google gave up. Facebook only has it as a fork with a public plea for the maintainers of CPython to mainline literally anything. The CPython maintainers feel like the biggest obstacle. No?
- pjmlp 4y ago
  The difference is that the other languages FFI don't expose internals like CPython does. For example, JNI only exposes handles and you need to convert an handle to a pointer, so the runtime knows for the time being that handle is special and being used by native code. When it is only an opaque handle, lots of optimizations can happen and the native code won't see them.
  - vlovich123 4y ago
    Doesn’t PyPy accomplish it via CPyExt? It sounds like Cinder, Instagram’s version is CPython+JIT (among other things). I haven’t looked at the details so maybe it’s not a sufficient speed up and that’s why all these parallel efforts haven’t been merged? The part I’m missing is how what you said makes it intractable when we have counter examples within and without the language. Sure. Maybe some optimizations aren’t possible. But that’s a world of difference from little to no benefit and impossible. Don’t get me wrong. I’m not passing a value judgement on the maintainers. But the reasons don’t feel technical to me.

klooney 4y ago

This is Pypy and Rpython!

KyeRussell 4y ago

Django makes use of a whole lot of Python’s fancypants stuff. For this reason, for instance, mypy doesn’t do well on Django projects without a purpose-built plugin. But I still take your point.

LtWorf 4y ago
pydantic also requires a mypy plugin… But it's just how it was designed. I designed typedload with mypy in mind, so it kinda works (except for some limitations in the type system that don't allow to express some things, as of now).

formerly_proven 4y ago

Most web stuff, but Django especially, relies heavily on dynamic Python magic to make cuter APIs.

hgomersall 4y ago

That would be things like cython (https://cython.org/ https://cython.org/) and rpython (https://rpython.readthedocs.io/en/latest/ https://rpython.readthedocs.io/en/latest/).

LtWorf 4y ago

> Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Yes. FastAPI depends on really slow pydantic (disclaimer: I'm the author of the faster typedload). All those dynamic typechecking modules rely on the dynamic nature of the language. The alternative would be to having to generate code at compile time instead. pydantic is also in the process of being rewritten in rust to be not so slow any longer, and in the process it will become incompatible with anything else than cpython (the normal python runtime). Which in turns means fastapi won't be able to run on anything else (unless they decouple from pydantic… which probably won't be easy).

dagw 4y ago

Can't we have a faster compiler for a subset of Python? Check out Pythran, that is exactly what they've done.

wageslave99 4y ago
Thanks! Sounds interesting! https://pythran.readthedocs.io/en/latest/ https://pythran.readthedocs.io/en/latest/

crabbone 4y ago

> Can't we have a faster compiler for a subset of Python? That's exactly how PyPy works.

thunky 4y ago
PyPy isn't really a subset of Python. From pypy.org: PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.8 and 3.9
- acdha 4y ago
  Yes - I think they meant RPython, which the PyPy team developed explicitly as the easy to optimize safer subset of Python.