3 ms·

I've found this, that says that python is slow because it doesn't have JIT: https://stackoverflow.com/questions/3033329/why-are-python-programs-often-slower-th

by jokoon 6y ago

I've found this, that says that python is slow because it doesn't have JIT: https://stackoverflow.com/questions/3033329/why-are-python-programs-often-slower-than-the-equivalent-program-written-in-c-or https://stackoverflow.com/questions/3033329/why-are-python-p... I'm curious to understand, why is it so hard to write a JIT runtime for python?

anentropic 6y ago
There are several The most well known is PyPy Pyston appears to be another with a different approach
linspace 6y ago
AFAIK there is Pypy that is much faster. Also, there is Numba. But the problem with Pypy is (was?) interoperability with existing C modules.
- Asraelite 6y ago
  A comment on another HN thread mentioned that the reason for CPython still being non-JIT was the developers wanted the interpreter codebase to remain simple and easy to understand. This sounds like the only reasonable explanation to me. JIT with C interoperability is hard, but it's not that hard.
cygx 6y ago
> I'm curious to understand, why is it so hard to write a JIT runtime for python? First, the CPython reference implementation is supposed to be kept simple. Second, it exposes interpreter implementation details to extension writers, meaning alternative implementations with a radically different architecture might be unable to use some parts of the ecosystem or have to implement costly workarounds (see eg the PyPy FAQ [1]). [1] https://doc.pypy.org/en/latest/faq.html#do-c-extension-modules-work-with-pypy https://doc.pypy.org/en/latest/faq.html#do-c-extension-modul...
- beagle3 6y ago
  https://github.com/microsoft/Pyjion https://github.com/microsoft/Pyjion was a really nice attempt to marry CPython with JIT in a rather generic way, but it was abandoned quickly.
  - pansa2 6y ago
    Development of Pyjion continues here: https://github.com/tonybaloney/Pyjion https://github.com/tonybaloney/Pyjion
- jeltz 6y ago
  Interestingly Ruby decided to write a JIT despite exposing about as much of the interpreter to extension writers. So far their JIT is not very usegul for most real life workloads but I think that is mostly due to lack of development resources rather than any actual limitations.
kzrdude 6y ago
It's certainly hard to write a JIT for Python, but Cinder, Pyston and Pypy have done it. You'd need to look at the CPython project (main Python project) why they don't want to include anything like that. I hope there will be steps taken.
mangecoeur 6y ago
Because much of the power of python comes from its libraries, many of which depend on C(++) extensions tied to details of CPython interpreter. It's very hard to write a JIT without breaking this. For example, all the numerical/scientific stack (numpy etc) is extremely fast by being written C, Fortran, etc with a Python interface. If you break this in a JIT then the plain python might be faster but actual useful code will be much slower. Additionally, one of the reasons C-extensions are so fast is because the CPython implementation is very clean which has made it easier to write stable and performant extensions.

by jokoon 6y ago

anentropic 6y ago

There are several The most well known is PyPy Pyston appears to be another with a different approach

linspace 6y ago

AFAIK there is Pypy that is much faster. Also, there is Numba. But the problem with Pypy is (was?) interoperability with existing C modules.

Asraelite 6y ago
A comment on another HN thread mentioned that the reason for CPython still being non-JIT was the developers wanted the interpreter codebase to remain simple and easy to understand. This sounds like the only reasonable explanation to me. JIT with C interoperability is hard, but it's not that hard.

cygx 6y ago

> I'm curious to understand, why is it so hard to write a JIT runtime for python? First, the CPython reference implementation is supposed to be kept simple. Second, it exposes interpreter implementation details to extension writers, meaning alternative implementations with a radically different architecture might be unable to use some parts of the ecosystem or have to implement costly workarounds (see eg the PyPy FAQ [1]). [1] https://doc.pypy.org/en/latest/faq.html#do-c-extension-modules-work-with-pypy https://doc.pypy.org/en/latest/faq.html#do-c-extension-modul...

beagle3 6y ago
https://github.com/microsoft/Pyjion https://github.com/microsoft/Pyjion was a really nice attempt to marry CPython with JIT in a rather generic way, but it was abandoned quickly.
- pansa2 6y ago
  Development of Pyjion continues here: https://github.com/tonybaloney/Pyjion https://github.com/tonybaloney/Pyjion
jeltz 6y ago
Interestingly Ruby decided to write a JIT despite exposing about as much of the interpreter to extension writers. So far their JIT is not very usegul for most real life workloads but I think that is mostly due to lack of development resources rather than any actual limitations.

kzrdude 6y ago

It's certainly hard to write a JIT for Python, but Cinder, Pyston and Pypy have done it. You'd need to look at the CPython project (main Python project) why they don't want to include anything like that. I hope there will be steps taken.

mangecoeur 6y ago

Because much of the power of python comes from its libraries, many of which depend on C(++) extensions tied to details of CPython interpreter. It's very hard to write a JIT without breaking this. For example, all the numerical/scientific stack (numpy etc) is extremely fast by being written C, Fortran, etc with a Python interface. If you break this in a JIT then the plain python might be faster but actual useful code will be much slower. Additionally, one of the reasons C-extensions are so fast is because the CPython implementation is very clean which has made it easier to write stable and performant extensions.