4 ms·

Python have a 99% compatible jit implementation called PyPy and nobody much know about it. It was production ready since 5 years ago but ignored by mainstream.

by v3ss0n 4y ago

Python have a 99% compatible jit implementation called PyPy and nobody much know about it. It was production ready since 5 years ago but ignored by mainstream. It is 400% faster than vanilla python on average and Guido and MS is working on for a few percent gain instead of promoting it or eventually making it default. PyPy.org

deleted 4y ago
[deleted]
Alir3z4 4y ago
Even if GvR promotes it fully, it won't be adopted unless c extensions work out of the box. Python without its crazy amount of extensions in c-api won't survive.
- whizzter 4y ago
  This is something that's up to the community really. The PyPy developers early on made a new FFI that is also available for CPython, sadly not entirely as easy as the classic API but completely functional and if there was a push to rewrite extensions to use that FFI (or provide dual-paths) then the gap could be bridged.
  - theLiminator 4y ago
    Are you referencing https://github.com/hpyproject/hpy https://github.com/hpyproject/hpy? I do hope it takes off.
- v3ss0n 4y ago
  They didn't update their website often , that is the main problem of people not knowing much about PyPy . But they had made CPyext compatible and works fine with several Data-science Libs over past 2-3 years . Even anaconda have a miniconda for pypy called pypyforge where many of CPyext are building fine with it and they have same performance comapre to CPython. ( Can't make C-Extension faster since they are already C anyways.)
hawski 4y ago
I wonder how much of it is related to how Python is being used for short running scripts more than Ruby, at least in my experience. In such cases startup time is more important than being faster in a long run.
Twirrim 4y ago
pypy is pretty well publicised and known about in the python community. Lots of popular libraries specifically call out compatibility etc. For quite a while it used to have some terrible performance if you ever reached out to C libraries in a hot loop. It's still a bit iffy on the C / FFI front, though the performance issues are much improved It's certainly good enough for production use, with careful evaluation.
- v3ss0n 4y ago
  Agreed, cpyext performance getting much better these days, still a little bit slower but general python performance improvement is massive. Non data science project get massive performance boost,for example web framework like Django.
eigenvalue 4y ago
I've tried multiple times to use PyPy on real projects and it was totally unusable if you rely on stuff like pandas, sklearn, keras, fastapi, etc. I found similar issues with Numba as well. These things only work if you keep to a relatively small subset of Python and stick to "pure python" functions. Which basically no one does (that I can see) except in toy projects and tutorials. Improving cpython by 1% (without breaking compatibility, which python 3.10+ did a poor job of in my view) would do more for the world than making PyPy even faster.
deleted 4y ago
[deleted]
int_19h 4y ago
"On average" is mostly meaningless because it depends so much on the kind of code you use it with that you need to do your own measurements to judge. I've seen a 30% improvement on one project, 200% on another, and none at all on yet another.

by v3ss0n 4y ago

deleted 4y ago

[deleted]

Alir3z4 4y ago

Even if GvR promotes it fully, it won't be adopted unless c extensions work out of the box. Python without its crazy amount of extensions in c-api won't survive.

whizzter 4y ago
This is something that's up to the community really. The PyPy developers early on made a new FFI that is also available for CPython, sadly not entirely as easy as the classic API but completely functional and if there was a push to rewrite extensions to use that FFI (or provide dual-paths) then the gap could be bridged.
- theLiminator 4y ago
  Are you referencing https://github.com/hpyproject/hpy https://github.com/hpyproject/hpy? I do hope it takes off.
v3ss0n 4y ago
They didn't update their website often , that is the main problem of people not knowing much about PyPy . But they had made CPyext compatible and works fine with several Data-science Libs over past 2-3 years . Even anaconda have a miniconda for pypy called pypyforge where many of CPyext are building fine with it and they have same performance comapre to CPython. ( Can't make C-Extension faster since they are already C anyways.)

hawski 4y ago

I wonder how much of it is related to how Python is being used for short running scripts more than Ruby, at least in my experience. In such cases startup time is more important than being faster in a long run.

Twirrim 4y ago

pypy is pretty well publicised and known about in the python community. Lots of popular libraries specifically call out compatibility etc. For quite a while it used to have some terrible performance if you ever reached out to C libraries in a hot loop. It's still a bit iffy on the C / FFI front, though the performance issues are much improved It's certainly good enough for production use, with careful evaluation.

v3ss0n 4y ago
Agreed, cpyext performance getting much better these days, still a little bit slower but general python performance improvement is massive. Non data science project get massive performance boost,for example web framework like Django.

eigenvalue 4y ago

I've tried multiple times to use PyPy on real projects and it was totally unusable if you rely on stuff like pandas, sklearn, keras, fastapi, etc. I found similar issues with Numba as well. These things only work if you keep to a relatively small subset of Python and stick to "pure python" functions. Which basically no one does (that I can see) except in toy projects and tutorials. Improving cpython by 1% (without breaking compatibility, which python 3.10+ did a poor job of in my view) would do more for the world than making PyPy even faster.

deleted 4y ago

[deleted]

int_19h 4y ago

"On average" is mostly meaningless because it depends so much on the kind of code you use it with that you need to do your own measurements to judge. I've seen a 30% improvement on one project, 200% on another, and none at all on yet another.