4 ms·

I think python 3.11 has effectively killed off both Pypy and Pyston. Now that the CPython team has finally shown both willingness and ability to deal with perfo

by patrec 4y ago

I think python 3.11 has effectively killed off both Pypy and Pyston. Now that the CPython team has finally shown both willingness and ability to deal with performance problems, few people are going to fool around with some esoteric version of python for an increasingly questionable performance-gains/headache ratio. Especially given how painful it already is to package and deploy normal python code and how hostile Guido always has been to alternative implementations. I don't think being maybe 2x faster right now is anywhere good enough to justify the additional risks and hassle, and it looks like the performance gap might shrink further with 3.12.

smcl 4y ago
Pyston may be considered estoreric, but Pypy is pretty well-established already and is still a good deal faster. It could be that CPython starts to eat into its user base as it accumulates more performance gains, but Pypy is definitely not done yet.
- wirrbel 4y ago
  I have been professionally programming Python now since I guess 2012 and its Pypy is an interesting one. Pypy seems to be in use overall in fairly specialized Python application, like research code that is 'too big to rewrite', there are some legacy python applications I heard of successfully running on pypy for years, also you are able to get professional support in onboarding python programs on pypy. So pypy is often used in software that cannot continue to be operated on CPython for performance reasons and rewriting is not feasible / desirable. Fond memories: I did use pypy or a predecessor in like 2004 I think when I took part in a student computer science competition and my algorithm searching for subgraphs wasn't performing well enough to terminate in time.
  - smcl 4y ago
    Ha funny, we have both (ab)used Pypy in exactly the same way :) I had an Advent of Code solution that I implemented horribly in Python, I knew it would spit out the right answer eventually but it was just taking too long. Rather than do a proper rewrite I figured I'd at least try it with Pypy just to see if I could be lazy, and sure enough I got my answer fairly quickly :D
- patrec 4y ago
  Pyston is already at the "our incredible journey" stage ("We’re very excited about these changes and [...] Marius and I (Kevin) are planning to gradually reduce our time investment in the project"). Pypy, as a practical software deployment runtime has been and will remain esoteric (I do absolutely think that they had a positive impact on the wider python community, both in terms of dissemination of ideas and also practical engineering artifacts). But what's their market share relative to CPython? A thousandth of a percent or less? Has anyone actually built a significant business on top of Pypy? There is, IMO no realistic path at that point that Pypy could become a viable CPython alternative. They are effectively competing with a hostile platform they need to maintain an extremely high amount of compatibility with, and that can and does move in directions that invalidate some fundamental engineering choices they make. Practical end results include that they're stuck w/ a lot of crippling design decisions (GIL, FFI API etc.) and a core part that is still in 2.7 land (RPython) and have historically mostly only been compatible to very outdated versions of python. This has improved a lot, but the next time CPython throws another curve ball the same thing is bound to happen again. The only chance they really had was to be compellingly enough faster or otherwise superior that community pressure would have forced the CPython team into adopting a much more collaborative stance. That seems very unlikely to happen, now after all this time, given that now CPython is catching up and they have the albatross of the C extension ecosystem around their neck. Almost anyone who cares about python performance outside of algorithmic programming competitions will be using C extensions where Pypy offers no compelling advantage, some disadvantages, and by the momentum of the existing eco-system is unable to develop a superior alternative.
  - jszymborski 4y ago
    At least among the scientific community, PyPy's FFI limitations are the main stumbling block. I'm not sure the other factors dissuade much from its adoption in that community (people have just started migrating to 3.6 from 2.7).
    - MiguelX413 4y ago
      I love Pyo3 because of its PyPy compatibility
deleted 4y ago
[deleted]
dragonwriter 4y ago
> how hostile Guido always has been to alternative implementations. Isn’t mypyc effectively an alternative (AOT-compiled) Python implementation? Guido doesn’t seem too hostile too it.
__mharrison__ 4y ago
Incidentally, I was working on a 15 year old Python 2 project for a client last week that used Pysco (the predecessor for PyPy). The cool thing about Pysco was that you could just import the library and it would make many operations much faster. If PyPy had a similar mode where you could load it as a library it would have a much easier time gaining traction.
- alanfranz 4y ago
  psyco!

by patrec 4y ago

smcl 4y ago

Pyston may be considered estoreric, but Pypy is pretty well-established already and is still a good deal faster. It could be that CPython starts to eat into its user base as it accumulates more performance gains, but Pypy is definitely not done yet.

wirrbel 4y ago
I have been professionally programming Python now since I guess 2012 and its Pypy is an interesting one. Pypy seems to be in use overall in fairly specialized Python application, like research code that is 'too big to rewrite', there are some legacy python applications I heard of successfully running on pypy for years, also you are able to get professional support in onboarding python programs on pypy. So pypy is often used in software that cannot continue to be operated on CPython for performance reasons and rewriting is not feasible / desirable. Fond memories: I did use pypy or a predecessor in like 2004 I think when I took part in a student computer science competition and my algorithm searching for subgraphs wasn't performing well enough to terminate in time.
- smcl 4y ago
  Ha funny, we have both (ab)used Pypy in exactly the same way :) I had an Advent of Code solution that I implemented horribly in Python, I knew it would spit out the right answer eventually but it was just taking too long. Rather than do a proper rewrite I figured I'd at least try it with Pypy just to see if I could be lazy, and sure enough I got my answer fairly quickly :D
patrec 4y ago
Pyston is already at the "our incredible journey" stage ("We’re very excited about these changes and [...] Marius and I (Kevin) are planning to gradually reduce our time investment in the project"). Pypy, as a practical software deployment runtime has been and will remain esoteric (I do absolutely think that they had a positive impact on the wider python community, both in terms of dissemination of ideas and also practical engineering artifacts). But what's their market share relative to CPython? A thousandth of a percent or less? Has anyone actually built a significant business on top of Pypy? There is, IMO no realistic path at that point that Pypy could become a viable CPython alternative. They are effectively competing with a hostile platform they need to maintain an extremely high amount of compatibility with, and that can and does move in directions that invalidate some fundamental engineering choices they make. Practical end results include that they're stuck w/ a lot of crippling design decisions (GIL, FFI API etc.) and a core part that is still in 2.7 land (RPython) and have historically mostly only been compatible to very outdated versions of python. This has improved a lot, but the next time CPython throws another curve ball the same thing is bound to happen again. The only chance they really had was to be compellingly enough faster or otherwise superior that community pressure would have forced the CPython team into adopting a much more collaborative stance. That seems very unlikely to happen, now after all this time, given that now CPython is catching up and they have the albatross of the C extension ecosystem around their neck. Almost anyone who cares about python performance outside of algorithmic programming competitions will be using C extensions where Pypy offers no compelling advantage, some disadvantages, and by the momentum of the existing eco-system is unable to develop a superior alternative.
- jszymborski 4y ago
  At least among the scientific community, PyPy's FFI limitations are the main stumbling block. I'm not sure the other factors dissuade much from its adoption in that community (people have just started migrating to 3.6 from 2.7).
  - MiguelX413 4y ago
    I love Pyo3 because of its PyPy compatibility

deleted 4y ago

[deleted]

dragonwriter 4y ago

> how hostile Guido always has been to alternative implementations. Isn’t mypyc effectively an alternative (AOT-compiled) Python implementation? Guido doesn’t seem too hostile too it.

__mharrison__ 4y ago

Incidentally, I was working on a 15 year old Python 2 project for a client last week that used Pysco (the predecessor for PyPy). The cool thing about Pysco was that you could just import the library and it would make many operations much faster. If PyPy had a similar mode where you could load it as a library it would have a much easier time gaining traction.

alanfranz 4y ago
psyco!