7 ms·

Pypy and Pyston are still quite a bit faster than CPython, but this is a huge improvement in one release. https://www.phoronix.com/review/python311-pyston-pypy

by iruoy 4y ago

Pypy and Pyston are still quite a bit faster than CPython, but this is a huge improvement in one release. https://www.phoronix.com/review/python311-pyston-pypy https://www.phoronix.com/review/python311-pyston-pypy

miohtama 4y ago
Are there any plans to change Python the language to make it faster? AFAIK most of the slowness comes from dynamic overhead, like object attributes may change or disappear in the middle of a loop and so on.
- cdavid 4y ago
  I doubt so, it would break almost any non trivial piece of python. And the ecosystem is a large reason for current python's success. Being dynamic make it harder to be fast, but JS/v8 is as dynamic as python, and much faster.
  - jokoon 4y ago
    If there was enough money, it would be possible to have a v8 equivalent to python.
  - beagle3 4y ago
    But some dynamic features, e.g. “with” make JS and its JITs give up and run slowly. Those features are not in common use. Python’s hard-to-optimize dynamic features are more commonly used.
  - mytherin 4y ago
    The hardest part about optimizing Python is the fact that there are so many packages out there relying on every single documented and undocumented aspect of the C/C++ interface, which is incredibly broad and directly tied to the internals of the language. If they change anything in a non-backwards compatible manner many of those packages will break, and those packages are the reason so many people use Python to begin with. Javascript never had this problem, as all code was always written in Javascript itself by necessity, so it was far easier to optimize as you did not have to worry about backwards compatibility of the internals.
    - cdavid 4y ago
      That's a excellent point. That ecosystem is also one of the main reason packaging in python is a bit of a clusterf*ck. For C/C++ extensions, I think there may be hope to support a slow/emulating C API and a faster, less internals-leaking new API that extensions could adopt. It would take years for the migration, but if speed gains was say 5x, I think it could be realistic. pypy managed to emulate the C API fairly well, after all. E.g. you can build numpy and pandas on top of pypy and it actually kinda works.
pjmlp 4y ago
Someday it will get a proper JIT I guess.
- nigma1337 4y ago
  Planned for 3.12 > Simple "JIT" compiler for small regions. Compile small regions of specialized code, using a relatively simple, fast compiler. https://github.com/markshannon/faster-cpython/blob/master/plan.md https://github.com/markshannon/faster-cpython/blob/master/pl...
  - pjmlp 4y ago
    I guess better than nothing then.
    - driscoll42 4y ago
      When you're dealing with one of the most popular languages in the world and the cost of getting it wrong and breaking things is astronomical, being risk adverse and starting small seems like the way to go.
      - pjmlp 4y ago
        Doesn't seem to hinder JavaScript runtimes.
        Sol- 4y ago
        Hasn't Microsoft experimented with disabling the JIT for security reasons?[1] Doesn't have much relevance for Python at the moment, but I'm just mentioning it to underline that a modern JIT can be very complex thing and keeping it simple for 3.11 seems like a very reasonable requirement. [1] Random google result as source: https://www.cyclonis.com/microsoft-edge-tests-disabling-javascript-jit-compiler-boost-security/ https://www.cyclonis.com/microsoft-edge-tests-disabling-java...
        4y ago
patrec 4y ago
I think python 3.11 has effectively killed off both Pypy and Pyston. Now that the CPython team has finally shown both willingness and ability to deal with performance problems, few people are going to fool around with some esoteric version of python for an increasingly questionable performance-gains/headache ratio. Especially given how painful it already is to package and deploy normal python code and how hostile Guido always has been to alternative implementations. I don't think being maybe 2x faster right now is anywhere good enough to justify the additional risks and hassle, and it looks like the performance gap might shrink further with 3.12.
- smcl 4y ago
  Pyston may be considered estoreric, but Pypy is pretty well-established already and is still a good deal faster. It could be that CPython starts to eat into its user base as it accumulates more performance gains, but Pypy is definitely not done yet.
  - wirrbel 4y ago
    I have been professionally programming Python now since I guess 2012 and its Pypy is an interesting one. Pypy seems to be in use overall in fairly specialized Python application, like research code that is 'too big to rewrite', there are some legacy python applications I heard of successfully running on pypy for years, also you are able to get professional support in onboarding python programs on pypy. So pypy is often used in software that cannot continue to be operated on CPython for performance reasons and rewriting is not feasible / desirable. Fond memories: I did use pypy or a predecessor in like 2004 I think when I took part in a student computer science competition and my algorithm searching for subgraphs wasn't performing well enough to terminate in time.
    - smcl 4y ago
      Ha funny, we have both (ab)used Pypy in exactly the same way :) I had an Advent of Code solution that I implemented horribly in Python, I knew it would spit out the right answer eventually but it was just taking too long. Rather than do a proper rewrite I figured I'd at least try it with Pypy just to see if I could be lazy, and sure enough I got my answer fairly quickly :D

by iruoy 4y ago

miohtama 4y ago

Are there any plans to change Python the language to make it faster? AFAIK most of the slowness comes from dynamic overhead, like object attributes may change or disappear in the middle of a loop and so on.

cdavid 4y ago
I doubt so, it would break almost any non trivial piece of python. And the ecosystem is a large reason for current python's success. Being dynamic make it harder to be fast, but JS/v8 is as dynamic as python, and much faster.
- jokoon 4y ago
  If there was enough money, it would be possible to have a v8 equivalent to python.
- beagle3 4y ago
  But some dynamic features, e.g. “with” make JS and its JITs give up and run slowly. Those features are not in common use. Python’s hard-to-optimize dynamic features are more commonly used.
- mytherin 4y ago
  The hardest part about optimizing Python is the fact that there are so many packages out there relying on every single documented and undocumented aspect of the C/C++ interface, which is incredibly broad and directly tied to the internals of the language. If they change anything in a non-backwards compatible manner many of those packages will break, and those packages are the reason so many people use Python to begin with. Javascript never had this problem, as all code was always written in Javascript itself by necessity, so it was far easier to optimize as you did not have to worry about backwards compatibility of the internals.
  - cdavid 4y ago
    That's a excellent point. That ecosystem is also one of the main reason packaging in python is a bit of a clusterf*ck. For C/C++ extensions, I think there may be hope to support a slow/emulating C API and a faster, less internals-leaking new API that extensions could adopt. It would take years for the migration, but if speed gains was say 5x, I think it could be realistic. pypy managed to emulate the C API fairly well, after all. E.g. you can build numpy and pandas on top of pypy and it actually kinda works.

pjmlp 4y ago

Someday it will get a proper JIT I guess.

nigma1337 4y ago
Planned for 3.12 > Simple "JIT" compiler for small regions. Compile small regions of specialized code, using a relatively simple, fast compiler. https://github.com/markshannon/faster-cpython/blob/master/plan.md https://github.com/markshannon/faster-cpython/blob/master/pl...
- pjmlp 4y ago
  I guess better than nothing then.
  - driscoll42 4y ago
    When you're dealing with one of the most popular languages in the world and the cost of getting it wrong and breaking things is astronomical, being risk adverse and starting small seems like the way to go.
    - pjmlp 4y ago
      Doesn't seem to hinder JavaScript runtimes.
      - Sol- 4y ago
        Hasn't Microsoft experimented with disabling the JIT for security reasons?[1] Doesn't have much relevance for Python at the moment, but I'm just mentioning it to underline that a modern JIT can be very complex thing and keeping it simple for 3.11 seems like a very reasonable requirement. [1] Random google result as source: https://www.cyclonis.com/microsoft-edge-tests-disabling-javascript-jit-compiler-boost-security/ https://www.cyclonis.com/microsoft-edge-tests-disabling-java...
        4y ago

patrec 4y ago

I think python 3.11 has effectively killed off both Pypy and Pyston. Now that the CPython team has finally shown both willingness and ability to deal with performance problems, few people are going to fool around with some esoteric version of python for an increasingly questionable performance-gains/headache ratio. Especially given how painful it already is to package and deploy normal python code and how hostile Guido always has been to alternative implementations. I don't think being maybe 2x faster right now is anywhere good enough to justify the additional risks and hassle, and it looks like the performance gap might shrink further with 3.12.

smcl 4y ago
Pyston may be considered estoreric, but Pypy is pretty well-established already and is still a good deal faster. It could be that CPython starts to eat into its user base as it accumulates more performance gains, but Pypy is definitely not done yet.
- wirrbel 4y ago
  I have been professionally programming Python now since I guess 2012 and its Pypy is an interesting one. Pypy seems to be in use overall in fairly specialized Python application, like research code that is 'too big to rewrite', there are some legacy python applications I heard of successfully running on pypy for years, also you are able to get professional support in onboarding python programs on pypy. So pypy is often used in software that cannot continue to be operated on CPython for performance reasons and rewriting is not feasible / desirable. Fond memories: I did use pypy or a predecessor in like 2004 I think when I took part in a student computer science competition and my algorithm searching for subgraphs wasn't performing well enough to terminate in time.
  - smcl 4y ago
    Ha funny, we have both (ab)used Pypy in exactly the same way :) I had an Advent of Code solution that I implemented horribly in Python, I knew it would spit out the right answer eventually but it was just taking too long. Rather than do a proper rewrite I figured I'd at least try it with Pypy just to see if I could be lazy, and sure enough I got my answer fairly quickly :D