9 ms·

Faster CPython Ideas – Issue Tracker for Faster CPython Project

226 points by patrick91 5y ago

patrick91 5y ago
Presentation from Guido van Rossum at the Python Language Summit: https://github.com/faster-cpython/ideas/blob/main/FasterCPythonDark.pdf https://github.com/faster-cpython/ideas/blob/main/FasterCPyt...
- antman 5y ago
  Comments on the points being made based on my experience - without breaking anyone's code (did that for the print function etc and delayed py3 adoption for a decade but not for the most user requested feature which is speed) - No large PRs (how a 5x speedup can take place with small patches that no one figured out yet?) - a small team (not enough money, less risky if they tried to adopt other pypy-esque projects that are more advanced speed wise, a more comprehensive plan that would invite donations, I would certainly give. For this presentation I don't know)
  - uluyol 5y ago
    WRT large PRs, it doesn't sound like they expect all the improvements to be from small changes. I think the point is the code will be changed in increments so that even large changes (overall) will be easier to review and offer feedback.
  - kzrdude 5y ago
    I don't think "without breaking anyone's code" even needs to be a goal. Python has rolling deprecations and feature removals planned in the 3.x release series, and careful such feature transitions could be used to help JIT features or similar, too. To back up that fact: Python 3.10 will remove long-deprecated features, see issue tracker: https://bugs.python.org/issue41165 https://bugs.python.org/issue41165 and What's new, the Removed section: https://docs.python.org/3.10/whatsnew/3.10.html#removed https://docs.python.org/3.10/whatsnew/3.10.html#removed
  - yjftsjthsd-h 5y ago
    > without breaking anyone's code (did that for the print function etc and delayed py3 adoption for a decade but not for the most user requested feature which is speed) The Python 3 debacle is a beautifully worked example of why being backwards compatible is so important, and I can 100% sympathize with them not wanting to go through that again.
- axaxs 5y ago
  Well, glad he's coming around. Years back he seemed to stand by the notion that Python wasn't slow, showing a toy example. I remember thinking it was just condescending, whether intentional or not.
  - takeda 5y ago
    I think years ago things were much different, also even today that might still be true. Among interpreted languages python is/was one of the faster ones. NodeJS has similar speed, and in some cases is faster. I don't know any other interpreted language though. People are also complaining about GIL, but ironically NodeJS is single threaded.
    - qsort 5y ago
      Nodejs is much faster than Python even for workloads where there's overlap in use-cases, like webapis and scripting. The only production language with performance in the ballpark of Python is Ruby. I agree that more often than not it simply isn't an issue, but CPython performance isn't much far ahead of toy languages, for a whole lot of reasons that we are all sick and tired of hearing.
milliams 5y ago
I posted the link to the first PEP yesterday (https://news.ycombinator.com/item?id=27134290 https://news.ycombinator.com/item?id=27134290) I think this looks like a very promising project.
- qteax 5y ago
  25-50% has been hoped for many times in the past 3 decades. I don't find that very impressive. CPython got big because of its relatively decent C-API and glue capabilities. 25% does not make a difference, decent C extensions have speedups in the order of 10-100 times (yes, times, not percent). If I had a dollar for every time someone proposed a Python speedup ...
  - pjmlp 5y ago
    Python is feeling the pressure of not having a JIT on the box. Microsoft is reactivating their JIT project for Python as well. Their talk tomorrow: "Talk: Restarting Pyjion, a general purpose JIT for Python – is it worth it?" https://us.pycon.org/2021/schedule/presentation/52/ https://us.pycon.org/2021/schedule/presentation/52/
    - aztec100 5y ago
      There have been many similar talks in the past decades. At some point Unladen Swallow was presented as "Google's project" (which is quite exaggerated). It looks more like Microsoft has JIT envy now that Instagram and others have open sourced (quite restricted) JIT projects and has to show presence again. I think every CPython JIT project will be full of corner cases in both performance and behavior, so it will add to the growing database of weird Python behavior that users have to memorize.
      - pjmlp 5y ago
        Indeed, it all boils down to the way Python community doesn't embrace such projects. Usually Python's high dynamism comes into the picture as main reason, however Smalltalk and SELF are just as dynamic, you can change the whole image at any given point in execution. In Smalltalk it suffices to send a become: message to change the complete meaning of an object, and references to it, across the whole image.
punnerud 5y ago
Deleted. Posted the wrong place. Thanks for the comments.
- Jiejeing 5y ago
  You probably meant to send those links there: https://news.ycombinator.com/item?id=27127387 https://news.ycombinator.com/item?id=27127387
  - BerislavLopac 5y ago
    pyodide != pydiode :D
kzrdude 5y ago
This is exciting! So that we don't miss the main point: Eric Snow, GvR and Mark Shannon will be working for Microsoft to improve Python performance. In a way, it seems like Mark Shannon found someone to take him up on his offer on a plan for speeding up Python.
- rqst 5y ago
  Why do people in the Python community get so excited over announcements? I'd rather see working code, but no one cares about that in the Python universe. It is always announcements, talks, conferences and if something emerges it is a bit weird like the pattern matching. Meanwhile the Erlang people quietly produced a JIT without any advertisements.
  - BiteCode_dev 5y ago
    > Why do people in the Python community get so excited over announcements? When you like a tech, knowing people are brewing improved perfs on it is kinky. On Python, the motto has always been "it's fast enough", "if you want perfs, don't use python", "C extensions will solve this", "we don't want to make the main implementation complicated", "python dynamism and GIL make it a hard problem", etc. So in the python world, it's particularly big news, especially given that previous attempts (gilectomy, unladen swallow, first pyston...) all died.
    - WesolyKubeczek 5y ago
      > So in the python world, it's particularly big news, especially given that previous attempts (gilectomy, unladen swallow, first pyston...) all died. This is precisely the point you seem to be missing. All those things you mentioned were similar big announcements back in their day, and they all have just died by fizzling. What should be setting this one apart?
      - BiteCode_dev 5y ago
        Because Guido is paid to work on it.
fdej 5y ago
I'd like to see optimizations targeting ctypes, or a successor to ctypes. I wish I could write elegant, performant C wrappers in pure Python. Right now the best choices are Cython, which is a hassle (separate, slow compilation, various warts), and ctypes, which is slow (and has some design problems of its own). Julia's ccall alone is a major selling point over Python right now.
- BiteCode_dev 5y ago
  It's not nuitka main purpose, but it can produce a lib and speed up the code: https://nuitka.net/ https://nuitka.net/
- ihnorton 5y ago
  CFFI [1] is a step in the right direction, inspired by LuaJIT's CFFI. It originated from the PyPy folks and is supported in PyPy [2]; it's also supported to some degree by Numba [3]. I don't know what level of C call optimization is available when using the JITs, so I can't speak to the performance, but I've used it casually via CPython and was impressed by the API. That said, it has been around for a while and the traction seems somewhat limited -- I would guess because most people who have this kind of problem also need more than "just" FFI. [1] https://cffi.readthedocs.io/en/latest/index.html https://cffi.readthedocs.io/en/latest/index.html [2] https://doc.pypy.org/en/latest/extending.html#cffi https://doc.pypy.org/en/latest/extending.html#cffi [3] https://numba.readthedocs.io/en/stable/reference/pysupported.html#cffi-support https://numba.readthedocs.io/en/stable/reference/pysupported...
- adsharma 5y ago
  Please take a look at py2many which does this. Looking for feedback.
  - adsharma 5y ago
    There is now a pull request for this: https://github.com/adsharma/py2many/issues/62 https://github.com/adsharma/py2many/issues/62
- adsharma 5y ago
sergiomattei 5y ago
This is the most exciting development I've seen in the Python scene in years! Thanks, Guido and team!
bla3 5y ago
It's good to see that python startup time is an explicit goal. Empty scripts taking 10-100ms is a problem if you use python in Makefiles, where you can have thousands of invocations in a build. This wasn't considered a use case upstream cared about for quite some time. I'm glad to see this is changing. Here's hoping the acknowledgement will result in concrete gains!
- dongping 5y ago
  Maybe you would want to check out Bazel as a build tool, since it uses a stripped down variant of Python called Starlark. https://docs.bazel.build/versions/master/skylark/language.html https://docs.bazel.build/versions/master/skylark/language.ht...
  - bla3 5y ago
    It uses that for its build configuration, not for build steps, as far as I know.
pca006132 5y ago
Just wondering, why is CPython a lot slower than JS? It seems to me that both languages are interpreted and comes with some reflection functionalities (like modifying object methods), but JS seems a lot faster in most of the benchmarks in https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Edit: Why is this being downvoted? Is my statement incorrect or offensive to some people?
- pansa2 5y ago
  > It seems to me that both languages are interpreted Node.js uses the V8 engine for JavaScript which isn't just an interpreter - it includes a JIT compiler.
  - dec0dedab0de 5y ago
    Yes, the JIT has got to be the biggest difference. I wonder how node compares to pypy. I think when GVR mentions "There's machine code generation in our future" that he is talking about a JIT
    - throwaway894345 5y ago
      I suspect Node is still quite a lot faster than Pypy because the former benefits from v8 which is a well-funded project which doesn’t have to worry about compatibility with the sprawling C-extension interface that CPython exposes.
- ForHackernews 5y ago
  I think mostly because Google has put a massive amount money and effort into making V8 faster. Performance has always been a lower-tier priority for cPython and other, faster, implementations (like PyPy) have not seen much mainstream adoption.
- epidemian 5y ago
  JS have been heavily optimized by browser vendors, fueled by the intense competition on the browser space, and the huge resources invested by companies like Google. IIRC Firefox 3 was the first browser to have what we would call a modern JS engine, with heavy emphasis on JIT compiling. Google Chrome soon followed and V8 dominated the JS perf story. Python doesn't have such fierce competition of implementations. And besides, it has the "escape hatch" of being able to implement performance-critical code paths as C extensions, which is what heavy-lifting libraries like numpy do.
ForHackernews 5y ago
Is this going to incorporate the work done as part of https://blog.pyston.org/2020/10/28/pyston-v2-20-faster-python/ https://blog.pyston.org/2020/10/28/pyston-v2-20-faster-pytho... or https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder ?
The_rationalist 5y ago
The python JIT that promise to get the highest throughput once maturity reached is https://github.com/oracle/graalpython https://github.com/oracle/graalpython
etaioinshrdlu 5y ago
I updated a large Django project from python2.7 to 3.9 recently. Afterward, I was pleased that the app was both faster and used less than half the memory. It was somewhat surprising that the new version was so much better - most software seems to typically get only heavier over time! So, I take this as a good sign that progress will continue.
- guggle 5y ago
  > most software seems to typically get only heavier over time! It's often the case for end-user applications, but I find that most languages/runtimes tend to get better with time (and work of course !). There's a similar trend with the PHP runtime.
  - MaxBarraclough 5y ago
    That rule of thumb applies to the web too. Browsers keep getting faster, websites keep getting slower.
  - cogman10 5y ago
    I can't think of a language/runtime that has actually gotten slower with time. The nearest I can really point to is the uptick in compile time that happens with languages like Rust/C++ every so often. That's not really a "language is slower" problem though.
    - kzrdude 5y ago
      rustc's trend the last twenty releases has been that the compiler is getting faster. I guess every dependency is slowly growing in size, offsetting this in your perception? The compiler benchmarks are of course held constant to be able to compare releases. See the perf website https://perf.rust-lang.org/dashboard.html https://perf.rust-lang.org/dashboard.html
- 1vuio0pswjnm7 5y ago
  But isn't the Python install getting heavier. 3 packages will be installed: expat-2.3.0_1 gdbm-1.19_1 python-2.7.18_3 Size required on disk: 18MB 3 packages will be installed: expat-2.3.0_1 gdbm-1.19_1 python3-3.9.4_1 Size required on disk: 22MB
etskinner 5y ago
Why isn't this hosted at https://github.com/python/cpython/issues https://github.com/python/cpython/issues ?
- _bohm 5y ago
  Presumably development will be taking place on this organization's fork of cpython? https://github.com/faster-cpython/cpython https://github.com/faster-cpython/cpython Looks like they've also got a repo with some tools for profiling bytecode: https://github.com/faster-cpython/tools https://github.com/faster-cpython/tools
  - kzrdude 5y ago
    Development is taking place in the main cpython repo and bugtracker, mark shannon already has a few merged PRs during the last month and a few more in flight. I.e the project has already started. See for example https://github.com/python/cpython/pull/25152 https://github.com/python/cpython/pull/25152
obviousbob 5y ago
CPython's arcane constraints are holding it back - GIL, C API with a global interpreter instance, ref-counting, to name a few. PyPy is already a drop-in replacement for CPython with JIT and using a fraction of the memory. https://dev.nextthought.com/blog/2018/08/cpython-vs-pypy-memory-usage.html https://dev.nextthought.com/blog/2018/08/cpython-vs-pypy-mem...
- fwip 5y ago
  Pypy also has the GIL: https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...
emily77 5y ago
BlockFi offers crypto interest-earning accounts with up to 8.6% APY. This allows clients holding crypto like Bitcoin & Ether to earn compounding interest. BlockFi also offers low-cost USD loans backed by crypto. Access crypto capital without selling.https://yazing.com/deals/blockfi/safi68 https://yazing.com/deals/blockfi/safi68
- thefounder 5y ago
  Spammer?
cpeterso 5y ago
Guido's slides in the repo mention speedup targets, but there is no mention of benchmarks. What is measured gets optimized, but which benchmarks are important for Python users? I'm sure Django users and Numpy users have very different performance bottlenecks.
- kzrdude 5y ago
  will speed.python.org be used for this?
morelisp 5y ago
Stage 2 should've been the selling point of Python 3. If there's really a 50% general speed boost sitting around in "normal" bytecode optimization work (and having read a decent amount of CPython I believe there could be), that's a criminal amount of energy over the past decade instead wasted dealing with what the type of a codepoint sequence should be named.
- Redoubts 5y ago
  What’s worse is that py3 was strictly slower than 2.7 for quite a few releases.
frankreyes 5y ago
This project is dead already. Lua is 4x faster out of the box compared to Python (without LuaJIT!). Lua got fast by having ints and floats at the local side of values, not as a reference. That's in, the int and float value is in the variable itself, not as a reference with a pointer indirection. In Python, every time you do a=a+1 new memory allocation is made for the a+1 object. Java has the same performance issue with boxed types (new Integer), and this is why C# is faster in many benchmarks. (C# cheated with the hindsight advantage of knowing which were Java mistakes)
guggle 5y ago
It really reminds me of this hilarious 2008 keynote by Cal Henderson: https://youtu.be/i6Fr65PFqfk?t=2255 https://youtu.be/i6Fr65PFqfk?t=2255 (relevant graphic: https://ibb.co/yV6bN9H https://ibb.co/yV6bN9H )

226 points by patrick91 5y ago

Presentation from Guido van Rossum at the Python Language Summit: https://github.com/faster-cpython/ideas/blob/main/FasterCPythonDark.pdf https://github.com/faster-cpython/ideas/blob/main/FasterCPyt...

antman 5y ago
Comments on the points being made based on my experience - without breaking anyone's code (did that for the print function etc and delayed py3 adoption for a decade but not for the most user requested feature which is speed) - No large PRs (how a 5x speedup can take place with small patches that no one figured out yet?) - a small team (not enough money, less risky if they tried to adopt other pypy-esque projects that are more advanced speed wise, a more comprehensive plan that would invite donations, I would certainly give. For this presentation I don't know)
- uluyol 5y ago
  WRT large PRs, it doesn't sound like they expect all the improvements to be from small changes. I think the point is the code will be changed in increments so that even large changes (overall) will be easier to review and offer feedback.
- kzrdude 5y ago
  I don't think "without breaking anyone's code" even needs to be a goal. Python has rolling deprecations and feature removals planned in the 3.x release series, and careful such feature transitions could be used to help JIT features or similar, too. To back up that fact: Python 3.10 will remove long-deprecated features, see issue tracker: https://bugs.python.org/issue41165 https://bugs.python.org/issue41165 and What's new, the Removed section: https://docs.python.org/3.10/whatsnew/3.10.html#removed https://docs.python.org/3.10/whatsnew/3.10.html#removed
- yjftsjthsd-h 5y ago
  > without breaking anyone's code (did that for the print function etc and delayed py3 adoption for a decade but not for the most user requested feature which is speed) The Python 3 debacle is a beautifully worked example of why being backwards compatible is so important, and I can 100% sympathize with them not wanting to go through that again.
axaxs 5y ago
Well, glad he's coming around. Years back he seemed to stand by the notion that Python wasn't slow, showing a toy example. I remember thinking it was just condescending, whether intentional or not.
- takeda 5y ago
  I think years ago things were much different, also even today that might still be true. Among interpreted languages python is/was one of the faster ones. NodeJS has similar speed, and in some cases is faster. I don't know any other interpreted language though. People are also complaining about GIL, but ironically NodeJS is single threaded.
  - qsort 5y ago
    Nodejs is much faster than Python even for workloads where there's overlap in use-cases, like webapis and scripting. The only production language with performance in the ballpark of Python is Ruby. I agree that more often than not it simply isn't an issue, but CPython performance isn't much far ahead of toy languages, for a whole lot of reasons that we are all sick and tired of hearing.

milliams 5y ago

I posted the link to the first PEP yesterday (https://news.ycombinator.com/item?id=27134290 https://news.ycombinator.com/item?id=27134290) I think this looks like a very promising project.

qteax 5y ago
25-50% has been hoped for many times in the past 3 decades. I don't find that very impressive. CPython got big because of its relatively decent C-API and glue capabilities. 25% does not make a difference, decent C extensions have speedups in the order of 10-100 times (yes, times, not percent). If I had a dollar for every time someone proposed a Python speedup ...
- pjmlp 5y ago
  Python is feeling the pressure of not having a JIT on the box. Microsoft is reactivating their JIT project for Python as well. Their talk tomorrow: "Talk: Restarting Pyjion, a general purpose JIT for Python – is it worth it?" https://us.pycon.org/2021/schedule/presentation/52/ https://us.pycon.org/2021/schedule/presentation/52/
  - aztec100 5y ago
    There have been many similar talks in the past decades. At some point Unladen Swallow was presented as "Google's project" (which is quite exaggerated). It looks more like Microsoft has JIT envy now that Instagram and others have open sourced (quite restricted) JIT projects and has to show presence again. I think every CPython JIT project will be full of corner cases in both performance and behavior, so it will add to the growing database of weird Python behavior that users have to memorize.
    - pjmlp 5y ago
      Indeed, it all boils down to the way Python community doesn't embrace such projects. Usually Python's high dynamism comes into the picture as main reason, however Smalltalk and SELF are just as dynamic, you can change the whole image at any given point in execution. In Smalltalk it suffices to send a become: message to change the complete meaning of an object, and references to it, across the whole image.

punnerud 5y ago

Deleted. Posted the wrong place. Thanks for the comments.

Jiejeing 5y ago
You probably meant to send those links there: https://news.ycombinator.com/item?id=27127387 https://news.ycombinator.com/item?id=27127387
- BerislavLopac 5y ago
  pyodide != pydiode :D

kzrdude 5y ago

This is exciting! So that we don't miss the main point: Eric Snow, GvR and Mark Shannon will be working for Microsoft to improve Python performance. In a way, it seems like Mark Shannon found someone to take him up on his offer on a plan for speeding up Python.

rqst 5y ago
Why do people in the Python community get so excited over announcements? I'd rather see working code, but no one cares about that in the Python universe. It is always announcements, talks, conferences and if something emerges it is a bit weird like the pattern matching. Meanwhile the Erlang people quietly produced a JIT without any advertisements.
- BiteCode_dev 5y ago
  > Why do people in the Python community get so excited over announcements? When you like a tech, knowing people are brewing improved perfs on it is kinky. On Python, the motto has always been "it's fast enough", "if you want perfs, don't use python", "C extensions will solve this", "we don't want to make the main implementation complicated", "python dynamism and GIL make it a hard problem", etc. So in the python world, it's particularly big news, especially given that previous attempts (gilectomy, unladen swallow, first pyston...) all died.
  - WesolyKubeczek 5y ago
    > So in the python world, it's particularly big news, especially given that previous attempts (gilectomy, unladen swallow, first pyston...) all died. This is precisely the point you seem to be missing. All those things you mentioned were similar big announcements back in their day, and they all have just died by fizzling. What should be setting this one apart?
    - BiteCode_dev 5y ago
      Because Guido is paid to work on it.

fdej 5y ago

I'd like to see optimizations targeting ctypes, or a successor to ctypes. I wish I could write elegant, performant C wrappers in pure Python. Right now the best choices are Cython, which is a hassle (separate, slow compilation, various warts), and ctypes, which is slow (and has some design problems of its own). Julia's ccall alone is a major selling point over Python right now.

BiteCode_dev 5y ago
It's not nuitka main purpose, but it can produce a lib and speed up the code: https://nuitka.net/ https://nuitka.net/
ihnorton 5y ago
CFFI [1] is a step in the right direction, inspired by LuaJIT's CFFI. It originated from the PyPy folks and is supported in PyPy [2]; it's also supported to some degree by Numba [3]. I don't know what level of C call optimization is available when using the JITs, so I can't speak to the performance, but I've used it casually via CPython and was impressed by the API. That said, it has been around for a while and the traction seems somewhat limited -- I would guess because most people who have this kind of problem also need more than "just" FFI. [1] https://cffi.readthedocs.io/en/latest/index.html https://cffi.readthedocs.io/en/latest/index.html [2] https://doc.pypy.org/en/latest/extending.html#cffi https://doc.pypy.org/en/latest/extending.html#cffi [3] https://numba.readthedocs.io/en/stable/reference/pysupported.html#cffi-support https://numba.readthedocs.io/en/stable/reference/pysupported...
adsharma 5y ago
Please take a look at py2many which does this. Looking for feedback.
- adsharma 5y ago
  There is now a pull request for this: https://github.com/adsharma/py2many/issues/62 https://github.com/adsharma/py2many/issues/62
adsharma 5y ago

sergiomattei 5y ago

This is the most exciting development I've seen in the Python scene in years! Thanks, Guido and team!

bla3 5y ago

It's good to see that python startup time is an explicit goal. Empty scripts taking 10-100ms is a problem if you use python in Makefiles, where you can have thousands of invocations in a build. This wasn't considered a use case upstream cared about for quite some time. I'm glad to see this is changing. Here's hoping the acknowledgement will result in concrete gains!

dongping 5y ago
Maybe you would want to check out Bazel as a build tool, since it uses a stripped down variant of Python called Starlark. https://docs.bazel.build/versions/master/skylark/language.html https://docs.bazel.build/versions/master/skylark/language.ht...
- bla3 5y ago
  It uses that for its build configuration, not for build steps, as far as I know.

pca006132 5y ago

Just wondering, why is CPython a lot slower than JS? It seems to me that both languages are interpreted and comes with some reflection functionalities (like modifying object methods), but JS seems a lot faster in most of the benchmarks in https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Edit: Why is this being downvoted? Is my statement incorrect or offensive to some people?

pansa2 5y ago
> It seems to me that both languages are interpreted Node.js uses the V8 engine for JavaScript which isn't just an interpreter - it includes a JIT compiler.
- dec0dedab0de 5y ago
  Yes, the JIT has got to be the biggest difference. I wonder how node compares to pypy. I think when GVR mentions "There's machine code generation in our future" that he is talking about a JIT
  - throwaway894345 5y ago
    I suspect Node is still quite a lot faster than Pypy because the former benefits from v8 which is a well-funded project which doesn’t have to worry about compatibility with the sprawling C-extension interface that CPython exposes.
ForHackernews 5y ago
I think mostly because Google has put a massive amount money and effort into making V8 faster. Performance has always been a lower-tier priority for cPython and other, faster, implementations (like PyPy) have not seen much mainstream adoption.
epidemian 5y ago
JS have been heavily optimized by browser vendors, fueled by the intense competition on the browser space, and the huge resources invested by companies like Google. IIRC Firefox 3 was the first browser to have what we would call a modern JS engine, with heavy emphasis on JIT compiling. Google Chrome soon followed and V8 dominated the JS perf story. Python doesn't have such fierce competition of implementations. And besides, it has the "escape hatch" of being able to implement performance-critical code paths as C extensions, which is what heavy-lifting libraries like numpy do.

ForHackernews 5y ago

Is this going to incorporate the work done as part of https://blog.pyston.org/2020/10/28/pyston-v2-20-faster-python/ https://blog.pyston.org/2020/10/28/pyston-v2-20-faster-pytho... or https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder ?

The_rationalist 5y ago

The python JIT that promise to get the highest throughput once maturity reached is https://github.com/oracle/graalpython https://github.com/oracle/graalpython

etaioinshrdlu 5y ago

I updated a large Django project from python2.7 to 3.9 recently. Afterward, I was pleased that the app was both faster and used less than half the memory. It was somewhat surprising that the new version was so much better - most software seems to typically get only heavier over time! So, I take this as a good sign that progress will continue.

guggle 5y ago
> most software seems to typically get only heavier over time! It's often the case for end-user applications, but I find that most languages/runtimes tend to get better with time (and work of course !). There's a similar trend with the PHP runtime.
- MaxBarraclough 5y ago
  That rule of thumb applies to the web too. Browsers keep getting faster, websites keep getting slower.
- cogman10 5y ago
  I can't think of a language/runtime that has actually gotten slower with time. The nearest I can really point to is the uptick in compile time that happens with languages like Rust/C++ every so often. That's not really a "language is slower" problem though.
  - kzrdude 5y ago
    rustc's trend the last twenty releases has been that the compiler is getting faster. I guess every dependency is slowly growing in size, offsetting this in your perception? The compiler benchmarks are of course held constant to be able to compare releases. See the perf website https://perf.rust-lang.org/dashboard.html https://perf.rust-lang.org/dashboard.html
1vuio0pswjnm7 5y ago
But isn't the Python install getting heavier. 3 packages will be installed: expat-2.3.0_1 gdbm-1.19_1 python-2.7.18_3 Size required on disk: 18MB 3 packages will be installed: expat-2.3.0_1 gdbm-1.19_1 python3-3.9.4_1 Size required on disk: 22MB

etskinner 5y ago

Why isn't this hosted at https://github.com/python/cpython/issues https://github.com/python/cpython/issues ?

_bohm 5y ago
Presumably development will be taking place on this organization's fork of cpython? https://github.com/faster-cpython/cpython https://github.com/faster-cpython/cpython Looks like they've also got a repo with some tools for profiling bytecode: https://github.com/faster-cpython/tools https://github.com/faster-cpython/tools
- kzrdude 5y ago
  Development is taking place in the main cpython repo and bugtracker, mark shannon already has a few merged PRs during the last month and a few more in flight. I.e the project has already started. See for example https://github.com/python/cpython/pull/25152 https://github.com/python/cpython/pull/25152

obviousbob 5y ago

CPython's arcane constraints are holding it back - GIL, C API with a global interpreter instance, ref-counting, to name a few. PyPy is already a drop-in replacement for CPython with JIT and using a fraction of the memory. https://dev.nextthought.com/blog/2018/08/cpython-vs-pypy-memory-usage.html https://dev.nextthought.com/blog/2018/08/cpython-vs-pypy-mem...

fwip 5y ago
Pypy also has the GIL: https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...

emily77 5y ago

BlockFi offers crypto interest-earning accounts with up to 8.6% APY. This allows clients holding crypto like Bitcoin & Ether to earn compounding interest. BlockFi also offers low-cost USD loans backed by crypto. Access crypto capital without selling.https://yazing.com/deals/blockfi/safi68 https://yazing.com/deals/blockfi/safi68

thefounder 5y ago
Spammer?

cpeterso 5y ago

Guido's slides in the repo mention speedup targets, but there is no mention of benchmarks. What is measured gets optimized, but which benchmarks are important for Python users? I'm sure Django users and Numpy users have very different performance bottlenecks.

kzrdude 5y ago
will speed.python.org be used for this?

morelisp 5y ago

Stage 2 should've been the selling point of Python 3. If there's really a 50% general speed boost sitting around in "normal" bytecode optimization work (and having read a decent amount of CPython I believe there could be), that's a criminal amount of energy over the past decade instead wasted dealing with what the type of a codepoint sequence should be named.

Redoubts 5y ago
What’s worse is that py3 was strictly slower than 2.7 for quite a few releases.

frankreyes 5y ago

This project is dead already. Lua is 4x faster out of the box compared to Python (without LuaJIT!). Lua got fast by having ints and floats at the local side of values, not as a reference. That's in, the int and float value is in the variable itself, not as a reference with a pointer indirection. In Python, every time you do a=a+1 new memory allocation is made for the a+1 object. Java has the same performance issue with boxed types (new Integer), and this is why C# is faster in many benchmarks. (C# cheated with the hindsight advantage of knowing which were Java mistakes)

guggle 5y ago

It really reminds me of this hilarious 2008 keynote by Cal Henderson: https://youtu.be/i6Fr65PFqfk?t=2255 https://youtu.be/i6Fr65PFqfk?t=2255 (relevant graphic: https://ibb.co/yV6bN9H https://ibb.co/yV6bN9H )