4 ms·

And yet… Did you know about this (serious) proposal to make CPython 5x faster? https://mail.python.org/archives/list/python-dev@python.org/thread/RDXLCH22T2EZD

by Radim 6y ago

And yet… Did you know about this (serious) proposal to make CPython 5x faster? https://mail.python.org/archives/list/python-dev@python.org/thread/RDXLCH22T2EZDRCBM6ZYYIUTBWQVVVWH/ https://mail.python.org/archives/list/python-dev@python.org/... Needed funding though.

tpetry 6y ago
Why does this not happen or anything like this? Ruby got MJT in a big effort to make Ruby a lot faster. PHP is constantly increasing it‘s performance and tries to change a lot to get faster (basic building blocks for JIT in PHP8). Why is python not iterating so much on it‘s performance? Or is it doing this but it‘s not as public in promoting it like my former two examples.
- dmurray 6y ago
  There's already PyPy. It's faster for, I estimate, 90% of workloads that don't use NumPy or Tensorflow, which offload work to faster implementations in different ways. It can certainly be 5 times faster. It doesn't have that much adoption, which suggests that performance isn't critical to Python's user base. If PyPy got significant market share, perhaps the CPython developers would prioritise an approach like this. (Or perhaps they'd have even more justification to say look, there's already a performance-focused implementation, we'll do our thing).
  - tpetry 6y ago
    Pypy isnt used that much because you may run into problems. And nobody wants to research strange runtime errors in production. But the speed improvements would be welcome if they were in cpython because you can be quite sure there will be no problems.
    - olau 6y ago
      Agreed (and I like pypy) One example is that Pypy switched away from reference-counting to fully garbage collected. They are probably right that's faster, but if I'm running a webserver, it's actually really comforting to know that any garbage produced by the processing of a request is collected when that request ends (barring cycles of course) so I don't end up with weird hangups.
      - pjmlp 6y ago
        Using a GC doesn't preclude doing a correct use of ResourceManager.
deleted 6y ago
[deleted]
jensus 6y ago
difficult to get this through without going through the established channels and dealing with the politics straight on
MobiusHorizons 6y ago
Very interesting thread!
cmeacham98 6y ago
The author claims to have _existing code_ for the first stage, but provides no code and no benchmarks in the post or linked github repo. Additionally, the repository does not seem to make a particularly strong case that each improvement will result in the massive 50% speedup compared to the previous as claimed. I'm not in the python community, so it's possible this person is well known and respected enough that part of this can be taken on faith. However, because the author does not provide benchmarks or any other proof of the effectiveness of the supposedly almost complete stage 1, I cannot personally call this proposal serious.
- olau 6y ago
  If you actually read the thread, it is evident that the author is in fact well-known and has worked on this for many years.
  - cmeacham98 6y ago
    I decided to read the entire thread, and my final conclusion remains roughly the same. Although "not serious" is perhaps too harsh, the author does not provide the level of detail I'd expect of someone asking for 250 thousand dollars for their existing work (as well as 750k for future work). The most information we got was "on a few simple benchmarks I'm seeing about a 70% speedup vs master for both default and LTO configures." However, this information is almost completely useless. What benchmarks? What architectures? What hardware? It would take the author 5 minutes of their time to say "here's the benchmark code I ran, it executed in [time] on CPython master commit [hash], and [lower time] on my patch, running on a machine with [hardware]." To be completely honest, I fail to see why they are not providing this information that would make their proposal significantly stronger.
- Alex3917 6y ago
  > The author claims to have _existing code_ for the first stage, but provides no code and no benchmarks in the post or linked github repo. Since he was only asking for payment once he hit his benchmarks for each stage, why does it matter?

by Radim 6y ago

tpetry 6y ago

Why does this not happen or anything like this? Ruby got MJT in a big effort to make Ruby a lot faster. PHP is constantly increasing it‘s performance and tries to change a lot to get faster (basic building blocks for JIT in PHP8). Why is python not iterating so much on it‘s performance? Or is it doing this but it‘s not as public in promoting it like my former two examples.

dmurray 6y ago
There's already PyPy. It's faster for, I estimate, 90% of workloads that don't use NumPy or Tensorflow, which offload work to faster implementations in different ways. It can certainly be 5 times faster. It doesn't have that much adoption, which suggests that performance isn't critical to Python's user base. If PyPy got significant market share, perhaps the CPython developers would prioritise an approach like this. (Or perhaps they'd have even more justification to say look, there's already a performance-focused implementation, we'll do our thing).
- tpetry 6y ago
  Pypy isnt used that much because you may run into problems. And nobody wants to research strange runtime errors in production. But the speed improvements would be welcome if they were in cpython because you can be quite sure there will be no problems.
  - olau 6y ago
    Agreed (and I like pypy) One example is that Pypy switched away from reference-counting to fully garbage collected. They are probably right that's faster, but if I'm running a webserver, it's actually really comforting to know that any garbage produced by the processing of a request is collected when that request ends (barring cycles of course) so I don't end up with weird hangups.
    - pjmlp 6y ago
      Using a GC doesn't preclude doing a correct use of ResourceManager.

deleted 6y ago

[deleted]

jensus 6y ago

difficult to get this through without going through the established channels and dealing with the politics straight on

MobiusHorizons 6y ago

Very interesting thread!

cmeacham98 6y ago

The author claims to have _existing code_ for the first stage, but provides no code and no benchmarks in the post or linked github repo. Additionally, the repository does not seem to make a particularly strong case that each improvement will result in the massive 50% speedup compared to the previous as claimed. I'm not in the python community, so it's possible this person is well known and respected enough that part of this can be taken on faith. However, because the author does not provide benchmarks or any other proof of the effectiveness of the supposedly almost complete stage 1, I cannot personally call this proposal serious.

olau 6y ago
If you actually read the thread, it is evident that the author is in fact well-known and has worked on this for many years.
- cmeacham98 6y ago
  I decided to read the entire thread, and my final conclusion remains roughly the same. Although "not serious" is perhaps too harsh, the author does not provide the level of detail I'd expect of someone asking for 250 thousand dollars for their existing work (as well as 750k for future work). The most information we got was "on a few simple benchmarks I'm seeing about a 70% speedup vs master for both default and LTO configures." However, this information is almost completely useless. What benchmarks? What architectures? What hardware? It would take the author 5 minutes of their time to say "here's the benchmark code I ran, it executed in [time] on CPython master commit [hash], and [lower time] on my patch, running on a machine with [hardware]." To be completely honest, I fail to see why they are not providing this information that would make their proposal significantly stronger.
Alex3917 6y ago
> The author claims to have _existing code_ for the first stage, but provides no code and no benchmarks in the post or linked github repo. Since he was only asking for payment once he hit his benchmarks for each stage, why does it matter?