18 ms·

Yes, yes, yes! This is what I've been waiting for so long. Python has emphasized readability for the reference implementation vs. the practical benefits of bet

Yes, yes, yes! This is what I've been waiting for so long. Python has emphasized readability for the reference implementation vs. the practical benefits of better performance for everyone, and the community is really hurting for it. Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better. Python can be ahead in the performance race. We just need to get real.

gonational 5y ago
[deleted]
jimbob45 5y ago
Nim has long had the best of Python’s readability mixed with C’s performance and low-level constructs. I urge you to check it out. https://nim-lang.org/ https://nim-lang.org/
- 1f60c 5y ago
  I think GP is talking about the readability of CPython itself.
- tastyminerals2 5y ago
  I disagree, it was the Nim syntax that kept me away from the language back in the Uni when I tried it out. If anything, C syntax is barely as easy to read.
StavrosK 5y ago
That would indeed be fantastic. V8 has had so much optimization whereas CPython has had very little of it, though Pypy deserves a mention here. They've been doing fantastic work, but I guess there's always the curse of the reference implementation being the popular one.
rtpg 5y ago
So much stuff just from the readme would introduce breaking changes to the Python ecosystem. Part of the reason Instagram can get away with this is they likely have almost complete control of their dependencies and the like. But the changes they decide to make would not just work for everything. That being said I think we could get a lot of neat PEPs from this. EDIT: this is just my opinion , but in a world where we have type annotations, JITs feel like a massive step back. Stuff like mypyc could get us way further into high performance stuff (and no black box hand waving for perf things).
- nine_k 5y ago
  I don't see why JIT cannot use type annotations to speed up the code. But JIT also works on untyped or incompletely typed code.
  - nimish 5y ago
    There's no requirement that type annotations be accurate. With hidden classes and such it likely wouldn't matter much. You'd need them anyway for full dynamism
    - _carljm 5y ago
      With Static Python we use type annotations in compilation, and we require them to be correct (they are runtime checked at boundaries with non-Static code and throw TypeError if wrong types are passed in.) We haven’t gone the hidden classes route so far because it’s simpler to just look at attributes assigned in __init__ and annotated on the class and lock those into slots. If you’re writing typed Python you probably don’t do a lot of tacking extra ad hoc attributes onto instances, since type checkers don’t like that either.
  - rtpg 5y ago
    I mean if you care about perf you can do it in one step. It’s not that huge of an ask, and engineering wise “compile some typed code” is a hell of a lot more straightforward than “try to guess what I should speculatively compile at runtime” There was a piece by v8 devs a while back where they ended up turning down JIT aggressiveness cuz it ended up making normal web pages slower (except for like Google Docs). This is just me backseat-BDFLing “why do the hard thing when we could do the easy thing” tho
skohan 5y ago
> Python can be ahead in the performance race. We just need to get real. I don't know if this is the case. Python's entire value proposition is that it is "executable pseudocode" with an extremely low barrier to entry - so somebody like a scientist or business analyst can solve problems without a deep understanding of computer science. Those goals are always going to run counter to performance, and python would have to re-invent itself at a foundational level to actually compete on performance with languages which were optimized for it from the ground up. Python's already winning in a lot of super relevant ways from playing to its strengths, and I don't think it makes sense to compromise those strengths to reach middle-of-the-pack performance. If you ask me, if the python community really wanted to advance their interests, the thing to focus on would be dependency management and project encapsulation. If I could have a ux like npm or cargo for python, I would surely be tempted to use it more outside of jupyter. But it would not be because of performance ;)
- akullpp 5y ago
  FYI: Poetry (https://python-poetry.org https://python-poetry.org) is actually close to something like npm.
  - bigfudge 5y ago
    Except like pipenv it is dog slow and totally inscrutable when dependencies clash.
    - stuaxo 5y ago
      It seems a lot faster than pipenv. From the little I've tried, it seems a lot more sane - but I'm biased - I hated pipenv and never saw the point of it.
    - korijn 5y ago
      So have you considered the possibility that it may not be the fault of these package managers, but that there may be a different problem underneath it all that a package manager cannot fix for you?
    - 5y ago
pjmlp 5y ago
I fear this will be another unladen swallow, it is not for lack of trying, rather the resistance to change.
- throwaway2037 5y ago
  That was the first thing I google'd when I saw this article: "unladen swallow"! For those unfamiliar: https://code.google.com/archive/p/unladen-swallow/ https://code.google.com/archive/p/unladen-swallow/ https://www.infoq.com/news/2011/03/unladen-swallow/ https://www.infoq.com/news/2011/03/unladen-swallow/ https://www.python.org/dev/peps/pep-3146/ https://www.python.org/dev/peps/pep-3146/
- sergiomattei 5y ago
  I honestly really wish this isn't the case again. Python is falling behind fast. Look at how well PHP has improved in performance and how Ruby is doing lately. This resistance is biting back.
  - pjmlp 5y ago
    This is why I see Julia's uptake as positive, because if that isn't going to change the community aversion to these attempts and will rather keep writing C, nothing else will.
sammorrowdrums 5y ago
I was doing a Hacker Rank puzzle yesterday in Python and after I optimised it, it still timed out on one input. Looked at the chat, and saw a comment "just run it with PyPy" and naturally it then passed all parts. It's easy to forget that CPython is not that optimised generally, but as soon as you need to deal with algorithms it is lacking often. I think even that the fact there are so many C implementations in libraries shows that the lack of optimisation for the sake of clean compiler code manifests as a readability / complexity issues elsewhere.
- FartyMcFarter 5y ago
  > It's easy to forget that CPython is not that optimised generally, "Not that optimised" is actually an understatement. In my experience, CPU-heavy code typically becomes 10-100x faster if you port it from Python to a language like C (without even trying to be clever or using SIMD assembly).
  - stevesimmons 5y ago
    Often you can put a "@numba.jit" decorator at the top of a calc-heavy function, and reclaim a good portion of C's performance. It is so simple it is definitely worth trying. I did it for Bloom tree implementation, and got to within around 3x of the C code it replaced.
Jiejeing 5y ago
> Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better. I don’t know here you are picking this from. What’s hurting performance is not readability, it is the flexibility of what you can do (hence: strict modules/static python in Cinder) which means it is really hard to perform optimizations as no assumptions hold unless you turn down the more permissive language features. Another one is backwards compatibility, and I might get some flak from people still complaining about it more than 10 years after python 3.0, but things are still 99.9% compatible with decades-old code in python 3.9, and you can’t throw that away. Finally, the numbers of people dedicated to improving CPython performance in an upstreamable fashion is ridiculously low compared to other languages, and especially compared to the number of businesses using it. There are quite few people working on performance, but not many of them appear to be doing it in the open and with the goal of upstreaming it in a non-breaking manner (kudos to the instagram engineers, I guess).
- Redoubts 5y ago
  > Finally, the numbers of people dedicated to improving CPython performance in an upstreamable fashion is ridiculously low compared to other languages Yeah because Guido and the maintainers have proclaimed multiple times that they prefer a simple implementation for teachability, over performance minded design. That’s where this entire complaint is coming from.
  - oblio 5y ago
    This was a reasonable requirement for an academic language but it's utterly dogmatic for one of the most used languages in the world.
sergiomattei 5y ago
To clear some confusion... Python's language readability is NOT the problem I allude to here. The problem is CPython's core team shooting down any performance improvement for the sake of keeping the compiler source code readable.

gonational 5y ago

[deleted]

jimbob45 5y ago

Nim has long had the best of Python’s readability mixed with C’s performance and low-level constructs. I urge you to check it out. https://nim-lang.org/ https://nim-lang.org/

1f60c 5y ago
I think GP is talking about the readability of CPython itself.
tastyminerals2 5y ago
I disagree, it was the Nim syntax that kept me away from the language back in the Uni when I tried it out. If anything, C syntax is barely as easy to read.

StavrosK 5y ago

That would indeed be fantastic. V8 has had so much optimization whereas CPython has had very little of it, though Pypy deserves a mention here. They've been doing fantastic work, but I guess there's always the curse of the reference implementation being the popular one.

rtpg 5y ago

So much stuff just from the readme would introduce breaking changes to the Python ecosystem. Part of the reason Instagram can get away with this is they likely have almost complete control of their dependencies and the like. But the changes they decide to make would not just work for everything. That being said I think we could get a lot of neat PEPs from this. EDIT: this is just my opinion , but in a world where we have type annotations, JITs feel like a massive step back. Stuff like mypyc could get us way further into high performance stuff (and no black box hand waving for perf things).

nine_k 5y ago
I don't see why JIT cannot use type annotations to speed up the code. But JIT also works on untyped or incompletely typed code.
- nimish 5y ago
  There's no requirement that type annotations be accurate. With hidden classes and such it likely wouldn't matter much. You'd need them anyway for full dynamism
  - _carljm 5y ago
    With Static Python we use type annotations in compilation, and we require them to be correct (they are runtime checked at boundaries with non-Static code and throw TypeError if wrong types are passed in.) We haven’t gone the hidden classes route so far because it’s simpler to just look at attributes assigned in __init__ and annotated on the class and lock those into slots. If you’re writing typed Python you probably don’t do a lot of tacking extra ad hoc attributes onto instances, since type checkers don’t like that either.
- rtpg 5y ago
  I mean if you care about perf you can do it in one step. It’s not that huge of an ask, and engineering wise “compile some typed code” is a hell of a lot more straightforward than “try to guess what I should speculatively compile at runtime” There was a piece by v8 devs a while back where they ended up turning down JIT aggressiveness cuz it ended up making normal web pages slower (except for like Google Docs). This is just me backseat-BDFLing “why do the hard thing when we could do the easy thing” tho

skohan 5y ago

> Python can be ahead in the performance race. We just need to get real. I don't know if this is the case. Python's entire value proposition is that it is "executable pseudocode" with an extremely low barrier to entry - so somebody like a scientist or business analyst can solve problems without a deep understanding of computer science. Those goals are always going to run counter to performance, and python would have to re-invent itself at a foundational level to actually compete on performance with languages which were optimized for it from the ground up. Python's already winning in a lot of super relevant ways from playing to its strengths, and I don't think it makes sense to compromise those strengths to reach middle-of-the-pack performance. If you ask me, if the python community really wanted to advance their interests, the thing to focus on would be dependency management and project encapsulation. If I could have a ux like npm or cargo for python, I would surely be tempted to use it more outside of jupyter. But it would not be because of performance ;)

akullpp 5y ago
FYI: Poetry (https://python-poetry.org https://python-poetry.org) is actually close to something like npm.
- bigfudge 5y ago
  Except like pipenv it is dog slow and totally inscrutable when dependencies clash.
  - stuaxo 5y ago
    It seems a lot faster than pipenv. From the little I've tried, it seems a lot more sane - but I'm biased - I hated pipenv and never saw the point of it.
  - korijn 5y ago
    So have you considered the possibility that it may not be the fault of these package managers, but that there may be a different problem underneath it all that a package manager cannot fix for you?
  - 5y ago

pjmlp 5y ago

I fear this will be another unladen swallow, it is not for lack of trying, rather the resistance to change.

throwaway2037 5y ago
That was the first thing I google'd when I saw this article: "unladen swallow"! For those unfamiliar: https://code.google.com/archive/p/unladen-swallow/ https://code.google.com/archive/p/unladen-swallow/ https://www.infoq.com/news/2011/03/unladen-swallow/ https://www.infoq.com/news/2011/03/unladen-swallow/ https://www.python.org/dev/peps/pep-3146/ https://www.python.org/dev/peps/pep-3146/
sergiomattei 5y ago
I honestly really wish this isn't the case again. Python is falling behind fast. Look at how well PHP has improved in performance and how Ruby is doing lately. This resistance is biting back.
- pjmlp 5y ago
  This is why I see Julia's uptake as positive, because if that isn't going to change the community aversion to these attempts and will rather keep writing C, nothing else will.

sammorrowdrums 5y ago

I was doing a Hacker Rank puzzle yesterday in Python and after I optimised it, it still timed out on one input. Looked at the chat, and saw a comment "just run it with PyPy" and naturally it then passed all parts. It's easy to forget that CPython is not that optimised generally, but as soon as you need to deal with algorithms it is lacking often. I think even that the fact there are so many C implementations in libraries shows that the lack of optimisation for the sake of clean compiler code manifests as a readability / complexity issues elsewhere.

FartyMcFarter 5y ago
> It's easy to forget that CPython is not that optimised generally, "Not that optimised" is actually an understatement. In my experience, CPU-heavy code typically becomes 10-100x faster if you port it from Python to a language like C (without even trying to be clever or using SIMD assembly).
- stevesimmons 5y ago
  Often you can put a "@numba.jit" decorator at the top of a calc-heavy function, and reclaim a good portion of C's performance. It is so simple it is definitely worth trying. I did it for Bloom tree implementation, and got to within around 3x of the C code it replaced.

Jiejeing 5y ago

> Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better. I don’t know here you are picking this from. What’s hurting performance is not readability, it is the flexibility of what you can do (hence: strict modules/static python in Cinder) which means it is really hard to perform optimizations as no assumptions hold unless you turn down the more permissive language features. Another one is backwards compatibility, and I might get some flak from people still complaining about it more than 10 years after python 3.0, but things are still 99.9% compatible with decades-old code in python 3.9, and you can’t throw that away. Finally, the numbers of people dedicated to improving CPython performance in an upstreamable fashion is ridiculously low compared to other languages, and especially compared to the number of businesses using it. There are quite few people working on performance, but not many of them appear to be doing it in the open and with the goal of upstreaming it in a non-breaking manner (kudos to the instagram engineers, I guess).

Redoubts 5y ago
> Finally, the numbers of people dedicated to improving CPython performance in an upstreamable fashion is ridiculously low compared to other languages Yeah because Guido and the maintainers have proclaimed multiple times that they prefer a simple implementation for teachability, over performance minded design. That’s where this entire complaint is coming from.
- oblio 5y ago
  This was a reasonable requirement for an academic language but it's utterly dogmatic for one of the most used languages in the world.

sergiomattei 5y ago

To clear some confusion... Python's language readability is NOT the problem I allude to here. The problem is CPython's core team shooting down any performance improvement for the sake of keeping the compiler source code readable.