8 ms·

> "Make Python faster" is just a losing game imo. It is fundamentally never going to be as fast as other languages, it's far too dynamic (and that's a huge appe

> "Make Python faster" is just a losing game imo. It is fundamentally never going to be as fast as other languages, it's far too dynamic (and that's a huge appeal of the language). This cannot be overstated. Unfortunately python is especially perversely dynamic. After all Javascript and Ruby are highly dynamic, but for a number of reasons have avenues that allow more meaningful optimization gains without changing language semantics. (And although it is true that Google had tremendous resources to pour into V8, it's not like this point doesn't stand - luajit as an example). Python imho took the performance/productivity trade off way too far. You can get some very effective dynamism with some minimal restrictions that won't so badly shoot yourself in the foot for performance opportunities later. Frankly, for mainstream development, Kotlin or C# gives very pleasant, productive languages with strong ecosystems without paying such a penalty. Swift is good. Go isn't personally my cup of tea, but sure. Python sort of got a foothold in science.. but that's never been known for being a field of quality software engineering.

Demiurge 6y ago
What exactly is known for being a field of quality software engineering? 100% of Python users find it to be fast enough for their use case. Let's stop beating this horse into a black hole, it doesn't need to be fast at converting integers to unicode 100,000 times, to allow quality software engineering. To compare speed of CPython with quality of anything is such a narrow view, it is self-defeating.
- staticassertion 6y ago
  > 100% of Python users find it to be fast enough for their use case. This is absolutely false. Lots of people outgrow Python and use another language.
  - Demiurge 6y ago
    That's the intended humor, they stop being Python users if they outgrow it. The language and ecosystem is growing, regardless. I've been using it professionally since 2007, and some tasks I know it is not good at, and thats when I know to use another tool. The point is that it is a tool that has its uses, and I consider it the best tool for certain categories. It doesn't need to be the best tool for all categories. That seems pretty "absolutist".
    - staticassertion 6y ago
      I'm not anti-Python, for what it's worth. I love Python, and part of that is because of the features that make it so slow.
- catblast 6y ago
  > 100% of Python users find it to be fast enough for their use case. This is unlikely and an odd thing to say especially in the context of a thread about people rewriting their software in a different stack in part because of python performance issues. There are plenty of people using python that feel the pain and need to spend resources on performance improvements. The fact very many line of business apps will require more and more complicated hardware resources compared to using "boring shit" like .NET or Java, or Go (which is actually pretty boring, startup hype aside). I'm no huge fan of Java, but I don't feel any less productive in Kotlin than I do python, for many things it is even better. Python aside from a few things is still looks like a 1989 language with a few newer features - the language other than being basically binding lazy doesn't have many amazing tricks up its sleeve. Meanwhile 30 years of progress has been rolled into the mainstream of C#, Swift, and Kotlin. > To compare speed of CPython with quality of anything is such a narrow view I'm not saying you can't write quality software with python. What I was saying is the only niche that Python has maybe picked up a significant mindshare compared to alternatives is scientific computing. And I say this with no insult, but as a former grad student in the sciences and having written quite a bit of monstrous python - it's not a field of quality software engineering.
  - Demiurge 6y ago
    I've seen badly written scientific Python code, and I've seen very badly written enterprise Java code. I’ve also written probably more than hundred web apps using Django, that never needed to be written in Java. And Django is great work of software engineering, having not much to do with scientific software. Some apps need to be re-written for performance reasons, or optimized, that is however, not the most common case. For all the companies that are complaining about the performance issues, I suspect they are complaining about the change of their incentives or circumstances, unless they made an uninformed choice with choosing Python to begin with. Which is the likely case?
  - Demiurge 6y ago
    The missing piece of the discussion here seems to be socio-political (human) aspect of computer language usage. When choosing a "slow" language, many other factors beside performance and ecosystem are considered. The primary usage of a computer language is actually communicating with the fellow human beings, and there is a huge cost and overhead associated with all the software writing and interpretation. That is why it baffles me when Python is dismissed entirely on a pure performance basis.
- josefx 6y ago
  > 100% of Python users find it to be fast enough for their use case. As someone who uses python because it tends to be installed on my targets and comes with a decent standard library, I would like the hours I spend optimizing my code back. Especially those hours I lost before I ended up porting the problematic code to C++ or Java.
jashmatthews 6y ago
Python isn't perversely dynamic, no more so than Self which was running within a factor of 2 of C like 30 years ago, and PyPy optimizes Python just fine. The major stumbling block is that the C API unintentionally sucks and requires lots of work to support https://morepypy.blogspot.com/2018/09/inside-cpyext-why-emulating-cpython-c.html https://morepypy.blogspot.com/2018/09/inside-cpyext-why-emul... LuaJIT is a great example of how experimentation and the right tradeoffs can give you a faster language runtime without a huge effort but it's not at all a great example of a highly optimizing compiler. There's no "map" or "hidden class" optimization in LuaJIT, for example, and getting good performance means avoiding repeated table lookups.
zmmmmm 6y ago
> Frankly, for mainstream development, Kotlin or C# gives very pleasant, productive languages with strong ecosystems without paying such a penalty If dynamic is important, Groovy is an excellent option - it preserves almost all of the desirable dynamic properties of Python but is by default 3-5x as fast and with trivial effort 10-20x as fast, approaching Java speeds essentially, and with all of the advantages of that runtime (full concurrency, etc).
pjmlp 6y ago
This is always mentioned as motivation factor, yet Self, Smalltalk, Dylan, Julia, Common Lisp, JavaScript are just as dynamic and manage to have a good set of JITs to choose from. For example in Smalltalk you can completely change the structure of a given class across the whole process space, thus invalidating everything that the JIT tried to optimize. So no, Python isn't any special snowflake, rather there isn't the willingness to actually spend the resources improving its JIT story.
- pas 6y ago
  Is Julia really just as dynamic? Isn't the problem with Python that the low-level C API still needs to be respected during JIT, which just kills performance. (That's why PyPy largely doesn't support it, right?) Even JS doesn't suffer from this, because folks can't just load V8 extensions in their browser, and Node went through quite a few NAPI versions - forced by V8 changes. That said, of course it'd be possible to speed up Python/CPython with pouring a lot more money into it. But ... the CPython codebase is already old and has many problems, a lot of backward compatibility gotchas, and relatively few people willing to work on it. Because there's not a big reason to do so. Whereas with JS Google (and thus Mozilla too) was very incentivized to make it fast.
  - pjmlp 6y ago
    Julia, I am not sure how far its dynamism goes, but Dylan, Common Lisp and Smalltalk certainly. You can even at any point in time stop execution in the debugger, rewrite part of the world and just press continue as if that was how the code was originally written to start with.
    - catblast 6y ago
      > You can even at any point in time stop execution in the debugger, rewrite part of the world and just press continue as if that was how the code was originally written to start with. Bit of a straw man, because you wouldn't do this regularly in code. Python on the other hand is a relatively large language at this point, and plain idiomatic python code leans relatively heavier on JIT unfriendly constructs compared to the other languages mentioned. Meanwhile, CL has a whole concept of "compile-time" that doesn't really exist in python. Hence the "perversely" part. PyPy has used similar tricks as Smalltalk, Self, and JS/V8, many which were old hat in the 90s, but PyPy demonstrates that writing a performant JIT with reasonable memory requirements for real world code is much harder for Python.

Demiurge 6y ago

What exactly is known for being a field of quality software engineering? 100% of Python users find it to be fast enough for their use case. Let's stop beating this horse into a black hole, it doesn't need to be fast at converting integers to unicode 100,000 times, to allow quality software engineering. To compare speed of CPython with quality of anything is such a narrow view, it is self-defeating.

staticassertion 6y ago
> 100% of Python users find it to be fast enough for their use case. This is absolutely false. Lots of people outgrow Python and use another language.
- Demiurge 6y ago
  That's the intended humor, they stop being Python users if they outgrow it. The language and ecosystem is growing, regardless. I've been using it professionally since 2007, and some tasks I know it is not good at, and thats when I know to use another tool. The point is that it is a tool that has its uses, and I consider it the best tool for certain categories. It doesn't need to be the best tool for all categories. That seems pretty "absolutist".
  - staticassertion 6y ago
    I'm not anti-Python, for what it's worth. I love Python, and part of that is because of the features that make it so slow.
catblast 6y ago
> 100% of Python users find it to be fast enough for their use case. This is unlikely and an odd thing to say especially in the context of a thread about people rewriting their software in a different stack in part because of python performance issues. There are plenty of people using python that feel the pain and need to spend resources on performance improvements. The fact very many line of business apps will require more and more complicated hardware resources compared to using "boring shit" like .NET or Java, or Go (which is actually pretty boring, startup hype aside). I'm no huge fan of Java, but I don't feel any less productive in Kotlin than I do python, for many things it is even better. Python aside from a few things is still looks like a 1989 language with a few newer features - the language other than being basically binding lazy doesn't have many amazing tricks up its sleeve. Meanwhile 30 years of progress has been rolled into the mainstream of C#, Swift, and Kotlin. > To compare speed of CPython with quality of anything is such a narrow view I'm not saying you can't write quality software with python. What I was saying is the only niche that Python has maybe picked up a significant mindshare compared to alternatives is scientific computing. And I say this with no insult, but as a former grad student in the sciences and having written quite a bit of monstrous python - it's not a field of quality software engineering.
- Demiurge 6y ago
  I've seen badly written scientific Python code, and I've seen very badly written enterprise Java code. I’ve also written probably more than hundred web apps using Django, that never needed to be written in Java. And Django is great work of software engineering, having not much to do with scientific software. Some apps need to be re-written for performance reasons, or optimized, that is however, not the most common case. For all the companies that are complaining about the performance issues, I suspect they are complaining about the change of their incentives or circumstances, unless they made an uninformed choice with choosing Python to begin with. Which is the likely case?
- Demiurge 6y ago
  The missing piece of the discussion here seems to be socio-political (human) aspect of computer language usage. When choosing a "slow" language, many other factors beside performance and ecosystem are considered. The primary usage of a computer language is actually communicating with the fellow human beings, and there is a huge cost and overhead associated with all the software writing and interpretation. That is why it baffles me when Python is dismissed entirely on a pure performance basis.
josefx 6y ago
> 100% of Python users find it to be fast enough for their use case. As someone who uses python because it tends to be installed on my targets and comes with a decent standard library, I would like the hours I spend optimizing my code back. Especially those hours I lost before I ended up porting the problematic code to C++ or Java.

jashmatthews 6y ago

Python isn't perversely dynamic, no more so than Self which was running within a factor of 2 of C like 30 years ago, and PyPy optimizes Python just fine. The major stumbling block is that the C API unintentionally sucks and requires lots of work to support https://morepypy.blogspot.com/2018/09/inside-cpyext-why-emulating-cpython-c.html https://morepypy.blogspot.com/2018/09/inside-cpyext-why-emul... LuaJIT is a great example of how experimentation and the right tradeoffs can give you a faster language runtime without a huge effort but it's not at all a great example of a highly optimizing compiler. There's no "map" or "hidden class" optimization in LuaJIT, for example, and getting good performance means avoiding repeated table lookups.

zmmmmm 6y ago

> Frankly, for mainstream development, Kotlin or C# gives very pleasant, productive languages with strong ecosystems without paying such a penalty If dynamic is important, Groovy is an excellent option - it preserves almost all of the desirable dynamic properties of Python but is by default 3-5x as fast and with trivial effort 10-20x as fast, approaching Java speeds essentially, and with all of the advantages of that runtime (full concurrency, etc).

pjmlp 6y ago

This is always mentioned as motivation factor, yet Self, Smalltalk, Dylan, Julia, Common Lisp, JavaScript are just as dynamic and manage to have a good set of JITs to choose from. For example in Smalltalk you can completely change the structure of a given class across the whole process space, thus invalidating everything that the JIT tried to optimize. So no, Python isn't any special snowflake, rather there isn't the willingness to actually spend the resources improving its JIT story.

pas 6y ago
Is Julia really just as dynamic? Isn't the problem with Python that the low-level C API still needs to be respected during JIT, which just kills performance. (That's why PyPy largely doesn't support it, right?) Even JS doesn't suffer from this, because folks can't just load V8 extensions in their browser, and Node went through quite a few NAPI versions - forced by V8 changes. That said, of course it'd be possible to speed up Python/CPython with pouring a lot more money into it. But ... the CPython codebase is already old and has many problems, a lot of backward compatibility gotchas, and relatively few people willing to work on it. Because there's not a big reason to do so. Whereas with JS Google (and thus Mozilla too) was very incentivized to make it fast.
- pjmlp 6y ago
  Julia, I am not sure how far its dynamism goes, but Dylan, Common Lisp and Smalltalk certainly. You can even at any point in time stop execution in the debugger, rewrite part of the world and just press continue as if that was how the code was originally written to start with.
  - catblast 6y ago
    > You can even at any point in time stop execution in the debugger, rewrite part of the world and just press continue as if that was how the code was originally written to start with. Bit of a straw man, because you wouldn't do this regularly in code. Python on the other hand is a relatively large language at this point, and plain idiomatic python code leans relatively heavier on JIT unfriendly constructs compared to the other languages mentioned. Meanwhile, CL has a whole concept of "compile-time" that doesn't really exist in python. Hence the "perversely" part. PyPy has used similar tricks as Smalltalk, Self, and JS/V8, many which were old hat in the 90s, but PyPy demonstrates that writing a performant JIT with reasonable memory requirements for real world code is much harder for Python.