7 ms·

I would argue that performance always matters, and that Python is never the right tool for the job in an absolute sense. Python may be the right tool for the j

I would argue that performance always matters, and that Python is never the right tool for the job in an absolute sense. Python may be the right tool for the job given the options we have today but there is no reason we cannot have a language exactly as nice to use as Python is, but that also provides good performance. Languages like Nim or F# approximate that ideal, for instance. And while I realize there are high perf variants of Python, these should be the primary standard, and only path. The slow path shouldn't exist. It is a failure of our community that we allow languages to proliferate while remaining slow. This is bad because allowing slow tools to become popular means people create slow things, which wastes other peoples time and energy. Electron becoming a standard way to make cross platform desktop apps is another example. Someone is not wrong to choose electron for that job, but we the software community are wrong to have let something that inefficient become the easiest way to do that job. You cannot simply dismiss this issue by saying "Well don't use the slow software you don't like then" as many of these things become de-facto standards that you cannot avoid. Your place of work may require Microsoft Teams as the chat software, and now you are using a huge % of your laptops ram and battery for simple text transmission. Atom becomes the popular target for language plugins and ends up the only usable way to get good IDE features for your language, and you suffer the performance hit for it. We can do better!

lmm 9y ago
> It is a failure of our community that we allow languages to proliferate while remaining slow. This is bad because allowing slow tools to become popular means people create slow things, which wastes other peoples time and energy. Make it work, then make it work right, then make it work fast. I mean yes, a lot of things are slower than they should be, but the level of outright correctness bugs in software today is mindblowing. So while replacing our tools with faster tools should be something we do in the long term, I'd put a higher priority on lowering defect rates and making it easier to produce working software.
- moocowtruck 9y ago
  i'd like to know how we make correct software
  - lmm 9y ago
    Type systems and languages that make them easy to use. I mean, other approaches are possible, but type systems are something we already all use and understand, and they seem to be enough.
- blub 9y ago
  This "make it work" adage makes no sense at all when scrutinized. Does anyone believe that other industries think like that? First let's invent a washing machine that washes, but occasionally sets clothes on fire and takes 24h for a washing cycle. Then we'll redesign it so it doesn't set things on fire, and finally redesign it yet again to finish in 2h. It's incredibly wasteful. For any complex project, making it work and fast only at the end will either result in massive cost overruns or an outright canceled project.
  - lmm 9y ago
    > Does anyone believe that other industries think like that? Software is very unlike other industries. > For any complex project, making it work and fast only at the end will either result in massive cost overruns or an outright canceled project. Citation needed, because my experience is the exact opposite: projects that put effort into making it fast from the first never get off the ground.
    - blub 9y ago
      Software is different, but it doesn't bend space and time. Redesign costs can be significantly cheaper, but they are still present, and if the project is complex enough can be just as high as in physical world project catastrophes. I assume you have heard of various software projects which cost millions and overshot their budget by 50%, 100%, 200%, etc... Now regarding the make -> work -> fast cliché: I tend to have architectural discussions at the beginning of the project, which include among other things performance. Depending on the project it might be a surface discussion, or go deeper. Then based on the performance requirements and design decisions, the system is implemented, performance is tracked and adjustments are made. So performance should not be put first, middle or last, it's an architectural level concern which is controlled throughout the project. How do those of us that like to complain about "premature optimisation" whenever performance is discussed design software? Because based on HN discussions it looks like code & pray.
      - lmm 9y ago
        > Redesign costs can be significantly cheaper, but they are still present, and if the project is complex enough can be just as high as in physical world project catastrophes. I assume you have heard of various software projects which cost millions and overshot their budget by 50%, 100%, 200%, etc... Indeed; in my experience such failures tend to be caused by too much rather than too little design and architecture up front. > How do those of us that like to complain about "premature optimisation" whenever performance is discussed design software? Because based on HN discussions it looks like code & pray. Don't try to design at the start when you don't know anything; let the design emerge. Do the simplest thing that might possibly work; 90% of the time it does work, the other 10% of the time you learn about another constraint. Get the simplest use case working, then iteratively expand to the full functionality. Refactor continuously and fearlessly (and adopt whatever testing/verification practices you need to make it fearless), driven by the changes that you need to make. It's easy to mock, but it works much better than trying to design as a separate activity.
- deleted 9y ago
  [deleted]
theli0nheart 9y ago
You're conflating Python the language and Python the default runtime implementation (CPython). PyPy, a Python JIT compiler, has shown you can have an incredibly fast Python implementation. In some cases, it's faster than C. Maybe this is what you're looking for. :) http://speed.pypy.org http://speed.pypy.org
- blub 9y ago
  Python and CPython are the same thing for the majority of Python developers, just like the Oracle JVM is a synonym of Java. Yes, there are other exotic runtimes, but only one official one and that's what people will use.
  - theli0nheart 9y ago
    I get that, and I agree, but my point was that if you're writing Python and finding that your code is too slow, it's far easier to just drop in a faster runtime than it is to rewrite your entire existing codebase in a new language.
    - MR4D 9y ago
      Not if you deploy your code to environments you don’t fully control.
      - theli0nheart 9y ago
        Well, then you're out of luck. :)
DrJosiah 9y ago
As a guy who just spent the past summer doing (his 19th year of) Python, (6rd year of) Cython(/Pyrex), and (8th year of) C, picking around PyPy, and writing a couple transpilers; the tools are incredible. And I think you're talking out your butt here. Speed is a matter of choice and a bit of manual tooling. You seem like a Golang advocate; great. I hope it continues to work for you. Python solves 95% of problems today, and with the continued bettering of tooling (via PyPy, Cython, Pandas, SciPy, PyTorch, etc.), Python and the dozens of active member groups (individually with 100k+ global users) work independently in every niche with performance improvements until it takes the market. We embrace, extend, wrap, and improve. I remember when Unladen Swallow was laid upon the laps of the CPython developers. That was a LLVM backend JIT for CPython (CPython had another JIT at the time, Psyco), combined with a collection of other improvements to the CPython runtime that Google laid on the CPython developer's laps. It was a patchset against an old (and unmaintained) version of CPython, and may have taken years of further development to merge back in. Meanwhile PyPy, Pyrex->Cython, etc., were all pretty solid, so the CPython devs left the Unlaiden Swallow, because it didn't seem worth it (I agree with them). See PEP 3146 for details. But that doesn't mean CPython devs haven't been at it; what the heck do you think all of those "optional" type annotations are for in Python 3.x? Type checkers for one, compilers for two. Oh... yeah, we've been moving towards optional static type checkers and static compilation in the Python community for years; and as a community, you can basically piece it all together. Is it 100% yet? No, but it's a solid 95% for most use-cases. Watching Golang over the years; it feels a lot like if it wasn't in Golang, it wasn't worth using with Go. I say this as a user of cgo from 2012-2013 in a failed attempt to make something in Golang + sockets + goroutines faster than the equivalent in Python + sockets + threads + C. But when I was seeing better performance in Python, better error reporting in Python, better C library wrapping in Python, and better tooling in Python - I went back to Python (funny how 20+ years of tooling will do that). Don't get me wrong, I love the speed of the Golang compiler. It's just mostly everything else in the ecosystem I don't like; including the lack of a viable C/Golang interface for anything nontrivial (like leaving C threads running with references to Golang objects/structres), the hilariously short official documentation for cgo, the basic need to re-implement the world in Golang to get good performance, and still being subject to the whims of Rob Pike - whose bad decisions (in the form of Sawzall) already wasted a week of my life when I was at Google. You want performance in Python? Okay. Where do you want performance? If there isn't already a library there to help you, I'd be very surprised.
ziftface 9y ago
I agree that we can do better. But I don't think you'll ever be able to avoid some trade-off between CPU performance and ease-of-use. We might shrink the gap, but there will always be a market for languages that allow easy prototyping and new developer onboarding, and those languages will always be slower than C++.

lmm 9y ago

> It is a failure of our community that we allow languages to proliferate while remaining slow. This is bad because allowing slow tools to become popular means people create slow things, which wastes other peoples time and energy. Make it work, then make it work right, then make it work fast. I mean yes, a lot of things are slower than they should be, but the level of outright correctness bugs in software today is mindblowing. So while replacing our tools with faster tools should be something we do in the long term, I'd put a higher priority on lowering defect rates and making it easier to produce working software.

moocowtruck 9y ago
i'd like to know how we make correct software
- lmm 9y ago
  Type systems and languages that make them easy to use. I mean, other approaches are possible, but type systems are something we already all use and understand, and they seem to be enough.
blub 9y ago
This "make it work" adage makes no sense at all when scrutinized. Does anyone believe that other industries think like that? First let's invent a washing machine that washes, but occasionally sets clothes on fire and takes 24h for a washing cycle. Then we'll redesign it so it doesn't set things on fire, and finally redesign it yet again to finish in 2h. It's incredibly wasteful. For any complex project, making it work and fast only at the end will either result in massive cost overruns or an outright canceled project.
- lmm 9y ago
  > Does anyone believe that other industries think like that? Software is very unlike other industries. > For any complex project, making it work and fast only at the end will either result in massive cost overruns or an outright canceled project. Citation needed, because my experience is the exact opposite: projects that put effort into making it fast from the first never get off the ground.
  - blub 9y ago
    Software is different, but it doesn't bend space and time. Redesign costs can be significantly cheaper, but they are still present, and if the project is complex enough can be just as high as in physical world project catastrophes. I assume you have heard of various software projects which cost millions and overshot their budget by 50%, 100%, 200%, etc... Now regarding the make -> work -> fast cliché: I tend to have architectural discussions at the beginning of the project, which include among other things performance. Depending on the project it might be a surface discussion, or go deeper. Then based on the performance requirements and design decisions, the system is implemented, performance is tracked and adjustments are made. So performance should not be put first, middle or last, it's an architectural level concern which is controlled throughout the project. How do those of us that like to complain about "premature optimisation" whenever performance is discussed design software? Because based on HN discussions it looks like code & pray.
    - lmm 9y ago
      > Redesign costs can be significantly cheaper, but they are still present, and if the project is complex enough can be just as high as in physical world project catastrophes. I assume you have heard of various software projects which cost millions and overshot their budget by 50%, 100%, 200%, etc... Indeed; in my experience such failures tend to be caused by too much rather than too little design and architecture up front. > How do those of us that like to complain about "premature optimisation" whenever performance is discussed design software? Because based on HN discussions it looks like code & pray. Don't try to design at the start when you don't know anything; let the design emerge. Do the simplest thing that might possibly work; 90% of the time it does work, the other 10% of the time you learn about another constraint. Get the simplest use case working, then iteratively expand to the full functionality. Refactor continuously and fearlessly (and adopt whatever testing/verification practices you need to make it fearless), driven by the changes that you need to make. It's easy to mock, but it works much better than trying to design as a separate activity.
deleted 9y ago
[deleted]

theli0nheart 9y ago

You're conflating Python the language and Python the default runtime implementation (CPython). PyPy, a Python JIT compiler, has shown you can have an incredibly fast Python implementation. In some cases, it's faster than C. Maybe this is what you're looking for. :) http://speed.pypy.org http://speed.pypy.org

blub 9y ago
Python and CPython are the same thing for the majority of Python developers, just like the Oracle JVM is a synonym of Java. Yes, there are other exotic runtimes, but only one official one and that's what people will use.
- theli0nheart 9y ago
  I get that, and I agree, but my point was that if you're writing Python and finding that your code is too slow, it's far easier to just drop in a faster runtime than it is to rewrite your entire existing codebase in a new language.
  - MR4D 9y ago
    Not if you deploy your code to environments you don’t fully control.
    - theli0nheart 9y ago
      Well, then you're out of luck. :)

DrJosiah 9y ago

As a guy who just spent the past summer doing (his 19th year of) Python, (6rd year of) Cython(/Pyrex), and (8th year of) C, picking around PyPy, and writing a couple transpilers; the tools are incredible. And I think you're talking out your butt here. Speed is a matter of choice and a bit of manual tooling. You seem like a Golang advocate; great. I hope it continues to work for you. Python solves 95% of problems today, and with the continued bettering of tooling (via PyPy, Cython, Pandas, SciPy, PyTorch, etc.), Python and the dozens of active member groups (individually with 100k+ global users) work independently in every niche with performance improvements until it takes the market. We embrace, extend, wrap, and improve. I remember when Unladen Swallow was laid upon the laps of the CPython developers. That was a LLVM backend JIT for CPython (CPython had another JIT at the time, Psyco), combined with a collection of other improvements to the CPython runtime that Google laid on the CPython developer's laps. It was a patchset against an old (and unmaintained) version of CPython, and may have taken years of further development to merge back in. Meanwhile PyPy, Pyrex->Cython, etc., were all pretty solid, so the CPython devs left the Unlaiden Swallow, because it didn't seem worth it (I agree with them). See PEP 3146 for details. But that doesn't mean CPython devs haven't been at it; what the heck do you think all of those "optional" type annotations are for in Python 3.x? Type checkers for one, compilers for two. Oh... yeah, we've been moving towards optional static type checkers and static compilation in the Python community for years; and as a community, you can basically piece it all together. Is it 100% yet? No, but it's a solid 95% for most use-cases. Watching Golang over the years; it feels a lot like if it wasn't in Golang, it wasn't worth using with Go. I say this as a user of cgo from 2012-2013 in a failed attempt to make something in Golang + sockets + goroutines faster than the equivalent in Python + sockets + threads + C. But when I was seeing better performance in Python, better error reporting in Python, better C library wrapping in Python, and better tooling in Python - I went back to Python (funny how 20+ years of tooling will do that). Don't get me wrong, I love the speed of the Golang compiler. It's just mostly everything else in the ecosystem I don't like; including the lack of a viable C/Golang interface for anything nontrivial (like leaving C threads running with references to Golang objects/structres), the hilariously short official documentation for cgo, the basic need to re-implement the world in Golang to get good performance, and still being subject to the whims of Rob Pike - whose bad decisions (in the form of Sawzall) already wasted a week of my life when I was at Google. You want performance in Python? Okay. Where do you want performance? If there isn't already a library there to help you, I'd be very surprised.

ziftface 9y ago

I agree that we can do better. But I don't think you'll ever be able to avoid some trade-off between CPU performance and ease-of-use. We might shrink the gap, but there will always be a market for languages that allow easy prototyping and new developer onboarding, and those languages will always be slower than C++.