5 ms·

The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honest

The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honestly think correct, 10-15 years ago. But things have changed. CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed really is more expensive than programmer time, if you are running that code at a big enough scale. Add in containerization and cloud VM platforms where the tradeoffs of space and performance versus money start to become very clear. Add in better, safer languages like Rust and Go for writing high-performance code. And today if you can spend three times the programming time writing in a faster and more efficient language, and it runs 100x as fast in a tenth the memory footprint, you are talking about massive overall cost savings.

commandlinefan 9y ago
I've been "premature optimization is the root of all evil"ed in more code reviews than I can count - yet when people complain about software, they first complain that it's ugly and then they complain that it's slow. Users don't care what it's written in.
- sitkack 9y ago
  There is fast (optimization) and slow (the most straightforward thing with no consideration to perf) and then not-slow. I think not-slow is a happy medium. Fast code should and probably is, totally unreadable, looking like AlphaGo optimized your program.
  - smitherfield 9y ago
    >Fast code should and probably is, totally unreadable, looking like AlphaGo optimized your program. Yes and no; multithreading and optimizing to remove branches can do a number on readability, but I find optimizing for cache locality and compile-time evaluation often makes code more readable. It depends on the language obviously.
  - commandlinefan 9y ago
    In theory I think you're correct - you _can_ go crazy overoptimizing code to the point where it's completely unreadable. However, in my experience, it takes a lot of effort to get to this point, especially in modern languages. It seems to me that most developers err in the other direction: whatever works as long as it took the least amount of time to program (although this might well be partly driven by the management "agile" obsession with accounting for how every 15-minute increment of time was spent).
- snowwrestler 9y ago
  If your users are complaining that your software is slow, then optimization would not be premature. Optimization is premature when the cost of creating or managing it outweighs the business benefits it provides.
theli0nheart 9y ago
> CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed really is more expensive than programmer time, if you are running that code at a big enough scale. When you're starting a startup, scaling out your application to hundreds, thousands, or millions of servers isn't something you're going to do right off the bat, and more likely, that's never going to happen, no matter how successful your company is. The number of companies operating at that sort of scale can be counted on one hand. Most startups can run on a few boxes. For those situations (which probably pertains to most new projects outside of big companies), programmer time is indeed, still the most important and costly input. If you're a startup and your 100x more performant Go code takes an extra 3-6 months to write, and a competitor beats you to market, no one is going to care how much faster your runtime performance is on the CPU, especially if you're a web app, when CPU time is probably should be last on the list of items which could lead to slow application performance to end users. 100x difference in CPU time is nothing compared to the 1000x loss in a cache hit or a 10000x disk read. I'd love to see an example where the CPU difference outweighs any influence from disk or memory.
- _dps 9y ago
  I was with you up until the last paragraph. Taken literally you seem to suggest there is no such thing as a CPU-bound workload. That's obviously not the case (cryptography is just one such example), but I would agree that many people think they are CPU-bound when they are really constrained by something else. Secondly, Python and the patterns its expressiveness encourages are terrible for cache performance. In a simple C program it's easy to do something non-trivial in the space provided by L1 cache — in Python it's quite difficult even to reason about what's going to be in L1 if you're using any of the fancy features.
  - smitherfield 9y ago
    >in Python it's quite difficult even to reason about what's going to be in L1 The interpreter's stack?
bunderbunder 9y ago
I also firmly believe that "easy to write but slow languages" sets up a false dilemma. Haskell, JavaScript and C# are all fairly instructive examples. Haskell for demonstrating the level of flexibility you can achieve with a really well-thought-out static type system. JavaScript for showing how much of the "dynamic tax" you can avoid at run-time with an aggressive enough JIT compiler. C# for showing that you can get a pretty nice final result out of pragmatically blending a little bit of dynamic typing and a little bit of the ML-style static typing experience into a language that started out with a Java-style type system.
smoe 9y ago
How many people are there who actually need to scale an application across hundreds, thousands, or millions of servers vs the ones that think they are going to need it? I agree that at some point CPU speed is more expensive than programming time, but how many applications in the wild are actually at or beyond that point? I would be surprised if it is more than a couple of percent. Sure if I know that a product has to serve millions from the get go, I'd choose a more performance focused language than Python. But if it is about building an MVP for a new startup it seems quite unreasonable to me to spent 3x the time to get it out of the door just for the extremely slim chance that the product takes off faster than Python is able to catch up with.
- stuaxo 9y ago
  I'd quite like to be able to write some plain python (not PIL or numpy or Cython or whatever) to change all the pixels in an image at a similar speed to say Java.
- walterstucco 9y ago
  The argument works both ways Imagine you spend $200 a month on Amazon and can cut it to just $50 with a faster and less memory taxing language You suddenly have more money to spend on something else
bbatha 9y ago
> The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honestly think correct, 10-15 years ago. But things have changed. I don't think things have fundamentally changed in the programmer time is cheaper than cpu time calculation. What has changed is: 1. Classic dynamic languages (ruby, python, etc) all heavily assume the world is single threaded and blocking io can be scaled well enough. Unfortunately, these maxims from the late 90s do not hold about modern CPUs. Languages that embrace non-blocking io such as node.js have been incredibly successful as a result. 2. Software projects have gotten bigger. Extremely anecdotally projects in dynamic languages get uncomfortable with about 10,000 lines and unworkable at about 100,000 lines. A lot of the popularity of these dynamic languages was that you didn't need to type out your types all the time. Newer languages languages have shown have been able to get the best of both worlds with type inference and structural typing. Typescript's meteoric rise is because of exactly this. Go and to a lesser extent Rust, have recognized these two issues facing last generation dynamic languages and have been successful because they address these issues head on. You can start a new Go application and have a similar time to market as python or ruby, but have it scale with your company, both in developer time and performance, for a long time. Why wouldn't you choose Go or some other new language?
- halbritt 9y ago
  To your point 1: This is the reason that AsyncIO exists in Python 3.6 and IIRC Facebook is pushing really, really hard to adopt it.
  - bbatha 9y ago
    Absolutely, but its arguably 10 years too late. Node has already eaten python's lunch in the server space. To get your existing python code working on async you may basically need a full rewrite including any libraries that you pulled in. At that point most companies ask themselves if python is really the right language to conduct a rewrite in.
    - deleted 9y ago
      [deleted]
abuckenheimer 9y ago
When people say cpython is slow they are generally pointing to two things 1) The interpreter is _slow_ 2) You can't achieve real thread based parallelism I think in general people have a high level concept of (1) as static vs. dynamic and interpreted vs compiled is an understandable trade off. CPython as an implementation generally gets type casted as slow for its default dynamism which have understandable negative performance implications[1]. However CPython also gives you lots of ergonomic ways to push your program towards the static/compiled end of the spectrum with things like pandas/numpy/numba/extensions. In general using these correctly put you within the ballpark of _faster_ languages, could you write faster assembly by hand? Sure! Is optimizing in another language worth your time? I don't know. I've never really understood (2) the lack of threading as a problem, multi process parallelism can be accomplished fairly easily if you are CPU bound or projects like uvloop[2] make async tasks fast enough to compete with any web framework out there. Furthermore even though your cpu cycle cost may be levered to a considerable point while operating over hundreds/thousands of servers doing distributed computing right is still hard and developing something like celery/airflow/luigi/dask from scratch is not cheap either. Leaning on CPython's massive ecosystem can massively lower the barrier to entry in a lot of big problems. I think there are plenty of examples of re-writes in go[3]/rust[4] that have worked great for people, I have no doubt that python is _not_ the end all be all language, but I think the "python is slow" worry is generally overplayed. Specific problems require specific solutions, I'm glad Haskell seems to work for the author in genetic analysis but think this could have been a more interesting article with some specific python-haskell comparisons rather than the generic python is slow argument. [1] http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/ http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/ [2] https://magic.io/blog/uvloop-blazing-fast-python-networking/ https://magic.io/blog/uvloop-blazing-fast-python-networking/ [3] https://web.archive.org/web/20170101002625/http://blog.parse.com/learn/how-we-moved-our-api-from-ruby-to-go-and-saved-our-sanity/ https://web.archive.org/web/20170101002625/http://blog.parse... [4] https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pocket/ https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pock...

commandlinefan 9y ago

I've been "premature optimization is the root of all evil"ed in more code reviews than I can count - yet when people complain about software, they first complain that it's ugly and then they complain that it's slow. Users don't care what it's written in.

sitkack 9y ago
There is fast (optimization) and slow (the most straightforward thing with no consideration to perf) and then not-slow. I think not-slow is a happy medium. Fast code should and probably is, totally unreadable, looking like AlphaGo optimized your program.
- smitherfield 9y ago
  >Fast code should and probably is, totally unreadable, looking like AlphaGo optimized your program. Yes and no; multithreading and optimizing to remove branches can do a number on readability, but I find optimizing for cache locality and compile-time evaluation often makes code more readable. It depends on the language obviously.
- commandlinefan 9y ago
  In theory I think you're correct - you _can_ go crazy overoptimizing code to the point where it's completely unreadable. However, in my experience, it takes a lot of effort to get to this point, especially in modern languages. It seems to me that most developers err in the other direction: whatever works as long as it took the least amount of time to program (although this might well be partly driven by the management "agile" obsession with accounting for how every 15-minute increment of time was spent).
snowwrestler 9y ago
If your users are complaining that your software is slow, then optimization would not be premature. Optimization is premature when the cost of creating or managing it outweighs the business benefits it provides.

theli0nheart 9y ago

> CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed really is more expensive than programmer time, if you are running that code at a big enough scale. When you're starting a startup, scaling out your application to hundreds, thousands, or millions of servers isn't something you're going to do right off the bat, and more likely, that's never going to happen, no matter how successful your company is. The number of companies operating at that sort of scale can be counted on one hand. Most startups can run on a few boxes. For those situations (which probably pertains to most new projects outside of big companies), programmer time is indeed, still the most important and costly input. If you're a startup and your 100x more performant Go code takes an extra 3-6 months to write, and a competitor beats you to market, no one is going to care how much faster your runtime performance is on the CPU, especially if you're a web app, when CPU time is probably should be last on the list of items which could lead to slow application performance to end users. 100x difference in CPU time is nothing compared to the 1000x loss in a cache hit or a 10000x disk read. I'd love to see an example where the CPU difference outweighs any influence from disk or memory.

_dps 9y ago
I was with you up until the last paragraph. Taken literally you seem to suggest there is no such thing as a CPU-bound workload. That's obviously not the case (cryptography is just one such example), but I would agree that many people think they are CPU-bound when they are really constrained by something else. Secondly, Python and the patterns its expressiveness encourages are terrible for cache performance. In a simple C program it's easy to do something non-trivial in the space provided by L1 cache — in Python it's quite difficult even to reason about what's going to be in L1 if you're using any of the fancy features.
- smitherfield 9y ago
  >in Python it's quite difficult even to reason about what's going to be in L1 The interpreter's stack?

bunderbunder 9y ago

I also firmly believe that "easy to write but slow languages" sets up a false dilemma. Haskell, JavaScript and C# are all fairly instructive examples. Haskell for demonstrating the level of flexibility you can achieve with a really well-thought-out static type system. JavaScript for showing how much of the "dynamic tax" you can avoid at run-time with an aggressive enough JIT compiler. C# for showing that you can get a pretty nice final result out of pragmatically blending a little bit of dynamic typing and a little bit of the ML-style static typing experience into a language that started out with a Java-style type system.

smoe 9y ago

How many people are there who actually need to scale an application across hundreds, thousands, or millions of servers vs the ones that think they are going to need it? I agree that at some point CPU speed is more expensive than programming time, but how many applications in the wild are actually at or beyond that point? I would be surprised if it is more than a couple of percent. Sure if I know that a product has to serve millions from the get go, I'd choose a more performance focused language than Python. But if it is about building an MVP for a new startup it seems quite unreasonable to me to spent 3x the time to get it out of the door just for the extremely slim chance that the product takes off faster than Python is able to catch up with.

stuaxo 9y ago
I'd quite like to be able to write some plain python (not PIL or numpy or Cython or whatever) to change all the pixels in an image at a similar speed to say Java.
walterstucco 9y ago
The argument works both ways Imagine you spend $200 a month on Amazon and can cut it to just $50 with a faster and less memory taxing language You suddenly have more money to spend on something else

bbatha 9y ago

> The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honestly think correct, 10-15 years ago. But things have changed. I don't think things have fundamentally changed in the programmer time is cheaper than cpu time calculation. What has changed is: 1. Classic dynamic languages (ruby, python, etc) all heavily assume the world is single threaded and blocking io can be scaled well enough. Unfortunately, these maxims from the late 90s do not hold about modern CPUs. Languages that embrace non-blocking io such as node.js have been incredibly successful as a result. 2. Software projects have gotten bigger. Extremely anecdotally projects in dynamic languages get uncomfortable with about 10,000 lines and unworkable at about 100,000 lines. A lot of the popularity of these dynamic languages was that you didn't need to type out your types all the time. Newer languages languages have shown have been able to get the best of both worlds with type inference and structural typing. Typescript's meteoric rise is because of exactly this. Go and to a lesser extent Rust, have recognized these two issues facing last generation dynamic languages and have been successful because they address these issues head on. You can start a new Go application and have a similar time to market as python or ruby, but have it scale with your company, both in developer time and performance, for a long time. Why wouldn't you choose Go or some other new language?

halbritt 9y ago
To your point 1: This is the reason that AsyncIO exists in Python 3.6 and IIRC Facebook is pushing really, really hard to adopt it.
- bbatha 9y ago
  Absolutely, but its arguably 10 years too late. Node has already eaten python's lunch in the server space. To get your existing python code working on async you may basically need a full rewrite including any libraries that you pulled in. At that point most companies ask themselves if python is really the right language to conduct a rewrite in.
  - deleted 9y ago
    [deleted]

abuckenheimer 9y ago

When people say cpython is slow they are generally pointing to two things 1) The interpreter is _slow_ 2) You can't achieve real thread based parallelism I think in general people have a high level concept of (1) as static vs. dynamic and interpreted vs compiled is an understandable trade off. CPython as an implementation generally gets type casted as slow for its default dynamism which have understandable negative performance implications[1]. However CPython also gives you lots of ergonomic ways to push your program towards the static/compiled end of the spectrum with things like pandas/numpy/numba/extensions. In general using these correctly put you within the ballpark of _faster_ languages, could you write faster assembly by hand? Sure! Is optimizing in another language worth your time? I don't know. I've never really understood (2) the lack of threading as a problem, multi process parallelism can be accomplished fairly easily if you are CPU bound or projects like uvloop[2] make async tasks fast enough to compete with any web framework out there. Furthermore even though your cpu cycle cost may be levered to a considerable point while operating over hundreds/thousands of servers doing distributed computing right is still hard and developing something like celery/airflow/luigi/dask from scratch is not cheap either. Leaning on CPython's massive ecosystem can massively lower the barrier to entry in a lot of big problems. I think there are plenty of examples of re-writes in go[3]/rust[4] that have worked great for people, I have no doubt that python is _not_ the end all be all language, but I think the "python is slow" worry is generally overplayed. Specific problems require specific solutions, I'm glad Haskell seems to work for the author in genetic analysis but think this could have been a more interesting article with some specific python-haskell comparisons rather than the generic python is slow argument. [1] http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/ http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/ [2] https://magic.io/blog/uvloop-blazing-fast-python-networking/ https://magic.io/blog/uvloop-blazing-fast-python-networking/ [3] https://web.archive.org/web/20170101002625/http://blog.parse.com/learn/how-we-moved-our-api-from-ruby-to-go-and-saved-our-sanity/ https://web.archive.org/web/20170101002625/http://blog.parse... [4] https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pocket/ https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pock...