4 ms·

> CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've pass

> CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed really is more expensive than programmer time, if you are running that code at a big enough scale. When you're starting a startup, scaling out your application to hundreds, thousands, or millions of servers isn't something you're going to do right off the bat, and more likely, that's never going to happen, no matter how successful your company is. The number of companies operating at that sort of scale can be counted on one hand. Most startups can run on a few boxes. For those situations (which probably pertains to most new projects outside of big companies), programmer time is indeed, still the most important and costly input. If you're a startup and your 100x more performant Go code takes an extra 3-6 months to write, and a competitor beats you to market, no one is going to care how much faster your runtime performance is on the CPU, especially if you're a web app, when CPU time is probably should be last on the list of items which could lead to slow application performance to end users. 100x difference in CPU time is nothing compared to the 1000x loss in a cache hit or a 10000x disk read. I'd love to see an example where the CPU difference outweighs any influence from disk or memory.

_dps 9y ago
I was with you up until the last paragraph. Taken literally you seem to suggest there is no such thing as a CPU-bound workload. That's obviously not the case (cryptography is just one such example), but I would agree that many people think they are CPU-bound when they are really constrained by something else. Secondly, Python and the patterns its expressiveness encourages are terrible for cache performance. In a simple C program it's easy to do something non-trivial in the space provided by L1 cache — in Python it's quite difficult even to reason about what's going to be in L1 if you're using any of the fancy features.
- smitherfield 9y ago
  >in Python it's quite difficult even to reason about what's going to be in L1 The interpreter's stack?
  - _dps 9y ago
    I haven't looked at it in a while, so I could be wrong, but I think with small enough programs you can still squeeze some payload into L1 in long tight loops where you're not jumping up and down the Python stack a lot. But your overall point stands: if you're writing non-trivial Python programs your L1 is usually spent on language/runtime overhead.
    - kevin_thibedeau 9y ago
      When such things matter you drop to Cython and avoid interacting with PyObjects. Then you get native performance for tight loops.
- theli0nheart 9y ago
  Good point. I see what you're saying and I was definitely not suggesting that. I was speaking mainly from a web application perspective, where program execution on the server is very low on the list of items affecting application speed as perceived by the end-user. Scientific computing, on the other hand, is a completely different animal.
uluyol 9y ago
> If you're a startup and your 100x more performant Go code takes an extra 3-6 months to write I can write Go code nearly as fast as Python, sometimes faster if I need to refactor. Obviously this depends on familiarity with languages, but I think most of difference is probably experience with the language more than anything else. > 100x difference in CPU time is nothing compared to the 1000x loss in a cache hit or a 10000x disk read. I'd love to see an example where the CPU difference outweighs any influence from disk or memory. This is a little confusing. A language like Python doesn't just use more/slower instructions to do things: it has worse cache locality too. You always deal with references to object instead of the data. So indirection is everywhere with no way to do anything about it. Primitives use more memory. On my machine, an int requires 28(!) bytes instead of 8. Then there's the fact that you have to fit the interpreter itself in the cache instead of just the code you wrote. If you care about performance, don't use Python. Unless of course you're using numpy or something similar
- jakear 9y ago
  To me the last sentence nullifies the entire argument. People doing a lot of numeric computations aren't (typically) doing it all in Python. They write critical sections in C (or some other "fast" language), or use existing libraries that already have done so. If they aren't writing those sections in C already, perhaps because they don't know how, or it would take too long to do right, then why would they ever choose to do the whole thing in C?
  - Chris2048 9y ago
    Maybe C-devs are easier to find than Python devs with competence in C?
  - apendleton 9y ago
    Firstly, not everyone doing performance-sensitive work is doing numeric work (that seems to have been a motivator for writing this article), so numpy isn't always practical. Secondly, I think the "if it's slow just rewrite the hard parts in C" is generally out of step with more modern options. Python gets you a really nice environment that's super pleasant to work with... until suddenly it doesn't and you're backed into a performance corner, and then you potentially need to conquer a major learning curve, take a huge usability hit, sacrifice memory safety, etc. There are increasingly common options now that let you get near-C performance for many applications while also totally avoiding that cliff. For applications that have even a decent chance of eventually needing that kind of work, I think it's reasonable to ask why you'd want to chance it when you could just write it in something both reasonably fast and much more pleasant than C from the get-go.
theandrewbailey 9y ago
"your 100x more performant Go code" Let me optimize that: "your 100x faster Go code"
weberc2 9y ago
I write Python for my day job, but I often prototype in Go because it's so much easier. Also, regarding I/O, Python does nothing during I/O unless you take care to write async code and only call async libraries. Go does async by default and truly outclasses Python at IO bound tasks. For CPU bound tasks, Go is only 2 orders of magnitude faster. For I/O bound tasks, Go is about 4-6 orders of magnitude faster unless Python is carefully optimized.

_dps 9y ago

I was with you up until the last paragraph. Taken literally you seem to suggest there is no such thing as a CPU-bound workload. That's obviously not the case (cryptography is just one such example), but I would agree that many people think they are CPU-bound when they are really constrained by something else. Secondly, Python and the patterns its expressiveness encourages are terrible for cache performance. In a simple C program it's easy to do something non-trivial in the space provided by L1 cache — in Python it's quite difficult even to reason about what's going to be in L1 if you're using any of the fancy features.

smitherfield 9y ago
>in Python it's quite difficult even to reason about what's going to be in L1 The interpreter's stack?
- _dps 9y ago
  I haven't looked at it in a while, so I could be wrong, but I think with small enough programs you can still squeeze some payload into L1 in long tight loops where you're not jumping up and down the Python stack a lot. But your overall point stands: if you're writing non-trivial Python programs your L1 is usually spent on language/runtime overhead.
  - kevin_thibedeau 9y ago
    When such things matter you drop to Cython and avoid interacting with PyObjects. Then you get native performance for tight loops.
theli0nheart 9y ago
Good point. I see what you're saying and I was definitely not suggesting that. I was speaking mainly from a web application perspective, where program execution on the server is very low on the list of items affecting application speed as perceived by the end-user. Scientific computing, on the other hand, is a completely different animal.

uluyol 9y ago

> If you're a startup and your 100x more performant Go code takes an extra 3-6 months to write I can write Go code nearly as fast as Python, sometimes faster if I need to refactor. Obviously this depends on familiarity with languages, but I think most of difference is probably experience with the language more than anything else. > 100x difference in CPU time is nothing compared to the 1000x loss in a cache hit or a 10000x disk read. I'd love to see an example where the CPU difference outweighs any influence from disk or memory. This is a little confusing. A language like Python doesn't just use more/slower instructions to do things: it has worse cache locality too. You always deal with references to object instead of the data. So indirection is everywhere with no way to do anything about it. Primitives use more memory. On my machine, an int requires 28(!) bytes instead of 8. Then there's the fact that you have to fit the interpreter itself in the cache instead of just the code you wrote. If you care about performance, don't use Python. Unless of course you're using numpy or something similar

jakear 9y ago
To me the last sentence nullifies the entire argument. People doing a lot of numeric computations aren't (typically) doing it all in Python. They write critical sections in C (or some other "fast" language), or use existing libraries that already have done so. If they aren't writing those sections in C already, perhaps because they don't know how, or it would take too long to do right, then why would they ever choose to do the whole thing in C?
- Chris2048 9y ago
  Maybe C-devs are easier to find than Python devs with competence in C?
- apendleton 9y ago
  Firstly, not everyone doing performance-sensitive work is doing numeric work (that seems to have been a motivator for writing this article), so numpy isn't always practical. Secondly, I think the "if it's slow just rewrite the hard parts in C" is generally out of step with more modern options. Python gets you a really nice environment that's super pleasant to work with... until suddenly it doesn't and you're backed into a performance corner, and then you potentially need to conquer a major learning curve, take a huge usability hit, sacrifice memory safety, etc. There are increasingly common options now that let you get near-C performance for many applications while also totally avoiding that cliff. For applications that have even a decent chance of eventually needing that kind of work, I think it's reasonable to ask why you'd want to chance it when you could just write it in something both reasonably fast and much more pleasant than C from the get-go.

theandrewbailey 9y ago

"your 100x more performant Go code" Let me optimize that: "your 100x faster Go code"

weberc2 9y ago

I write Python for my day job, but I often prototype in Go because it's so much easier. Also, regarding I/O, Python does nothing during I/O unless you take care to write async code and only call async libraries. Go does async by default and truly outclasses Python at IO bound tasks. For CPU bound tasks, Go is only 2 orders of magnitude faster. For I/O bound tasks, Go is about 4-6 orders of magnitude faster unless Python is carefully optimized.