6 ms·

Blog author here. I somewhat agree, somewhat disagree. This line makes me uneasy: > I always ask them to either prove it or write the simple thing. If the code

Blog author here. I somewhat agree, somewhat disagree. This line makes me uneasy: > I always ask them to either prove it or write the simple thing. If the code in question isn't hot enough to bother benchmarking it, the performance benefits probably aren't worth it _even if they exist_. One of my philosophies is that death by a thousand cuts is fine, but death by ten thousand cuts isn’t. A team of 10 engineers can probably fix most of a thousand cuts in two or three months. But if you have ten thousand cuts you’re probably doomed. And those don’t show up cleanly in a flame graph. Now for some context my background is video games. Which means the team knows they need to hit an aggressive performance bar. This isn’t true for many projects. shared_ptr is a canonical example of death by ten thousand cuts. That said, I strongly agree with the principle of “just do the simple thing”. However I think it’s important to have “sane defaults”. A project can easily have a thousand or ten thousand papercuts that kill performance. But you can’t microbench every tiny decision. And microbenches are only a vague approximation of what actually matters. I’m also wary of “the compiler will make it fast”. Because that’s true… until it’s not! Although these days you don’t have any choice but to lean heavily on the compiler and “trust but verify”. No one wants a super complex solution if it’s not needed. However I am very amenable to “do a slightly more complex thing if you know it’s correct and we can never think about this ever again”. It’s much easier to do the fast thing upfront than for someone else to try and speed it up in two years when we’re doing a papercut pass.

klyrs 4y ago
> But if you have ten thousand cuts you’re probably doomed. And those don’t show up cleanly in a flame graph. I am reminded of the lovely nanosecond/microsecond talk by Grace Hopper. If your code does a little bit of setup and then spends all of its time in a single hotspot, fine. But if your code is full of microsecond-suboptimal speed bumps, you can probably hide your hotspot altogether. And a flat-ish flame graph looks fine: nothing stands out as a problem! It's valuable to do micro-benchmarks, not just to hone your optimization skills, but to learn optimal patterns in your language of choice. Then, when you're "in the zone" and laying down new code, you just do the optimal thing reflexively. Or, when you're reviewing or rewriting something, those micro-hotspots jump out and grab your attention. There's a reason that ancient software running on ancient hardware is way more responsive & snappy than what we have today. Laziness.
- Dylan16807 4y ago
  > There's a reason that ancient software running on ancient hardware is way more responsive & snappy than what we have today. Laziness. Laziness in terms of using entirely inappropriate algorithms, sure. Laziness in not microbenchmarking minutia? It shouldn't be. There's a limit on how much that can hurt you. I would say much less than a factor of ten, but let's go with 10x just for argument's sake. If you have a CPU that's 500x faster, and use easy code that's 10x slower, you're doing just fine. This is not the problem with modern unresponsiveness.
  - klyrs 4y ago
    When I rewrite python code in C, I often hit 1000x speedups, and sub-100x is rare. And that's line-for-line. When I fix an accidentally-quadratic issue, for example, I've seen speedups in the billions without even changing the language. People have lionized Knuth's quote about premature optimization, and used that to ignore performance issues across the board. Since the early '00s, we have not seen a 500x improvement in CPU speed. It's less than 2x on frequency, and let's say 8x on core-count for most users (which doesn't help your single-core lazy programmer). In my experience, programmers will make projections based on a 500x-faster processors that will never arrive, because it's easier than honing their skills and keeping them sharp. And even if these magical THz-frequency chips arrive, if you have three layers of 10x slowdowns, you're back down to GHz.
    - josephg 4y ago
      This has been my experience too. I wrote a text crdt last year which improved automerge’s (then) 5 minute runtime. My code currently takes 6ms to do the same work. Automerge’s design assumed this stuff would always be slow, so they had this whole frontend / backend code split so they could put the expensive operations on worker threads. Good optimizations in the right places make all that complexity unnecessary. The new automerge is shaping up to be simpler as well as faster.
    - Dylan16807 4y ago
      > When I rewrite python code in C, I often hit 1000x speedups, and sub-100x is rare. And that's line-for-line. When I fix an accidentally-quadratic issue, for example, I've seen speedups in the billions without even changing the language. And neither of those is a microbenchmark thing, which is kind of my point. I'm surprised language would hurt that much, but that's enough to break things on its own without any layering. > Since the early '00s, we have not seen a 500x improvement in CPU speed. It's less than 2x on frequency, and let's say 8x on core-count for most users (which doesn't help your single-core lazy programmer). I don't think people are talking about 2004 when they talk about the responsiveness of ancient software on ancient hardware. I interpret that as more like an Apple II. But instructions per clock have also gone up a lot since the pentium 4 days, and having more than one core in your CPU has a huge impact even for single-threaded programs.
Dylan16807 4y ago
Very little of your code is in hot loops. If the code that takes half a millisecond per frame could be twice as fast, but the hot loop is very optimized, then it doesn't really matter. And that's what I would think of by default for having many many cuts. Better to spend the optimization effort elsewhere. > shared_ptr is a canonical example of death by ten thousand cuts Why does that count as ten thousand cuts rather than one cut? That doesn't sound intractable to fix if you have months.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
- kllrnohj 4y ago
  > Very little of your code is in hot loops. Very few programs have "hot loops" as small, contained things. "hot loops" is at this point as much of a bad trope as "but premature optimizations!" is. > Why does that count as ten thousand cuts rather than one cut? That doesn't sound intractable to fix if you have months. Why spend months re-writing code you could have just written correctly the first time if a single person had spent an hour messing around with benchmarks to figure out the proper guidance?
  - Dylan16807 4y ago
    > Very few programs have "hot loops" as small, contained things. "hot loops" is at this point as much of a bad trope as "but premature optimizations!" is. If you say so. I can pretty easily and meaningfully divide my current program into "happens once per frame or less" and "happens hundreds or more times in a row". > Why spend months re-writing code you could have just written correctly the first time if a single person had spent an hour messing around with benchmarks to figure out the proper guidance? If they could have figured it out that easily I understand even less how this example works. Let me put it this way: If a benchmark can change huge swaths of significantly performance important code, then it's "hot enough to bother" by orders of magnitude. I thought we were talking about microbenchmarks for individual cuts!
bjackman 4y ago
Haha "death by a thousand cuts" is exactly the phrase I encounter in these debates! And actually I still disagree - e.g. I once took over a DMA management firmware and the TL told me "we are really trying to avoid DBATC so we take care to write every line efficiently". But the thing was that once you have a holistic understanding of the systems performance you tend to find _only a small fraction of the code ever affects the metrics you care about_! E.g. in that case the CPU was so rarely the bottleneck that it really didn't matter, we could have rewritten half the code in Python (if we'd had the memory) without hurting latency or throughput. Admittedly I can see how games or like JS engines might be a kinda special case here, where the OVERALL compute bandwidth begins to become a concern (almost like an HPC system) and maybe then every line really does count.

klyrs 4y ago

> But if you have ten thousand cuts you’re probably doomed. And those don’t show up cleanly in a flame graph. I am reminded of the lovely nanosecond/microsecond talk by Grace Hopper. If your code does a little bit of setup and then spends all of its time in a single hotspot, fine. But if your code is full of microsecond-suboptimal speed bumps, you can probably hide your hotspot altogether. And a flat-ish flame graph looks fine: nothing stands out as a problem! It's valuable to do micro-benchmarks, not just to hone your optimization skills, but to learn optimal patterns in your language of choice. Then, when you're "in the zone" and laying down new code, you just do the optimal thing reflexively. Or, when you're reviewing or rewriting something, those micro-hotspots jump out and grab your attention. There's a reason that ancient software running on ancient hardware is way more responsive & snappy than what we have today. Laziness.

Dylan16807 4y ago
> There's a reason that ancient software running on ancient hardware is way more responsive & snappy than what we have today. Laziness. Laziness in terms of using entirely inappropriate algorithms, sure. Laziness in not microbenchmarking minutia? It shouldn't be. There's a limit on how much that can hurt you. I would say much less than a factor of ten, but let's go with 10x just for argument's sake. If you have a CPU that's 500x faster, and use easy code that's 10x slower, you're doing just fine. This is not the problem with modern unresponsiveness.
- klyrs 4y ago
  When I rewrite python code in C, I often hit 1000x speedups, and sub-100x is rare. And that's line-for-line. When I fix an accidentally-quadratic issue, for example, I've seen speedups in the billions without even changing the language. People have lionized Knuth's quote about premature optimization, and used that to ignore performance issues across the board. Since the early '00s, we have not seen a 500x improvement in CPU speed. It's less than 2x on frequency, and let's say 8x on core-count for most users (which doesn't help your single-core lazy programmer). In my experience, programmers will make projections based on a 500x-faster processors that will never arrive, because it's easier than honing their skills and keeping them sharp. And even if these magical THz-frequency chips arrive, if you have three layers of 10x slowdowns, you're back down to GHz.
  - josephg 4y ago
    This has been my experience too. I wrote a text crdt last year which improved automerge’s (then) 5 minute runtime. My code currently takes 6ms to do the same work. Automerge’s design assumed this stuff would always be slow, so they had this whole frontend / backend code split so they could put the expensive operations on worker threads. Good optimizations in the right places make all that complexity unnecessary. The new automerge is shaping up to be simpler as well as faster.
  - Dylan16807 4y ago
    > When I rewrite python code in C, I often hit 1000x speedups, and sub-100x is rare. And that's line-for-line. When I fix an accidentally-quadratic issue, for example, I've seen speedups in the billions without even changing the language. And neither of those is a microbenchmark thing, which is kind of my point. I'm surprised language would hurt that much, but that's enough to break things on its own without any layering. > Since the early '00s, we have not seen a 500x improvement in CPU speed. It's less than 2x on frequency, and let's say 8x on core-count for most users (which doesn't help your single-core lazy programmer). I don't think people are talking about 2004 when they talk about the responsiveness of ancient software on ancient hardware. I interpret that as more like an Apple II. But instructions per clock have also gone up a lot since the pentium 4 days, and having more than one core in your CPU has a huge impact even for single-threaded programs.

Dylan16807 4y ago

Very little of your code is in hot loops. If the code that takes half a millisecond per frame could be twice as fast, but the hot loop is very optimized, then it doesn't really matter. And that's what I would think of by default for having many many cuts. Better to spend the optimization effort elsewhere. > shared_ptr is a canonical example of death by ten thousand cuts Why does that count as ten thousand cuts rather than one cut? That doesn't sound intractable to fix if you have months.

deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
kllrnohj 4y ago
> Very little of your code is in hot loops. Very few programs have "hot loops" as small, contained things. "hot loops" is at this point as much of a bad trope as "but premature optimizations!" is. > Why does that count as ten thousand cuts rather than one cut? That doesn't sound intractable to fix if you have months. Why spend months re-writing code you could have just written correctly the first time if a single person had spent an hour messing around with benchmarks to figure out the proper guidance?
- Dylan16807 4y ago
  > Very few programs have "hot loops" as small, contained things. "hot loops" is at this point as much of a bad trope as "but premature optimizations!" is. If you say so. I can pretty easily and meaningfully divide my current program into "happens once per frame or less" and "happens hundreds or more times in a row". > Why spend months re-writing code you could have just written correctly the first time if a single person had spent an hour messing around with benchmarks to figure out the proper guidance? If they could have figured it out that easily I understand even less how this example works. Let me put it this way: If a benchmark can change huge swaths of significantly performance important code, then it's "hot enough to bother" by orders of magnitude. I thought we were talking about microbenchmarks for individual cuts!

bjackman 4y ago

Haha "death by a thousand cuts" is exactly the phrase I encounter in these debates! And actually I still disagree - e.g. I once took over a DMA management firmware and the TL told me "we are really trying to avoid DBATC so we take care to write every line efficiently". But the thing was that once you have a holistic understanding of the systems performance you tend to find _only a small fraction of the code ever affects the metrics you care about_! E.g. in that case the CPU was so rarely the bottleneck that it really didn't matter, we could have rewritten half the code in Python (if we'd had the memory) without hurting latency or throughput. Admittedly I can see how games or like JS engines might be a kinda special case here, where the OVERALL compute bandwidth begins to become a concern (almost like an HPC system) and maybe then every line really does count.