17 ms·

It is absolutely true that some hot-path code needs to be mangled into an ugly mess to reach performance requirements. The problem is that I have encountered pe

by ch_123 4y ago

It is absolutely true that some hot-path code needs to be mangled into an ugly mess to reach performance requirements. The problem is that I have encountered people who somehow take this as a blanket justification for writing unreadable code, and who operate on a false dichotomy about a choice between readable code and performant code. It is important to keep in mind that: 1) Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot and does not need specific optimizations (i.e. as long as the code does not do stupidly inefficient things, it's probably good enough). 2) Even in performance hotspot codepath, you should not write unnecessarily hard to read code unless it is strictly necessary to achieve required performance. In both cases, the key point is to benchmark and profile to find the specific places where the ugly hacks need to be introduced, and do not introduce more than is strictly necessary to get the job done.

louthy 4y ago
Agree on this. Although the 80% figure is probably more like 99% (I have no metrics, but 80% feels too low to make this point)
- WalterBright 4y ago
  It's about 99% if you haven't run it through a profiler and optimized the hot spots. With such, you can get it down to 90% or so.
  - pclmulqdq 4y ago
    I have noticed that great code and terrible code often have the same characteristics on profiles: in both cases, slowness is spread around everywhere.
    - preseinger 4y ago
      i've noticed the opposite, usually performance is bottlenecked by specific and unpredictable hot spots
    - kolbe 4y ago
      In this video, David Gross says something along the lines of "if your data is laid out in a memory inefficient manner, it will be the bottleneck everywhere." If you're dependent on a database query for everything, that will be your bottleneck. If your in-memory structures are cache inefficient, that will be your bottleneck. But once you've fixed all of this (if you can), hot paths and whatnot matter https://m.youtube.com/watch?v=8uAW5FQtcvE https://m.youtube.com/watch?v=8uAW5FQtcvE
systematical 4y ago
Saved me a lot of typing.
thethirdone 4y ago
I definitely agree on the false dichotomy between performance and readable code. > 1) Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot and does not need specific optimizations (i.e. as long as the code does not do stupidly inefficient things, it's probably good enough). The hard part is knowing what "stupidly inefficient things" are. If you never do excessive optimization, its easy to drift towards less efficient over time because your baseline for what performance is possible slows down. Knowing how to do performance optimization that is not worth it and knowing what the best level of efficiency to shoot for is the mark of a good engineer.
- jackmott42 4y ago
  Yeah, absent an effort to speed up a code base, it will tend to get slower, unless the team has a culture of performance, everyone is looking for easy performance wins and taking them, and common performance mistakes and getting rid of them. To do that the team needs experience with performance, and most of internet programmer culture is just to lecture people about how programmers are cheaper than hardware if anyone asks or talks about performance. So many people don't get that experience.
  - radlad 4y ago
    > programmers are cheaper than hardware Did you mean the reverse of this, I assume?
    - 908B64B197 4y ago
      Depends on your install base/target. At Apple scale, paying a few performance engineers 1M/y is much cheaper than shipping bibber and more powerful CPU in each iPhone. Same thing for AWS or Azure.
      - josephg 4y ago
        Modern macos is pretty unusably slow on old apple hardware. The same hardware ran earlier versions of the OS just fine. I’m not sure what’s going wrong at apple, but I don’t think they have enough performance engineers working on the Mac. I run a CPU monitor in my dash (and have for years). The efficiency cores are very often busy doing some rubbish tasks I didn’t ask for while my computer is idle. Things like reindexing files, or re-scanning my photos for faces or something like that. I’ve started to think of them as the CPU cores apple put in my computer to keep their useless internal projects from ruining their hardware. Linux annoys me regularly, but at least my cores are almost totally idle whenever my hands aren’t on the keyboard.
dahfizz 4y ago
This makes assumptions about what the performance requirements are. In latency sensitive code, _all_ code needs to be optimized. It doesn't matter if your slow function is only called once - it adds N microseconds of latency when it doesn't need to.
- preseinger 4y ago
  optimizations always have to be data-driven, from profiles, starting at the slowest stuff and working your way down, until you satisfy your latency requirements unless your slow function materially impacts your performance metrics in a way that you can reproducibly demonstrate, it's a mistake to optimize that code to shave off a few microseconds
- __d 4y ago
  If there's a choice of spending X hours on performance tuning, fix the thing which gives the biggest win. There's no sense spending time saving 3us when you could have saved 3ms. And sure, once that's done, you can keep going to picoseconds, but there's only a limited number of application areas where that makes economic sense. It's also important to consider the entire system: you can optimize your code to the picosecond level, occupying the time of expensive engineers for months, but perhaps that $500k is better deployed on a PCIe5 bus, or a better Ethernet card, or a kernel-bypass stack, or even shorter network cables (every 300mm is a nanosecond, after all).
jeffreyrogers 4y ago
The problem is that if you don't know what is required to write fast code you won't design your program to be performant from the start. Lots of applications can be sped up dramatically by rearchitecting them, but you're not going to hit on the right architecture without thinking really hard about memory layout, data access, network usage, etc. It's not really true that 80% of the code doesn't matter. 80% of the code might not matter the way it is currently designed, but that doesn't mean a different design isn't dramatically better. Benchmarking and profiling doesn't help with that. Real expertise is required and that is developed over time by people who care about improving performance and make it a priority.
- szundi 4y ago
  80% does not matter, period. I mean 20% is a lot. Hotspots are - spots. Like 0,5% of your code. Not UI for example. You know, when money is flowing in, you have to hire some good people, let them spend time on fixing hotspots and in some rare occasions, rearchitecture a part or maybe two. Probably won’t pay off, but if you are growing fast enough, it will pay 100x. Also you’re decreasing all future development speed with all this unreadable shit everywhere, so beware of optimizing unnecessary stuff.
  - jeffreyrogers 4y ago
    For seriously high performance systems like webservers or exchanges, you need to get the architecture right from the start. You can't rearchitect a part or two or solve design problems by making skilled hires later on. The 80% part is a distraction (it's also factually untrue that 80% of code doesn't matter. Some applications don't have hotspots, the performance impact is basically smeared all across the code. It really depends on the application. Sqlite found major speedups by implementing hundreds of little changes that were each almost undistinguishable from noise). High performance doesn't mean unreadable. Microoptimizations can make code less readable, true, but they also give you the least amount of speedup (in the best case you can get several times speedup if you found some hotspot that can be vectorized, but that's pretty rare). You can get huge performance improvements by optimizing networking and system calls and doing that doesn't make much of an impact on readability.
zwieback 4y ago
I think that's where the craft of programming comes in. I feel like after 30 years in the field I have a reasonable idea how and when to optimize but it's not something I learned from classes or textbooks.
flavius29663 4y ago
80% seems very low, in my experience it's 1% or less of the code that is a hotspot. On the other hand, we should talk about "doing stupid things", or having a bad design/architecture. For example, if you architect your application such that each web request is serviced by 20 micro-services, and those micro-services make 20 requests of their own...no matter how fast your code is, the application will be slow.
- atq2119 4y ago
  I suspect the 1% number is typical only of programs that haven't been optimized. Once people start to care about performance and address these hotspots, which are often actually quite low hanging fruit, the profile starts to become flatter relatively quickly.
patrulek 4y ago
Its true for services/applications. In a case you write library, others will import into their projects, you should always treat performance seriously imo.
xg15 4y ago
> 1) Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot and does not need specific optimizations (i.e. as long as the code does not do stupidly inefficient things, it's probably good enough). That's true, however, I've seen enough code where the author used this argument as a justification to do stupidly inefficient things, such as building monstrous mountains of abstraction layers (or re-scan the entire codebase at runtime because why not? [1]) all covered by the "premature optimization is the root of all evil" mantra. You can absolutely write code that is both opaque and inefficient. [1] https://docs.spring.io/spring-framework/docs/3.0.0.M4/spring-framework-reference/html/ch03s10.html https://docs.spring.io/spring-framework/docs/3.0.0.M4/spring...
taeric 4y ago
The problem is that so many practices we have gravitated to are actively harmful to performance. Worse, they are borderline harmful for maintenance. Specifically, some practices are better for larger teams than they are smaller teams. Difficulty in that last is that the best practice for how to maintain code changes as you get older on the project. This is easy to see in code that is a bit of a mungled mess. But it is also visible on code that it is a beautiful stalled out mess of needing too much to get small contributions in.
Shorel 4y ago
I agree so much with you, "in theory". In practice, most of the industry has been writing ridiculously inefficient code for decades, and I wish more people would pay attention to that 20%, or the 3% that Knuth described. Or better yet, I wish some groups don't irreversible sacrifice performance with frameworks and architectures that are impossible to optimize, no matter how the code is written.
- commandlinefan 4y ago
  > I wish more people would pay attention to that 20%, I agree with both of you - there's no reason why software should be as inefficient as it is, and there's also no reason why code should be as unreadable as it is, and those goals aren't mutually exclusive.
  - simplotek 4y ago
    > I agree with both of you - there's no reason why software should be as inefficient as it is, Except that it is, and it tends to be development speed. For example, real world React apps tend to suffer a lot from performance issues, specially when compared with vanilla html+javascript or even the old but true server-side rendered HTML, but it also makes it trivial to implement and rewrite complex UIs that UX designers love to put together. Would it make sense to argue for performance options which lead to productivity hits?
    - commandlinefan 4y ago
      > argue for performance options which lead to productivity hits Always.
tombert 4y ago
I had a manager who would write the most hideous, impossible-to-debug code imaginable and always have some sort of microbenchmark as some kind of justification. At the time I was young enough in my career to just think he was wiser than I was, but upon reflection (and remembering some of the stuff he did involving FFI-ing C into Haskell all the time), I realized that he just didn't he didn't like people criticizing his code, though the first hint should have been when he refused to answer my questions about actual performance profiling of our codebase.
commandlinefan 4y ago
> blanket justification for writing unreadable code Well, ironically, the site itself is down (so the code must be beautiful?) but assuming I understand the content from the title, another thing that most of these "code quality doesn't matter" types overlook is that software changes over time, quite a bit. That's why it's called "soft"ware, it's supposed to be soft. Readable is changeable. If all the mattered were performance, we'd engrave the code path onto a circuit board.
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
jjav 4y ago
> Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot Yes, and no. That's what makes it hard to write high performance systems, you really need people with solid experience in designing for performance. Sure, always go for the low hanging fruit which are the hotspots identified by profiling and improve those. But what often happens after that is that all the code is slow everywhere! So there are no big hotspots, profiling shows nothing egregious. Teams without high performance systems experience might conclude the code is about as good as it can get given the absence of hotspots. Seen this happen often. But there might be 10x or 100x improvements remaining.
- bick_nyers 4y ago
  In other words, what is the speed of light solution? The fastest possible solution that could be achieved?
tincholio 4y ago
At the same time, one should not automatically assume that "clean code" is actually readable. I've had to deal with architecture astronauts of the Uncle Bob school of "clean code", and their code was anything but readable.

by ch_123 4y ago

louthy 4y ago

Agree on this. Although the 80% figure is probably more like 99% (I have no metrics, but 80% feels too low to make this point)

WalterBright 4y ago
It's about 99% if you haven't run it through a profiler and optimized the hot spots. With such, you can get it down to 90% or so.
- pclmulqdq 4y ago
  I have noticed that great code and terrible code often have the same characteristics on profiles: in both cases, slowness is spread around everywhere.
  - preseinger 4y ago
    i've noticed the opposite, usually performance is bottlenecked by specific and unpredictable hot spots
  - kolbe 4y ago
    In this video, David Gross says something along the lines of "if your data is laid out in a memory inefficient manner, it will be the bottleneck everywhere." If you're dependent on a database query for everything, that will be your bottleneck. If your in-memory structures are cache inefficient, that will be your bottleneck. But once you've fixed all of this (if you can), hot paths and whatnot matter https://m.youtube.com/watch?v=8uAW5FQtcvE https://m.youtube.com/watch?v=8uAW5FQtcvE

systematical 4y ago

Saved me a lot of typing.

thethirdone 4y ago

I definitely agree on the false dichotomy between performance and readable code. > 1) Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot and does not need specific optimizations (i.e. as long as the code does not do stupidly inefficient things, it's probably good enough). The hard part is knowing what "stupidly inefficient things" are. If you never do excessive optimization, its easy to drift towards less efficient over time because your baseline for what performance is possible slows down. Knowing how to do performance optimization that is not worth it and knowing what the best level of efficiency to shoot for is the mark of a good engineer.

jackmott42 4y ago
Yeah, absent an effort to speed up a code base, it will tend to get slower, unless the team has a culture of performance, everyone is looking for easy performance wins and taking them, and common performance mistakes and getting rid of them. To do that the team needs experience with performance, and most of internet programmer culture is just to lecture people about how programmers are cheaper than hardware if anyone asks or talks about performance. So many people don't get that experience.
- radlad 4y ago
  > programmers are cheaper than hardware Did you mean the reverse of this, I assume?
  - 908B64B197 4y ago
    Depends on your install base/target. At Apple scale, paying a few performance engineers 1M/y is much cheaper than shipping bibber and more powerful CPU in each iPhone. Same thing for AWS or Azure.
    - josephg 4y ago
      Modern macos is pretty unusably slow on old apple hardware. The same hardware ran earlier versions of the OS just fine. I’m not sure what’s going wrong at apple, but I don’t think they have enough performance engineers working on the Mac. I run a CPU monitor in my dash (and have for years). The efficiency cores are very often busy doing some rubbish tasks I didn’t ask for while my computer is idle. Things like reindexing files, or re-scanning my photos for faces or something like that. I’ve started to think of them as the CPU cores apple put in my computer to keep their useless internal projects from ruining their hardware. Linux annoys me regularly, but at least my cores are almost totally idle whenever my hands aren’t on the keyboard.

dahfizz 4y ago

This makes assumptions about what the performance requirements are. In latency sensitive code, _all_ code needs to be optimized. It doesn't matter if your slow function is only called once - it adds N microseconds of latency when it doesn't need to.

preseinger 4y ago
optimizations always have to be data-driven, from profiles, starting at the slowest stuff and working your way down, until you satisfy your latency requirements unless your slow function materially impacts your performance metrics in a way that you can reproducibly demonstrate, it's a mistake to optimize that code to shave off a few microseconds
__d 4y ago
If there's a choice of spending X hours on performance tuning, fix the thing which gives the biggest win. There's no sense spending time saving 3us when you could have saved 3ms. And sure, once that's done, you can keep going to picoseconds, but there's only a limited number of application areas where that makes economic sense. It's also important to consider the entire system: you can optimize your code to the picosecond level, occupying the time of expensive engineers for months, but perhaps that $500k is better deployed on a PCIe5 bus, or a better Ethernet card, or a kernel-bypass stack, or even shorter network cables (every 300mm is a nanosecond, after all).

jeffreyrogers 4y ago

The problem is that if you don't know what is required to write fast code you won't design your program to be performant from the start. Lots of applications can be sped up dramatically by rearchitecting them, but you're not going to hit on the right architecture without thinking really hard about memory layout, data access, network usage, etc. It's not really true that 80% of the code doesn't matter. 80% of the code might not matter the way it is currently designed, but that doesn't mean a different design isn't dramatically better. Benchmarking and profiling doesn't help with that. Real expertise is required and that is developed over time by people who care about improving performance and make it a priority.

szundi 4y ago
80% does not matter, period. I mean 20% is a lot. Hotspots are - spots. Like 0,5% of your code. Not UI for example. You know, when money is flowing in, you have to hire some good people, let them spend time on fixing hotspots and in some rare occasions, rearchitecture a part or maybe two. Probably won’t pay off, but if you are growing fast enough, it will pay 100x. Also you’re decreasing all future development speed with all this unreadable shit everywhere, so beware of optimizing unnecessary stuff.
- jeffreyrogers 4y ago
  For seriously high performance systems like webservers or exchanges, you need to get the architecture right from the start. You can't rearchitect a part or two or solve design problems by making skilled hires later on. The 80% part is a distraction (it's also factually untrue that 80% of code doesn't matter. Some applications don't have hotspots, the performance impact is basically smeared all across the code. It really depends on the application. Sqlite found major speedups by implementing hundreds of little changes that were each almost undistinguishable from noise). High performance doesn't mean unreadable. Microoptimizations can make code less readable, true, but they also give you the least amount of speedup (in the best case you can get several times speedup if you found some hotspot that can be vectorized, but that's pretty rare). You can get huge performance improvements by optimizing networking and system calls and doing that doesn't make much of an impact on readability.

zwieback 4y ago

I think that's where the craft of programming comes in. I feel like after 30 years in the field I have a reasonable idea how and when to optimize but it's not something I learned from classes or textbooks.

flavius29663 4y ago

80% seems very low, in my experience it's 1% or less of the code that is a hotspot. On the other hand, we should talk about "doing stupid things", or having a bad design/architecture. For example, if you architect your application such that each web request is serviced by 20 micro-services, and those micro-services make 20 requests of their own...no matter how fast your code is, the application will be slow.

atq2119 4y ago
I suspect the 1% number is typical only of programs that haven't been optimized. Once people start to care about performance and address these hotspots, which are often actually quite low hanging fruit, the profile starts to become flatter relatively quickly.

patrulek 4y ago

Its true for services/applications. In a case you write library, others will import into their projects, you should always treat performance seriously imo.

xg15 4y ago

> 1) Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot and does not need specific optimizations (i.e. as long as the code does not do stupidly inefficient things, it's probably good enough). That's true, however, I've seen enough code where the author used this argument as a justification to do stupidly inefficient things, such as building monstrous mountains of abstraction layers (or re-scan the entire codebase at runtime because why not? [1]) all covered by the "premature optimization is the root of all evil" mantra. You can absolutely write code that is both opaque and inefficient. [1] https://docs.spring.io/spring-framework/docs/3.0.0.M4/spring-framework-reference/html/ch03s10.html https://docs.spring.io/spring-framework/docs/3.0.0.M4/spring...

taeric 4y ago

The problem is that so many practices we have gravitated to are actively harmful to performance. Worse, they are borderline harmful for maintenance. Specifically, some practices are better for larger teams than they are smaller teams. Difficulty in that last is that the best practice for how to maintain code changes as you get older on the project. This is easy to see in code that is a bit of a mungled mess. But it is also visible on code that it is a beautiful stalled out mess of needing too much to get small contributions in.

Shorel 4y ago

I agree so much with you, "in theory". In practice, most of the industry has been writing ridiculously inefficient code for decades, and I wish more people would pay attention to that 20%, or the 3% that Knuth described. Or better yet, I wish some groups don't irreversible sacrifice performance with frameworks and architectures that are impossible to optimize, no matter how the code is written.

commandlinefan 4y ago
> I wish more people would pay attention to that 20%, I agree with both of you - there's no reason why software should be as inefficient as it is, and there's also no reason why code should be as unreadable as it is, and those goals aren't mutually exclusive.
- simplotek 4y ago
  > I agree with both of you - there's no reason why software should be as inefficient as it is, Except that it is, and it tends to be development speed. For example, real world React apps tend to suffer a lot from performance issues, specially when compared with vanilla html+javascript or even the old but true server-side rendered HTML, but it also makes it trivial to implement and rewrite complex UIs that UX designers love to put together. Would it make sense to argue for performance options which lead to productivity hits?
  - commandlinefan 4y ago
    > argue for performance options which lead to productivity hits Always.

tombert 4y ago

I had a manager who would write the most hideous, impossible-to-debug code imaginable and always have some sort of microbenchmark as some kind of justification. At the time I was young enough in my career to just think he was wiser than I was, but upon reflection (and remembering some of the stuff he did involving FFI-ing C into Haskell all the time), I realized that he just didn't he didn't like people criticizing his code, though the first hint should have been when he refused to answer my questions about actual performance profiling of our codebase.

commandlinefan 4y ago

> blanket justification for writing unreadable code Well, ironically, the site itself is down (so the code must be beautiful?) but assuming I understand the content from the title, another thing that most of these "code quality doesn't matter" types overlook is that software changes over time, quite a bit. That's why it's called "soft"ware, it's supposed to be soft. Readable is changeable. If all the mattered were performance, we'd engrave the code path onto a circuit board.

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

jjav 4y ago

> Most code, i.e. at least 80% of the code in a codebase, will never be a performance hotspot Yes, and no. That's what makes it hard to write high performance systems, you really need people with solid experience in designing for performance. Sure, always go for the low hanging fruit which are the hotspots identified by profiling and improve those. But what often happens after that is that all the code is slow everywhere! So there are no big hotspots, profiling shows nothing egregious. Teams without high performance systems experience might conclude the code is about as good as it can get given the absence of hotspots. Seen this happen often. But there might be 10x or 100x improvements remaining.

bick_nyers 4y ago
In other words, what is the speed of light solution? The fastest possible solution that could be achieved?

tincholio 4y ago

At the same time, one should not automatically assume that "clean code" is actually readable. I've had to deal with architecture astronauts of the Uncle Bob school of "clean code", and their code was anything but readable.