3 ms·

> Yet we should not pass up our opportunities in that critical 3 % The funny thing is, we forgot how to write programs that spend most of their time in 3% of t

> Yet we should not pass up our opportunities in that critical 3 % The funny thing is, we forgot how to write programs that spend most of their time in 3% of the code. Profile a modern application and you see 50 level deep stacks and tiny slices of 0.3% of CPU time spent here and there. And yet these slices amount to 100%.

aw1621107 1y ago
I think the C# dev team had an interesting way to describe that kind of profile [0]: > In every .NET release, there are a multitude of welcome PRs that make small improvements. These changes on their own typically don’t “move the needle,” don’t on their own make very measurable end-to-end changes. However, an allocation removed here, an unnecessary bounds check removed there, it all adds up. Constantly working to remove this “peanut butter,” as we often refer to it (a thin smearing of overhead across everything), helps improve the performance of the platform in the aggregate. [0]: https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/#peanut-butter https://devblogs.microsoft.com/dotnet/performance-improvemen...
- BanterTrouble 1y ago
  They are describing "marginal gains theory". Lots of small improvements to something add up to a larger more tangible improvement.
- sfn42 1y ago
  This is a bit different because C# is designed and implemented by industry experts - far above the average code monkey in terms of skill, understanding and experience, with added help from community experts, and the software in question is used by millions of systems all over the world. Practically every part of C# is a hot path because it runs on probably billions of devices every minute of every day. This code is worth micro-optimizing. A 1% improvement in a popular C# library probably saves megawatts of electricity (not to mention the time saves) on a global scale. Most code is not like that at all. Most code is not worth putting this kind of effort into. Most code is fine being 10% or even 500% slower than it technically could be because computers are lightning fast and can crunch numbers at an incredible speed. Even if the computer is doing 5 times as much work as it needs to it'll still be instant for most use cases. The problem for these kinds of applications arises when it's not doing 5 times as much work as it needs to but 5000 times or 5 million times. Or when the data is just really large and even the optimal solution would take a noticeable amount of time.
IshKebab 1y ago
Most programs inherently don't have a small bit of code they spend most of their time in. There are exceptions like scientific computing, AI, etc. But most programs have their runtime distributed through lots of code so the idea that you can ignore performance except to profile and optimise some hotspots is nonsense. Most programs - after dealing with bugs & low hanging fruit - don't have hotspots.
- globular-toast 1y ago
  I'm not sure about the "most programs" bit. I think you're forgetting whole swathes of programs like system utilities, databases, tools etc. But, in any case, this is why we build large programs using modular architectures. You don't profile the Linux kernel hoping to find a hotspot in some obscure driver somewhere, you profile the driver directly. Same for filesystems, networking etc. Similarly we build libraries for things like arithmetic and SAT solvers etc. that will be used everywhere and optimise them directly. Your comment reads like you're disagreeing with your parent, but in fact you are reinforcing the point. We've forgotten how to build programs like this and end up with big balls of mud that can't be profiled properly.
- sfn42 1y ago
  I completely disagree. If the program doesn't have "hotspots" then it's probably pretty fast and it's following Knuth's advice. But in my experience most applications are completely littered with "hotspots" - endpoints taking 5+ seconds to respond, reports taking minutes or hours to generate, stuff like that. I've never worked on an application where I didn't quickly find multiple easily avoidable hotspots. The average developer, at least in my area of the world, just really doesn't understand how to write performant code at all. They write algorithms with terrible time and/or space complexity, they write inefficient database queries, design their database schemas in ways that don't even allow efficient queries etc. Then they add caching to compensate for their crappy system design and stuff like that, bloating the system with lots of unnecessary code. They store the data naively then spend significant processing time preparing the data for delivery rather than just storing the data in an ideal format in the first place. Lots of stuff like that.
another_twist 1y ago
Pick your top 3 and optimize if you really have to. But if the whole chain is 0.3% here and there, theres very little room for large optimization wins without some redesign.
trgn 1y ago
Yep. There are no silver bullets, and the only way through is shaving down the inefficiencies one by one. Really hard to advocate for in an org.

aw1621107 1y ago

I think the C# dev team had an interesting way to describe that kind of profile [0]: > In every .NET release, there are a multitude of welcome PRs that make small improvements. These changes on their own typically don’t “move the needle,” don’t on their own make very measurable end-to-end changes. However, an allocation removed here, an unnecessary bounds check removed there, it all adds up. Constantly working to remove this “peanut butter,” as we often refer to it (a thin smearing of overhead across everything), helps improve the performance of the platform in the aggregate. [0]: https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/#peanut-butter https://devblogs.microsoft.com/dotnet/performance-improvemen...

BanterTrouble 1y ago
They are describing "marginal gains theory". Lots of small improvements to something add up to a larger more tangible improvement.
sfn42 1y ago
This is a bit different because C# is designed and implemented by industry experts - far above the average code monkey in terms of skill, understanding and experience, with added help from community experts, and the software in question is used by millions of systems all over the world. Practically every part of C# is a hot path because it runs on probably billions of devices every minute of every day. This code is worth micro-optimizing. A 1% improvement in a popular C# library probably saves megawatts of electricity (not to mention the time saves) on a global scale. Most code is not like that at all. Most code is not worth putting this kind of effort into. Most code is fine being 10% or even 500% slower than it technically could be because computers are lightning fast and can crunch numbers at an incredible speed. Even if the computer is doing 5 times as much work as it needs to it'll still be instant for most use cases. The problem for these kinds of applications arises when it's not doing 5 times as much work as it needs to but 5000 times or 5 million times. Or when the data is just really large and even the optimal solution would take a noticeable amount of time.

IshKebab 1y ago

Most programs inherently don't have a small bit of code they spend most of their time in. There are exceptions like scientific computing, AI, etc. But most programs have their runtime distributed through lots of code so the idea that you can ignore performance except to profile and optimise some hotspots is nonsense. Most programs - after dealing with bugs & low hanging fruit - don't have hotspots.

globular-toast 1y ago
I'm not sure about the "most programs" bit. I think you're forgetting whole swathes of programs like system utilities, databases, tools etc. But, in any case, this is why we build large programs using modular architectures. You don't profile the Linux kernel hoping to find a hotspot in some obscure driver somewhere, you profile the driver directly. Same for filesystems, networking etc. Similarly we build libraries for things like arithmetic and SAT solvers etc. that will be used everywhere and optimise them directly. Your comment reads like you're disagreeing with your parent, but in fact you are reinforcing the point. We've forgotten how to build programs like this and end up with big balls of mud that can't be profiled properly.
sfn42 1y ago
I completely disagree. If the program doesn't have "hotspots" then it's probably pretty fast and it's following Knuth's advice. But in my experience most applications are completely littered with "hotspots" - endpoints taking 5+ seconds to respond, reports taking minutes or hours to generate, stuff like that. I've never worked on an application where I didn't quickly find multiple easily avoidable hotspots. The average developer, at least in my area of the world, just really doesn't understand how to write performant code at all. They write algorithms with terrible time and/or space complexity, they write inefficient database queries, design their database schemas in ways that don't even allow efficient queries etc. Then they add caching to compensate for their crappy system design and stuff like that, bloating the system with lots of unnecessary code. They store the data naively then spend significant processing time preparing the data for delivery rather than just storing the data in an ideal format in the first place. Lots of stuff like that.

another_twist 1y ago

Pick your top 3 and optimize if you really have to. But if the whole chain is 0.3% here and there, theres very little room for large optimization wins without some redesign.

trgn 1y ago

Yep. There are no silver bullets, and the only way through is shaving down the inefficiencies one by one. Really hard to advocate for in an org.