3 ms·

This is known as the hotspot or bottleneck model of optimization. It has its flaws: https://lemire.me/blog/2023/04/27/hotspot-performance-engineering-fails/ ht

This is known as the hotspot or bottleneck model of optimization. It has its flaws: https://lemire.me/blog/2023/04/27/hotspot-performance-engineering-fails/ https://lemire.me/blog/2023/04/27/hotspot-performance-engine...

ggm 3y ago
Nice article. I didn't see it say "don't do it" I saw it say "try not to have to"
IshKebab 3y ago
That article is 100% right, but gmm said "do another 20" - that's not really what hotspot people expect to do. They think there's one or two hotspots that they need to optimise and then they'll get a 10x speedup. You can definitely do 20-100 small optimisations and get a 2-3x speedup. The best example I've seen of that is Nicholas Nethercote's work on speeding up the Rust compiler. A ton of tiny speed improvements that add up to 2x or 3x speed improvements overall. I've done similar work on optimising a compiler. You're never going to get 10x without architectural changes but you can get 3x with a lot of work. Of course the "we'll just optimise the hotspots" people have written their code so badly they need at least 10x.
hinkley 3y ago
I compare this to man-to-man versus zone defense. Zone defense, as any parents with three+ children will tell you, is the only way to stay sane. By looking at a module or concern as a whole, you get to do small-to-medium architectural improvements instead of none-to-small improvements. You get a broader picture, and can pull in more small areas of improvement in the process. The fastest code is code that doesn't run, and 'tall tent-pole' optimizations miss at least half of those opportunities. In some cases they just push them around, like broccoli on a child's plate. Many teams get to the end of this road and then have absolutely no idea what to do next. I find it infuriating for a bunch of my 'betters' to listen to someone on the business side say, "Our customers are complaining about our speed/leaving for a competitor with a faster app" and then pull up a rectangular flame chart to 'prove' "We've done all we can". You have only done the easy parts, probably not even the best parts. You might have actually broken the best parts. These apps are 'death by a thousand cuts'. No one part of the app is particularly slow, it's just uniformly mediocre coding and an architecture that squeaks by with 'adequate' (to them, but not the business) attention to capacity planning. Either a force of nature shows up and fixes it, or the company fades away. The only boss I ever followed to a new job had two sayings that I stole. 1) "If you want to complete a meeting in an hour you need to complete half the meeting in half an hour" and the other was about scaling: The first order of magnitude is simple. The second is difficult. The third requires talent (sometimes tacking on "which not all teams have.") You can interpret that a number of ways, but if your founding team isn't paying homage to the problem of scaling up to 1000 times the users you have at the beginning, you probably don't have the talent to get there (you didn't buy it, and you didn't build it along the way). I don't know what percentage of companies that claim "we did all the right things and still failed" have this problem, but I sincerely suspect it's at least 10%, which is a very big failure mode to ignore.

ggm 3y ago

Nice article. I didn't see it say "don't do it" I saw it say "try not to have to"

IshKebab 3y ago

That article is 100% right, but gmm said "do another 20" - that's not really what hotspot people expect to do. They think there's one or two hotspots that they need to optimise and then they'll get a 10x speedup. You can definitely do 20-100 small optimisations and get a 2-3x speedup. The best example I've seen of that is Nicholas Nethercote's work on speeding up the Rust compiler. A ton of tiny speed improvements that add up to 2x or 3x speed improvements overall. I've done similar work on optimising a compiler. You're never going to get 10x without architectural changes but you can get 3x with a lot of work. Of course the "we'll just optimise the hotspots" people have written their code so badly they need at least 10x.

hinkley 3y ago

I compare this to man-to-man versus zone defense. Zone defense, as any parents with three+ children will tell you, is the only way to stay sane. By looking at a module or concern as a whole, you get to do small-to-medium architectural improvements instead of none-to-small improvements. You get a broader picture, and can pull in more small areas of improvement in the process. The fastest code is code that doesn't run, and 'tall tent-pole' optimizations miss at least half of those opportunities. In some cases they just push them around, like broccoli on a child's plate. Many teams get to the end of this road and then have absolutely no idea what to do next. I find it infuriating for a bunch of my 'betters' to listen to someone on the business side say, "Our customers are complaining about our speed/leaving for a competitor with a faster app" and then pull up a rectangular flame chart to 'prove' "We've done all we can". You have only done the easy parts, probably not even the best parts. You might have actually broken the best parts. These apps are 'death by a thousand cuts'. No one part of the app is particularly slow, it's just uniformly mediocre coding and an architecture that squeaks by with 'adequate' (to them, but not the business) attention to capacity planning. Either a force of nature shows up and fixes it, or the company fades away. The only boss I ever followed to a new job had two sayings that I stole. 1) "If you want to complete a meeting in an hour you need to complete half the meeting in half an hour" and the other was about scaling: The first order of magnitude is simple. The second is difficult. The third requires talent (sometimes tacking on "which not all teams have.") You can interpret that a number of ways, but if your founding team isn't paying homage to the problem of scaling up to 1000 times the users you have at the beginning, you probably don't have the talent to get there (you didn't buy it, and you didn't build it along the way). I don't know what percentage of companies that claim "we did all the right things and still failed" have this problem, but I sincerely suspect it's at least 10%, which is a very big failure mode to ignore.