3 ms·

Literally the thread I’m responding too is asking what the value of optimize then profile is. My experience is that you are likely hosed if you’ve gotten to thi

Literally the thread I’m responding too is asking what the value of optimize then profile is. My experience is that you are likely hosed if you’ve gotten to this point in the 80% case. That isn’t to say these techniques have no value, I’ve used many of them myself. But the question was, “how often does it happen that your optimizations are for a flat profile”. For me, the answer is “most of the time”.

joshuamorton 7y ago
And the broader context is when talking about micro-optimizations versus reasonable architectures. You shouldn't micro-optimize before profiling, because it likely won't matter. Bluntly, if you have a flat profile, none of the optimizations in this article are relevant anyway. You'll be able to pull out single digit percentage speedups, maybe. The optimize then profile argument isn't meant to be about architecture. Yes, you should build performant architecture. Yes, you should take time to plan a performant architecture before building[+]. But the question of profile than optimize is never (except in the strange way you're bringing it up) about doing macro-optimizations before you've written a line of code. It's almost always in the context of "don't just try to optimize what you think is slow, because you're almost always wrong". Big-O style speedups from architectural changes aren't micro-optimizations, they generally sit outside of that conversation entirely. As an aside, flat profiles are in practice exceedingly rare. Most (useful) programs do the same thing many times. Its very unusual to see a program that isn't, in essence, a loop. And the area inside the loop is going to be hot. The pareto principle applies to execution time too. [+]: Maybe startups who gotta ship it to survive as the exception.
- shereadsthenews 7y ago
  Kanev[1] disagrees. Flat profiles are the common case in actual practice. Edited to add: Since apparently you also work at Google, you should walk over to Svilen's desk and just ask him if profiles of production software are generally flat, or if they generally have hot spots. 1: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44271.pdf https://static.googleusercontent.com/media/research.google.c...
  - joshuamorton 7y ago
    You're citing a paper using, as an example, an already highly hand optimized and performance-guided optimized binary. That's what you get after you profile and optimize.
- kasey_junk 7y ago
  > You shouldn't micro-optimize before profiling, because it likely won't matter. Bluntly, if you have a flat profile, none of the optimizations in this article are relevant anyway. You'll be able to pull out single digit percentage speedups, maybe. Glad we agree. > The optimize then profile argument isn’t meant to be about architecture Glad we agree. If only all the people who tell me “correct before performant” agreed with us. In practice, in my experience, this is not the case. People use it in day to day conversations at the earliest parts of conversations about architecture all the time. If they didn’t I wouldn’t have nearly the problems I do with the statement. > As an aside, flat profiles are in practice exceedingly rare This seems to be the most controversial part of our disagreement. In my experience, that is flatly untrue. Especially when talking about systems where the performance does not meet the requirements. I can count on 1 hand the number of times I’ve seen systems go from “unacceptable” performance to “acceptable” via micro optimizations. I’ve never seen one go to “great”. I don’t know how to quantify this though, so I’m willing to leave this in the realm of my experience is different than yours. All that is to say, my experience says that systems that don’t treat performance as first class requirements don’t tend to meet their performance expectations. All of which is neither here nor there based on the article but is directly related to the question of ‘what do you do with a flat profile’?

joshuamorton 7y ago

And the broader context is when talking about micro-optimizations versus reasonable architectures. You shouldn't micro-optimize before profiling, because it likely won't matter. Bluntly, if you have a flat profile, none of the optimizations in this article are relevant anyway. You'll be able to pull out single digit percentage speedups, maybe. The optimize then profile argument isn't meant to be about architecture. Yes, you should build performant architecture. Yes, you should take time to plan a performant architecture before building[+]. But the question of profile than optimize is never (except in the strange way you're bringing it up) about doing macro-optimizations before you've written a line of code. It's almost always in the context of "don't just try to optimize what you think is slow, because you're almost always wrong". Big-O style speedups from architectural changes aren't micro-optimizations, they generally sit outside of that conversation entirely. As an aside, flat profiles are in practice exceedingly rare. Most (useful) programs do the same thing many times. Its very unusual to see a program that isn't, in essence, a loop. And the area inside the loop is going to be hot. The pareto principle applies to execution time too. [+]: Maybe startups who gotta ship it to survive as the exception.

shereadsthenews 7y ago
Kanev[1] disagrees. Flat profiles are the common case in actual practice. Edited to add: Since apparently you also work at Google, you should walk over to Svilen's desk and just ask him if profiles of production software are generally flat, or if they generally have hot spots. 1: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44271.pdf https://static.googleusercontent.com/media/research.google.c...
- joshuamorton 7y ago
  You're citing a paper using, as an example, an already highly hand optimized and performance-guided optimized binary. That's what you get after you profile and optimize.
kasey_junk 7y ago
> You shouldn't micro-optimize before profiling, because it likely won't matter. Bluntly, if you have a flat profile, none of the optimizations in this article are relevant anyway. You'll be able to pull out single digit percentage speedups, maybe. Glad we agree. > The optimize then profile argument isn’t meant to be about architecture Glad we agree. If only all the people who tell me “correct before performant” agreed with us. In practice, in my experience, this is not the case. People use it in day to day conversations at the earliest parts of conversations about architecture all the time. If they didn’t I wouldn’t have nearly the problems I do with the statement. > As an aside, flat profiles are in practice exceedingly rare This seems to be the most controversial part of our disagreement. In my experience, that is flatly untrue. Especially when talking about systems where the performance does not meet the requirements. I can count on 1 hand the number of times I’ve seen systems go from “unacceptable” performance to “acceptable” via micro optimizations. I’ve never seen one go to “great”. I don’t know how to quantify this though, so I’m willing to leave this in the realm of my experience is different than yours. All that is to say, my experience says that systems that don’t treat performance as first class requirements don’t tend to meet their performance expectations. All of which is neither here nor there based on the article but is directly related to the question of ‘what do you do with a flat profile’?