5 ms·

> Tony Hoare's famous maxim "Premature optimization is the root of all evil." Actually that was Donald Knuth - it's an urban legend that it's an urban legend t

> Tony Hoare's famous maxim "Premature optimization is the root of all evil." Actually that was Donald Knuth - it's an urban legend that it's an urban legend that it was originally Knuth. Hoare was quoting Knuth, but Knuth forgot he said it, and re-mis-attributed the quote to Hoare.

RoutinePlayer 6y ago
This reminds me of that Woody Allen joke about someone translating all the T.S. Eliot’s poems into English after some vandals had broken into the school library and translated them into French.
karmakaze 6y ago
And it is usually quoted out of its context. "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."
- hombre_fatal 6y ago
  I don't think that changes the meaning. Once that 3% matters to you and you've invested the work to measure that 3%, it's not premature anymore. That "premature" and "optimization" are undefined and left up for debate is what makes it trite.
  - YZF 6y ago
    It does change the meaning IMO. It means than 3% of the time you should be doing "premature" optimization. The point of most people referring to this quote is never try to optimize anything as you write it. First build your system(?), then measure it, then optimize it. Knuth's point is that this attitude is ok most of the time, but sometimes it's not ok. Another way of putting this is that most of the code, for most applications, isn't performance critical. But some code is. Sure, you can't always tell in advance, but sometimes you can. This is sort of the difference between terrible software that will always suck and well crafted software, no amount of measurement or after-the-fact optimization will turn that terrible software into well crafted software. The other aspect that I think is often missed is that these observations are often made at different scales. You can look at relatively short algorithm (let's say merge sort) and it may not be obvious which instructions are the ones that need to be optimized and what the bottlenecks will be, execution units, data access e.g. So you start with a reasonable but maybe naive implementation and then you optimize from there. That's a pretty solid idea. But taking that idea to a higher scale level, e.g. saying we're going to build this huge system with a billion lines of code and so we'll just throw something together and measure it isn't exactly the same thing, that's a pretty problematic idea. You need to be able to anticipate what the bottlenecks in your billion line system are going to be because finding that out after you've written a billion lines could be a big deal. [EDIT: and really this whole long story is why these sort of rules don't work. Because the people who know (have the experience/craftsmanship) don't need the rule and the people who don't know won't understand it. It's like reading a book about sword fighting and then trying to go into a sword fight... The reading can complement your training but can't be substitute...]
- eps 6y ago
  It's also often interpreted literally. Premature complex optimization is a bad idea, but simple (read, cheap to code) optimization for common bottleneck patterns is a perfectly reasonable thing to do.
  - FridgeSeal 6y ago
    It's also often (ab)used far too often to justify performing no optimisation at all.
    - userbinator 6y ago
      ...or even pessimisation.
      - NohatCoder 6y ago
        That is a great word.
  - danybittel 6y ago
    I like what Chandler Carruth said, "The death of a thousand cuts", on why is my code slow.
karmakaze 6y ago
Ha. So the truth is that Knuth did quote Hoare, not aware that he was quoting Knuth--indirectly Knuth was quoting himself.
bendbro 6y ago
I've always been uncomfortable with these kinds of ideas. The odds that the idea will be correctly applied is heavily tied to intelligence, culture, and situation. Instead of reducing the space of options you must consider, all it says is that you should "do it this way when you should and do it the other when you shouldn't." I suppose perhaps it is useful to highlight that the decision exists, but I would be surprised if anyone working in the space is unaware of the existence of the decision. The scientific method has a similar problem. A scientist should form their hypothesis before gathering data to evaluate the hypothesis. If a scientist fails to do this, and starts engaging in p-hacking or data dredging, the quality of their research greatly declines. But proving that a hypothesis was obtained before data was collected is not usually provable when just looking at the publication itself. And further, there are ways that data dredging can unintentionally sneak into the scientific process, especially around the phase before hypothesis- observation. This kind of idea has large technical impact, but doesn't have a solid technical reason. It's proof is closer to aesthetics than reason. And much like other aesthetic beliefs, a population believes it based on no deeper reasoning. Only exclusion or indoctrination can ensure the population's view, and only illogical rhetoric will change it.

RoutinePlayer 6y ago

This reminds me of that Woody Allen joke about someone translating all the T.S. Eliot’s poems into English after some vandals had broken into the school library and translated them into French.

karmakaze 6y ago

And it is usually quoted out of its context. "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

hombre_fatal 6y ago
I don't think that changes the meaning. Once that 3% matters to you and you've invested the work to measure that 3%, it's not premature anymore. That "premature" and "optimization" are undefined and left up for debate is what makes it trite.
- YZF 6y ago
  It does change the meaning IMO. It means than 3% of the time you should be doing "premature" optimization. The point of most people referring to this quote is never try to optimize anything as you write it. First build your system(?), then measure it, then optimize it. Knuth's point is that this attitude is ok most of the time, but sometimes it's not ok. Another way of putting this is that most of the code, for most applications, isn't performance critical. But some code is. Sure, you can't always tell in advance, but sometimes you can. This is sort of the difference between terrible software that will always suck and well crafted software, no amount of measurement or after-the-fact optimization will turn that terrible software into well crafted software. The other aspect that I think is often missed is that these observations are often made at different scales. You can look at relatively short algorithm (let's say merge sort) and it may not be obvious which instructions are the ones that need to be optimized and what the bottlenecks will be, execution units, data access e.g. So you start with a reasonable but maybe naive implementation and then you optimize from there. That's a pretty solid idea. But taking that idea to a higher scale level, e.g. saying we're going to build this huge system with a billion lines of code and so we'll just throw something together and measure it isn't exactly the same thing, that's a pretty problematic idea. You need to be able to anticipate what the bottlenecks in your billion line system are going to be because finding that out after you've written a billion lines could be a big deal. [EDIT: and really this whole long story is why these sort of rules don't work. Because the people who know (have the experience/craftsmanship) don't need the rule and the people who don't know won't understand it. It's like reading a book about sword fighting and then trying to go into a sword fight... The reading can complement your training but can't be substitute...]
eps 6y ago
It's also often interpreted literally. Premature complex optimization is a bad idea, but simple (read, cheap to code) optimization for common bottleneck patterns is a perfectly reasonable thing to do.
- FridgeSeal 6y ago
  It's also often (ab)used far too often to justify performing no optimisation at all.
  - userbinator 6y ago
    ...or even pessimisation.
    - NohatCoder 6y ago
      That is a great word.
- danybittel 6y ago
  I like what Chandler Carruth said, "The death of a thousand cuts", on why is my code slow.

karmakaze 6y ago

Ha. So the truth is that Knuth did quote Hoare, not aware that he was quoting Knuth--indirectly Knuth was quoting himself.

bendbro 6y ago

I've always been uncomfortable with these kinds of ideas. The odds that the idea will be correctly applied is heavily tied to intelligence, culture, and situation. Instead of reducing the space of options you must consider, all it says is that you should "do it this way when you should and do it the other when you shouldn't." I suppose perhaps it is useful to highlight that the decision exists, but I would be surprised if anyone working in the space is unaware of the existence of the decision. The scientific method has a similar problem. A scientist should form their hypothesis before gathering data to evaluate the hypothesis. If a scientist fails to do this, and starts engaging in p-hacking or data dredging, the quality of their research greatly declines. But proving that a hypothesis was obtained before data was collected is not usually provable when just looking at the publication itself. And further, there are ways that data dredging can unintentionally sneak into the scientific process, especially around the phase before hypothesis- observation. This kind of idea has large technical impact, but doesn't have a solid technical reason. It's proof is closer to aesthetics than reason. And much like other aesthetic beliefs, a population believes it based on no deeper reasoning. Only exclusion or indoctrination can ensure the population's view, and only illogical rhetoric will change it.