4 ms·

One thing I haven't seen in the comments yet is the simple observation that premature optimization is, by definition, a mistake -- otherwise it'd merely be "opt

One thing I haven't seen in the comments yet is the simple observation that premature optimization is, by definition, a mistake -- otherwise it'd merely be "optimization". The real question is, what makes it "premature"? I think Knuth (in his famous "... root of all evil" quote) was referring specifically to programming, and to spending time on optimizations prior to functional completion. The mantra, "First make it. Then make it work. Then make it better." [or similar] holds water. But it's pointless to debate in general terms what precisely constitutes a "premature" optimization, vs a merely timely one, or adherence to best practices, given the infinite combinations of circumstance and context relating to software development projects.

username90 7y ago
> "First make it. Then make it work. Then make it better." That only works when you do simple things, for harder things the "make it better" part usually requires a complete rewrite if you didn't properly think things through from the beginning. A better mantra would be "First make it, then make it again, then make the real version".
- yahwrong 7y ago
  No. Engineering is an iterative process. Even if you're 're-writing' everything, you're still not starting again from scratch, rather you're building off your previous approaches.
- valand 7y ago
  Few would admit that there is "the real version", though.
eloisant 7y ago
The real learning here is that "premature optimization" is a thing, you shouldn't optimize right away.
amelius 7y ago
> The mantra, "First make it. Then make it work. Then make it better." On HN we would say: first make it, then sell it, then make it work, and finally, make it better :)
- lioeters 7y ago
  I like it. Sometimes the selling comes first, before even making it. :)
gwd 7y ago
> The real question is, what makes it "premature"? When this principle has come up for me it's not so much about optimization, but about the cost you're paying to have that optimization: In particular, the cost in terms of code complexity. If you've already got generic code for a self-balancing red-black tree, and you can just plug your new data structure into it, and you're reasonably sure that your thing will get 10 nodes on occasion, then sure, use it, even if you don't know whether you'll ever get much larger than that. But if you don't have that code handy, then don't write a self-balancing red-black tree unless you have benchmarks to show that it will help. And don't use a self-balancing R-B tree if you don't know whether the number of nodes ever goes past 2. (Or, I mean, unless it's a personal project and you want to practice writing a self-balancing R-B tree; but at that point you're not optimizing, you're practicing / having fun, so the advice isn't applicable.) If code structure A and code structure B are both about equally comprehensible, and you think code structure A will be more effecient, go ahead and use it. But if structure A is extremely complicated, and will make the understanding and maintenance of the system more difficult and prone to error, then don't use it unless you know it's actually worth the cost.
- ncmncm 7y ago
  Word to the wise: if you are tempted to code a red-black tree, it probably means you are using the wrong language. In a more capable language, that code has already been written and put in a library, and been very thoroughly tested and optimized already, and is faster than you could afford to do just now. Less capable languages can't express such a library in usable form, or call into one. It is not an accident that Linux and BSD kernels have numerous hand-specialized red-black trees in them, now languishing for the attention that is needed to modernize them to perform reasonably on current hardware.
terminaljunkid 7y ago
Premature optimization is when micro optimizations triumph readability..
_the_inflator 7y ago
Unless we are talking about speed I think that "premature optimization" refers to readability issues with code. I work in web development and we never had to run a profiler even once to find performance issues. "Premature optimization" was in my experience always the discussion about readability. In the JavaScript world there is always this one guy who can do things in 3 lines of code instead of using 5 methods to understand the code in 5 month again. We called it "Perl coding" vs "standard code".
mpweiher 7y ago
> I think Knuth was referring specifically... We don't have to speculate, the paper is online[1] and he is very explicit about what he meant. The paper is, in fact, an ode to optimisation and the necessity of optimisation, including, very specifically, micro-optimisation. The "root of all evil..." part is a "yes, but". You actually have to leave out much of the actual sentence in order to strip it of its meaning: "We should forget about small efficiencies, say about 97% of the time, premature optimization is the root of all evil." If that wasn't clear, the very next sentence is as follows: "Yet we should not pass up our opportunities in that critical 3%" A little later in the paper: "The conventional wisdom [..] calls for ignoring efficiency in the small; but I believe this is simply an overreaction [..] In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering." [1] https://www.cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammingGoTo.pdf https://www.cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammi...
77544cec 7y ago
Optimising code without taking into effect the context it runs in is what makes it premature. For instance, the complexity of the calling code, or even some situation where it will never matter in practice (efficiently sorting UI elements, which must be few to be relevantly called UI). And there is early optimisation, when you know upfront the hill is steep because of the sheer amount of calculation involved in the very nature of the project (gamedev, big data, etc). And this can lead to low level optimizations (e.g McCarmack's fast inverse square root) or architectural measures (distributing code accross computers). Premature Architecture ?
greyfade 7y ago
> I think Knuth (in his famous "... root of all evil" quote) was referring specifically to programming, and to spending time on optimizations prior to functional completion. What he said is, quote, "... we should forget about the small efficiencies, say, about 97% of the time" and "we should not pass up our opportunities in that critical 3%." What he was talking about in the article[1] is the tendency of programmers to concern themselves with the efficiency of things like the modulo operator, when much larger efficiencies are far more important, such as the design of the algorithm or data structures. Premature optimization, as when deciding that this small efficiency is important enough to optimize, causes you to forget about the larger efficiencies that cause your program to be slow. [1] https://web.archive.org/web/20130731202547/http://pplab.snu.ac.kr/courses/adv_pl05/papers/p261-knuth.pdf https://web.archive.org/web/20130731202547/http://pplab.snu....

username90 7y ago

> "First make it. Then make it work. Then make it better." That only works when you do simple things, for harder things the "make it better" part usually requires a complete rewrite if you didn't properly think things through from the beginning. A better mantra would be "First make it, then make it again, then make the real version".

yahwrong 7y ago
No. Engineering is an iterative process. Even if you're 're-writing' everything, you're still not starting again from scratch, rather you're building off your previous approaches.
valand 7y ago
Few would admit that there is "the real version", though.

eloisant 7y ago

The real learning here is that "premature optimization" is a thing, you shouldn't optimize right away.

amelius 7y ago

> The mantra, "First make it. Then make it work. Then make it better." On HN we would say: first make it, then sell it, then make it work, and finally, make it better :)

lioeters 7y ago
I like it. Sometimes the selling comes first, before even making it. :)

gwd 7y ago

> The real question is, what makes it "premature"? When this principle has come up for me it's not so much about optimization, but about the cost you're paying to have that optimization: In particular, the cost in terms of code complexity. If you've already got generic code for a self-balancing red-black tree, and you can just plug your new data structure into it, and you're reasonably sure that your thing will get 10 nodes on occasion, then sure, use it, even if you don't know whether you'll ever get much larger than that. But if you don't have that code handy, then don't write a self-balancing red-black tree unless you have benchmarks to show that it will help. And don't use a self-balancing R-B tree if you don't know whether the number of nodes ever goes past 2. (Or, I mean, unless it's a personal project and you want to practice writing a self-balancing R-B tree; but at that point you're not optimizing, you're practicing / having fun, so the advice isn't applicable.) If code structure A and code structure B are both about equally comprehensible, and you think code structure A will be more effecient, go ahead and use it. But if structure A is extremely complicated, and will make the understanding and maintenance of the system more difficult and prone to error, then don't use it unless you know it's actually worth the cost.

ncmncm 7y ago
Word to the wise: if you are tempted to code a red-black tree, it probably means you are using the wrong language. In a more capable language, that code has already been written and put in a library, and been very thoroughly tested and optimized already, and is faster than you could afford to do just now. Less capable languages can't express such a library in usable form, or call into one. It is not an accident that Linux and BSD kernels have numerous hand-specialized red-black trees in them, now languishing for the attention that is needed to modernize them to perform reasonably on current hardware.

terminaljunkid 7y ago

Premature optimization is when micro optimizations triumph readability..

_the_inflator 7y ago

Unless we are talking about speed I think that "premature optimization" refers to readability issues with code. I work in web development and we never had to run a profiler even once to find performance issues. "Premature optimization" was in my experience always the discussion about readability. In the JavaScript world there is always this one guy who can do things in 3 lines of code instead of using 5 methods to understand the code in 5 month again. We called it "Perl coding" vs "standard code".

mpweiher 7y ago

> I think Knuth was referring specifically... We don't have to speculate, the paper is online[1] and he is very explicit about what he meant. The paper is, in fact, an ode to optimisation and the necessity of optimisation, including, very specifically, micro-optimisation. The "root of all evil..." part is a "yes, but". You actually have to leave out much of the actual sentence in order to strip it of its meaning: "We should forget about small efficiencies, say about 97% of the time, premature optimization is the root of all evil." If that wasn't clear, the very next sentence is as follows: "Yet we should not pass up our opportunities in that critical 3%" A little later in the paper: "The conventional wisdom [..] calls for ignoring efficiency in the small; but I believe this is simply an overreaction [..] In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering." [1] https://www.cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammingGoTo.pdf https://www.cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammi...

77544cec 7y ago

Optimising code without taking into effect the context it runs in is what makes it premature. For instance, the complexity of the calling code, or even some situation where it will never matter in practice (efficiently sorting UI elements, which must be few to be relevantly called UI). And there is early optimisation, when you know upfront the hill is steep because of the sheer amount of calculation involved in the very nature of the project (gamedev, big data, etc). And this can lead to low level optimizations (e.g McCarmack's fast inverse square root) or architectural measures (distributing code accross computers). Premature Architecture ?

greyfade 7y ago

> I think Knuth (in his famous "... root of all evil" quote) was referring specifically to programming, and to spending time on optimizations prior to functional completion. What he said is, quote, "... we should forget about the small efficiencies, say, about 97% of the time" and "we should not pass up our opportunities in that critical 3%." What he was talking about in the article[1] is the tendency of programmers to concern themselves with the efficiency of things like the modulo operator, when much larger efficiencies are far more important, such as the design of the algorithm or data structures. Premature optimization, as when deciding that this small efficiency is important enough to optimize, causes you to forget about the larger efficiencies that cause your program to be slow. [1] https://web.archive.org/web/20130731202547/http://pplab.snu.ac.kr/courses/adv_pl05/papers/p261-knuth.pdf https://web.archive.org/web/20130731202547/http://pplab.snu....