4 ms·

I don't think Casey would object to "optimizing for human comprehension". If you are willing to give up an order of magnitude of performance, you can probably j

I don't think Casey would object to "optimizing for human comprehension". If you are willing to give up an order of magnitude of performance, you can probably just do it without much care. I agree that "optimizing for human comprehension" is a worthy goal, but it is very hard to actually know what is easiest for humans to understand. I don't think that guidelines like "clean code" actually are particularly effective at making understandable code. My personal guideline is to prefer code which is "simple" for both humans and computers. > So by default, I follow Knuth's advice that premature optimization is the root of all evil. I write clean code but go into Casey mode when the numbers justify it. I hold the controversial opinion that Knuth's advice is not relevant to most modern programmers. Most modern programmers have never done the "optimization" that Knuth was referring to in 1974. Optimizing only the hotspots of your program is very relevant technique, but if you write terribly performant code everywhere, there won't be significant hotspots. Everything will be lukewarm.

taeric 4y ago
Knuth very much pushed for efficient code. One need only look at the code he writes, to see. Which is a large part of what makes that attribution amusing.
- inimino 4y ago
  Since this gets repeated so often, it is probably worth reproducing a little more of Knuth's quote. "There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of non-critical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. "Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code, but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail." The context for this statement, is of course his article [1] saying that goto statements should not be regarded as bad for religious reasons, but should be used appropriately. How common is the pragmatic approach to goto today, vs the "religious" response to goto? So this is the context in which it's appropriate to quote Knuth on this: if you are thinking about efficiency all the time, making intuitive guesses about where your programs will be slow, using measurement tools after the fact, and in danger of spending your optimization efforts on 100% of the code instead of 3%, and if you have a rational rather than emotional response to seeing goto statements in your codebase, then Knuth's quote is for you. What he was really arguing for is a practical, rational approach rather than religious, emotional responses. Which was also Dijkstra's point about goto. The whole article is well worth a read if you haven't already seen it. [1]: https://dl.acm.org/doi/pdf/10.1145/356635.356640 https://dl.acm.org/doi/pdf/10.1145/356635.356640
  - dceddia 4y ago
    Much like Chuck Norris references in movies from 2004, I get the sense that this advice is a product of its time. One of those “it makes more sense if you were there” things. (It even starts off acknowledging that it’s an article about current trends!) Not to say it doesn’t generally still hold true, but this part stuck out to me: > Programmers waste enormous amounts of time thinking about, or worrying about, the speed of non-critical parts of their programs I don’t think this is widely true today (outside e.g. game development anyway). If it were, we’d probably have lots of fast software that’s very hard to read! But computers were a lot slower then, compilers weren’t as advanced, and it would make a lot of sense if performance was top of mind for most programmers at the time.
    - bqmjjx0kac 4y ago
      > I don’t think this is widely true today (outside e.g. game development anyway). If it were, we’d probably have lots of fast software that’s very hard to read! I think the point was that programmers' intuition about performance is wrong. So we'd wind up with a lot of slow, buggy software — buggy because the code became hard to read when it was prematurely optimized.
  - taeric 4y ago
    I'd go further and focus on "small efficiencies" part of that. The assumption used to be high that you would stay cognizant of large efficiencies. In a very real sense, this is like a lot of the rhetoric around Big O analysis. From all that you ever hear online, you would think that Knuth's analysis was focused on only the Big O descriptions of algorithms. Reading the book, however, you find much more detailed takes of individual algorithms. Such that the aim was never to not be able to do the small, but realize that the comparison is dominated by the big. Such is it with efficiency. The assumption was that you did not pick purposely inefficient methods, at large. And that you can focus on the very small level details where you get the best return on them. Now, I think "clean code" is getting a bit thrown under here. Picking the specific examples that were easy to talk about in a pedagogical manner is not doing any favors to the general ideas from either side. And I consider myself mostly on the anti side of the "clean code" debate.
- rhdunn 4y ago
  That quote isn't saying not to optimize. It's saying not to optimize at the start. When you initially write your code, you won't know where the bottlenecks are. So first choose and write a sensible implementation using appropriate algorithms and data structures to complete the task. Then, when you have something working, measure its performance against real world data, not against synthetic benchmarks. You can use synthetic data that models the real world (e.g. a large number of user records in the system) to amplify the performance issues, but collecting the data from real world data will be better. With that performance profile, you can see exactly where the performance issues are. Those will then allow you to write more complex, or harder to read code, that improves the performance of that part of the codebase. This will then allow you to write code that actually improves performance, and not things you think will improve performance, such as: 1. The optimized inverse square root function in Doom (https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overview_of_the_code https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...) 2. Improving the git performance of the sha1 algorithm. (There was a discussion about rewriting the old code a while ago in assembly in the mailing lists that I can't find due to Google not understanding my search queries. In those discussions, IIRC Linus ended up creating a C implementation that compilers were able to compile into an efficient assembly version.)
  - taeric 4y ago
    The quote specifically calls out "small efficiency gains" as a premature optimization. Specific example is a goto statement that is not as "clean" as many other control structures, but still makes a big impact on a particular implementation of a routine in question. Implicit, I argue, is the idea that you were also not reaching to methods that were adding inefficiencies. That is, I think the argument is fine that you can and should try to write efficient code at a high level.
alpaca128 4y ago
> Most modern programmers have never done the "optimization" that Knuth was referring to in 1974 I don't know the exact boundaries of what Knuth meant, but there definitely is a kind of "premature optimization" which has nothing to do with squeezing out the last nanoseconds at a low level. When writing code it often feels like a good idea to make it reasonably efficient or flexible from the start, for example by using more specialized datastructures or creating more abstractions which makes the code using them a bit less straightforward. In my experience this usually backfires and only means I spent more time writing code before discarding the first iteration. At first glance this is a different kind of optimization but the outcome is the same and so I think Knuth's famous quote can still be applied. Other than that I agree, optimization at the degree e.g. Casey Muratori made in the video referenced in this post is only relevant in tiny hotspots if at all.

taeric 4y ago

Knuth very much pushed for efficient code. One need only look at the code he writes, to see. Which is a large part of what makes that attribution amusing.

inimino 4y ago
Since this gets repeated so often, it is probably worth reproducing a little more of Knuth's quote. "There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of non-critical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. "Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code, but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail." The context for this statement, is of course his article [1] saying that goto statements should not be regarded as bad for religious reasons, but should be used appropriately. How common is the pragmatic approach to goto today, vs the "religious" response to goto? So this is the context in which it's appropriate to quote Knuth on this: if you are thinking about efficiency all the time, making intuitive guesses about where your programs will be slow, using measurement tools after the fact, and in danger of spending your optimization efforts on 100% of the code instead of 3%, and if you have a rational rather than emotional response to seeing goto statements in your codebase, then Knuth's quote is for you. What he was really arguing for is a practical, rational approach rather than religious, emotional responses. Which was also Dijkstra's point about goto. The whole article is well worth a read if you haven't already seen it. [1]: https://dl.acm.org/doi/pdf/10.1145/356635.356640 https://dl.acm.org/doi/pdf/10.1145/356635.356640
- dceddia 4y ago
  Much like Chuck Norris references in movies from 2004, I get the sense that this advice is a product of its time. One of those “it makes more sense if you were there” things. (It even starts off acknowledging that it’s an article about current trends!) Not to say it doesn’t generally still hold true, but this part stuck out to me: > Programmers waste enormous amounts of time thinking about, or worrying about, the speed of non-critical parts of their programs I don’t think this is widely true today (outside e.g. game development anyway). If it were, we’d probably have lots of fast software that’s very hard to read! But computers were a lot slower then, compilers weren’t as advanced, and it would make a lot of sense if performance was top of mind for most programmers at the time.
  - bqmjjx0kac 4y ago
    > I don’t think this is widely true today (outside e.g. game development anyway). If it were, we’d probably have lots of fast software that’s very hard to read! I think the point was that programmers' intuition about performance is wrong. So we'd wind up with a lot of slow, buggy software — buggy because the code became hard to read when it was prematurely optimized.
- taeric 4y ago
  I'd go further and focus on "small efficiencies" part of that. The assumption used to be high that you would stay cognizant of large efficiencies. In a very real sense, this is like a lot of the rhetoric around Big O analysis. From all that you ever hear online, you would think that Knuth's analysis was focused on only the Big O descriptions of algorithms. Reading the book, however, you find much more detailed takes of individual algorithms. Such that the aim was never to not be able to do the small, but realize that the comparison is dominated by the big. Such is it with efficiency. The assumption was that you did not pick purposely inefficient methods, at large. And that you can focus on the very small level details where you get the best return on them. Now, I think "clean code" is getting a bit thrown under here. Picking the specific examples that were easy to talk about in a pedagogical manner is not doing any favors to the general ideas from either side. And I consider myself mostly on the anti side of the "clean code" debate.
rhdunn 4y ago
That quote isn't saying not to optimize. It's saying not to optimize at the start. When you initially write your code, you won't know where the bottlenecks are. So first choose and write a sensible implementation using appropriate algorithms and data structures to complete the task. Then, when you have something working, measure its performance against real world data, not against synthetic benchmarks. You can use synthetic data that models the real world (e.g. a large number of user records in the system) to amplify the performance issues, but collecting the data from real world data will be better. With that performance profile, you can see exactly where the performance issues are. Those will then allow you to write more complex, or harder to read code, that improves the performance of that part of the codebase. This will then allow you to write code that actually improves performance, and not things you think will improve performance, such as: 1. The optimized inverse square root function in Doom (https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overview_of_the_code https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...) 2. Improving the git performance of the sha1 algorithm. (There was a discussion about rewriting the old code a while ago in assembly in the mailing lists that I can't find due to Google not understanding my search queries. In those discussions, IIRC Linus ended up creating a C implementation that compilers were able to compile into an efficient assembly version.)
- taeric 4y ago
  The quote specifically calls out "small efficiency gains" as a premature optimization. Specific example is a goto statement that is not as "clean" as many other control structures, but still makes a big impact on a particular implementation of a routine in question. Implicit, I argue, is the idea that you were also not reaching to methods that were adding inefficiencies. That is, I think the argument is fine that you can and should try to write efficient code at a high level.

alpaca128 4y ago

> Most modern programmers have never done the "optimization" that Knuth was referring to in 1974 I don't know the exact boundaries of what Knuth meant, but there definitely is a kind of "premature optimization" which has nothing to do with squeezing out the last nanoseconds at a low level. When writing code it often feels like a good idea to make it reasonably efficient or flexible from the start, for example by using more specialized datastructures or creating more abstractions which makes the code using them a bit less straightforward. In my experience this usually backfires and only means I spent more time writing code before discarding the first iteration. At first glance this is a different kind of optimization but the outcome is the same and so I think Knuth's famous quote can still be applied. Other than that I agree, optimization at the degree e.g. Casey Muratori made in the video referenced in this post is only relevant in tiny hotspots if at all.