4 ms·

Worth pointing out that this is actually what Knuth meant by premature optimisation. Nothing to do with optimising too early in a project or some sense of YAGNI

by thom 3y ago

Worth pointing out that this is actually what Knuth meant by premature optimisation. Nothing to do with optimising too early in a project or some sense of YAGNI, but rather optimising before you know where the actual time is spent.

jraph 3y ago
I didn't know this, thanks for pointing it out. I think this has been a costly misinterpretation of "premature optimisation". This phrase has been taken as justification for writing inefficient code, understanding it as "performance doesn't matter [for what we do]". It almost always matters, and I believe it is possible to write both efficiently and in a readable way. Unfortunately, inefficient code is well established, in habits and even in (sometimes mandatory) coding styles. This inefficiency also often translates to hard to follow code because of too much abstraction. Maybe we could counter this misinterpretation of "premature optimisation" with "premature abstraction".
- ReleaseCandidat 3y ago
  > This phrase has been taken as justification for writing inefficient code, understanding it as "performance doesn't matter [for what we do]". There is no contradiction. Writing inefficient code is perfectly fine until you _know_ (this means you have measurements that show that) it is _too_ inefficient and has to be optimised. How do you measure the "optimisation" without "inefficient code" as baseline to begin with?
  - edflsafoiewq 3y ago
    In that case, how do you know you're measuring the "optimization" and not "how awful the original code was"?
  - jraph 3y ago
    > Writing inefficient code is perfectly fine until you _know_ (this means you have measurements that show that) it is _too_ inefficient and has to be optimised. What about inefficient code only if you _know_ that it does not matter? I'm not talking about optimizing. I'm talking about avoiding inefficiencies in the first place. This is what I'm talking about: we are in "inefficient by default" mode. That leads to requiring powerful hardware to display the most lambda websites.
  - vlovich123 3y ago
    Depends what you mean by inefficient. The other thing Knuth meant is that you should focus your efficiency energy first on overall design efficiency. Algorithmic and other efficiencies are irrelevant if your design is bad and a bad design cannot be fixed by looking at the profiler and fixing hotspots
norir 3y ago
I don't think the original paper by Knuth (https://dl.acm.org/doi/10.1145/356635.356640 https://dl.acm.org/doi/10.1145/356635.356640) supports this interpretation. In the paper, Knuth describes how as one develops as a programmer one learns how to write in a style that is generally reasonably efficient. In particular, you should take note of hot loops, which are often quite obvious, and then write in the most readable style outside of the hot loops without too much thought about efficiency. By accepting some inefficiency, but writing in a generally efficient style, you can move much more quickly than if you are constantly bogged down in the details trying to squeeze out every possible cpu cycle. What people tend to miss in my view is that he is advocating writing in an efficient style, which may not be obvious to an inexperienced developer. For example, writing a linear algebra routine in which you traverse column wise in an inner loop when the data is laid out row wise is going to be very slow and a simple change of algorithm or restructuring of the data can lead to an enormous speed up. This should be done immediately if discovered. But a skilled programmer will also be unlikely to traverse in the wrong order anyway because they already know the data layout and have thought about the order of traversal as part of their thinking process. In other words, the "optimization" of traversing in the fast way is something they would just do naturally and has no influence on the development time. Conversely, they may also suspect that with some clever bit-twiddling hacks, they may be able to squeeze another 5-10% performance out of the routine. Unless you know that the code is being distributed at scale, this is almost certainly a waste of time. It will be hard to write as well as brittle and error prone. It deflects your attention from more important problems. It is evil both because of the opportunity cost of chasing these small efficiencies as well as the increased complexity that they often introduce for minimal gain.
- HALtheWise 3y ago
  I'm curious how this interacts with language selection. In particular, it looks like most of the programming languages in widespread use in 1974 were all relatively efficient compiled and statically typed languages (Fortran, cobol, C) with the notable exception of Basic. Would Knuth have considered choosing a performant language as "optimization" or just as part of "writing in an efficient style as any good programmer should do"? The distinction is a lot more important today with the ~100x speed and power consumption differences between Ruby/Python/etc and C++/Java/Go/Rust/etc.

by thom 3y ago

jraph 3y ago

I didn't know this, thanks for pointing it out. I think this has been a costly misinterpretation of "premature optimisation". This phrase has been taken as justification for writing inefficient code, understanding it as "performance doesn't matter [for what we do]". It almost always matters, and I believe it is possible to write both efficiently and in a readable way. Unfortunately, inefficient code is well established, in habits and even in (sometimes mandatory) coding styles. This inefficiency also often translates to hard to follow code because of too much abstraction. Maybe we could counter this misinterpretation of "premature optimisation" with "premature abstraction".

ReleaseCandidat 3y ago
> This phrase has been taken as justification for writing inefficient code, understanding it as "performance doesn't matter [for what we do]". There is no contradiction. Writing inefficient code is perfectly fine until you _know_ (this means you have measurements that show that) it is _too_ inefficient and has to be optimised. How do you measure the "optimisation" without "inefficient code" as baseline to begin with?
- edflsafoiewq 3y ago
  In that case, how do you know you're measuring the "optimization" and not "how awful the original code was"?
- jraph 3y ago
  > Writing inefficient code is perfectly fine until you _know_ (this means you have measurements that show that) it is _too_ inefficient and has to be optimised. What about inefficient code only if you _know_ that it does not matter? I'm not talking about optimizing. I'm talking about avoiding inefficiencies in the first place. This is what I'm talking about: we are in "inefficient by default" mode. That leads to requiring powerful hardware to display the most lambda websites.
- vlovich123 3y ago
  Depends what you mean by inefficient. The other thing Knuth meant is that you should focus your efficiency energy first on overall design efficiency. Algorithmic and other efficiencies are irrelevant if your design is bad and a bad design cannot be fixed by looking at the profiler and fixing hotspots

norir 3y ago

I don't think the original paper by Knuth (https://dl.acm.org/doi/10.1145/356635.356640 https://dl.acm.org/doi/10.1145/356635.356640) supports this interpretation. In the paper, Knuth describes how as one develops as a programmer one learns how to write in a style that is generally reasonably efficient. In particular, you should take note of hot loops, which are often quite obvious, and then write in the most readable style outside of the hot loops without too much thought about efficiency. By accepting some inefficiency, but writing in a generally efficient style, you can move much more quickly than if you are constantly bogged down in the details trying to squeeze out every possible cpu cycle. What people tend to miss in my view is that he is advocating writing in an efficient style, which may not be obvious to an inexperienced developer. For example, writing a linear algebra routine in which you traverse column wise in an inner loop when the data is laid out row wise is going to be very slow and a simple change of algorithm or restructuring of the data can lead to an enormous speed up. This should be done immediately if discovered. But a skilled programmer will also be unlikely to traverse in the wrong order anyway because they already know the data layout and have thought about the order of traversal as part of their thinking process. In other words, the "optimization" of traversing in the fast way is something they would just do naturally and has no influence on the development time. Conversely, they may also suspect that with some clever bit-twiddling hacks, they may be able to squeeze another 5-10% performance out of the routine. Unless you know that the code is being distributed at scale, this is almost certainly a waste of time. It will be hard to write as well as brittle and error prone. It deflects your attention from more important problems. It is evil both because of the opportunity cost of chasing these small efficiencies as well as the increased complexity that they often introduce for minimal gain.

HALtheWise 3y ago
I'm curious how this interacts with language selection. In particular, it looks like most of the programming languages in widespread use in 1974 were all relatively efficient compiled and statically typed languages (Fortran, cobol, C) with the notable exception of Basic. Would Knuth have considered choosing a performant language as "optimization" or just as part of "writing in an efficient style as any good programmer should do"? The distinction is a lot more important today with the ~100x speed and power consumption differences between Ruby/Python/etc and C++/Java/Go/Rust/etc.