3 ms·

You can optimize even further by creating a custom chip to compute the area of shapes in the order of billions per second. But what's the point? Where is the va

You can optimize even further by creating a custom chip to compute the area of shapes in the order of billions per second. But what's the point? Where is the value? Can't say it better than Knuth: We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. Clean code has never been about performance, it's about the other people that will read your code – future you included. Performance of people is more valuable for any product than code performance[1]. Only when the performance becomes a bottleneck, or you want to optimize energy efficiency then sure, don't pass that 3% opportunity. [1] I would argue that it still holds for products that need high(er) performance code like video games, embedded systems, particle physics, ... These products just happen to hit bottlenecks way faster and some have hard cutoffs (eg. 60 fps for a game). Still, not everything needs to be optimized to the extreme: the algorithm to sort an in-game inventory does not need to handle 4B+ items.

IshKebab 4y ago
There's a big difference between "premature optimisation" and thinking about performance. You shouldn't take that quote to mean you can entirely ignore performance 97% of the time.
gnuvince 4y ago
Another quote from Knuth, from that same paper: > The improvement in speed from Example 2 to Example 2a is only about 12%, and many people would pronounce that insignificant. The conventional wisdom shared by many of today's software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by penny-wise-and-pound-foolish programmers, who can't debug or maintain their "optimized" programs. In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering. Of course I wouldn't bother making such optimizations on a one-shot job, but when it's a question of preparing quality programs, I don't want to restrict myself to tools that deny me such efficiencies. That paper was published in 1974 and yet it captures the mindset of many a programmer in 2023 perfectly. The part I like about this paragraph is the "easily obtained" sentence; I saw a comment from someone mentioning that they made a JavaScript program 10 times faster by replacing the common functional programming combinators (map, filter, reduce, etc.) with for loops. I think most of us would say such a change is easily obtained, does not make the code impossible to debug or maintain, and gives such a massive improvement that it should be a no brainer to reach for it.
- cranium 4y ago
  Good point, and also why the topic of performance is so prone to heated discussions. To paraphrase Knuth "97% of the time, don't optimize except if it's an easy 12% improvement". I don't think there exists a good heuristic that works each time or for all languages. The right decision is left (pun intended) to the programmer that has to live with it. For me the ideal decision took into account the performance tradeoffs of each level of abstraction and chose the "appropriate" one. Obviously, the "appropriate one" depends on an uncountable number of variables including future scaling issues, probability of refactoring/irrelevance, cost/reward of spending the time to optimize the function, ...
_dain_ 4y ago
A 20x performance improvement is not a "small" efficiency.
- cranium 4y ago
  A 10000x performance gain could still be insignificant, see Amdahl's law. That's why you need to understand bottlenecks in your system before going around and start optimizing things, as it could be pointless or even counter productive.
  - _dain_ 4y ago
    No, I don't actually agree with this at all. Sometimes -- a lot of the time actually -- you can have a pretty good idea ahead of time what will be fast and what will be slow. If you design the system before thinking about this, you can paint yourself into a corner and lock-in a fundamentally slow architecture, just like you can lock yourself into a fundamentally un-maintainable architecture. Like, when you're choosing a big data structure that is going to have lots of by-value lookups, you don't implement it as an array with O(n) lookup first and only move to a hashtable after benchmarking it. That would be absurd. You just use a hashtable with O(1) lookup right at the start. Because in the overwhelming majority of cases that's the right thing to do and it doesn't need any justification. And the reason you can do that is because you're an engineer and you know a damned thing about the domain you're working in. Structural engineers don't build a skyscraper out of paper mache first, and then rebuild it in concrete when it collapses in a stiff breeze. They just build it out of concrete the first time around. What so many people thoughtlessly call "premature optimization" isn't "premature" at all, it's just "knowing about the problem and knowing how the computer works". When you deliberately ignore this, what you're actually doing is "premature pessimization". Coding as if you don't know the difference between cache and RAM in 2023 is like coding as if you don't know the difference between RAM and disk in 1973. It's negligence. You know better! And look at what the Clean Code people want you to do instead. All these rules are geared towards future extensibility. Is that not a form of premature optimization? But optimizing for extensibility, not speed. And you end up with codebases littered with abstract interfaces that only have (and will only ever have) one implementation. But you pay for that premature abstraction in both cognitive load and CPU load. You ain't gonna need it!

IshKebab 4y ago

There's a big difference between "premature optimisation" and thinking about performance. You shouldn't take that quote to mean you can entirely ignore performance 97% of the time.

gnuvince 4y ago

Another quote from Knuth, from that same paper: > The improvement in speed from Example 2 to Example 2a is only about 12%, and many people would pronounce that insignificant. The conventional wisdom shared by many of today's software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by penny-wise-and-pound-foolish programmers, who can't debug or maintain their "optimized" programs. In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering. Of course I wouldn't bother making such optimizations on a one-shot job, but when it's a question of preparing quality programs, I don't want to restrict myself to tools that deny me such efficiencies. That paper was published in 1974 and yet it captures the mindset of many a programmer in 2023 perfectly. The part I like about this paragraph is the "easily obtained" sentence; I saw a comment from someone mentioning that they made a JavaScript program 10 times faster by replacing the common functional programming combinators (map, filter, reduce, etc.) with for loops. I think most of us would say such a change is easily obtained, does not make the code impossible to debug or maintain, and gives such a massive improvement that it should be a no brainer to reach for it.

cranium 4y ago
Good point, and also why the topic of performance is so prone to heated discussions. To paraphrase Knuth "97% of the time, don't optimize except if it's an easy 12% improvement". I don't think there exists a good heuristic that works each time or for all languages. The right decision is left (pun intended) to the programmer that has to live with it. For me the ideal decision took into account the performance tradeoffs of each level of abstraction and chose the "appropriate" one. Obviously, the "appropriate one" depends on an uncountable number of variables including future scaling issues, probability of refactoring/irrelevance, cost/reward of spending the time to optimize the function, ...

_dain_ 4y ago

A 20x performance improvement is not a "small" efficiency.

cranium 4y ago
A 10000x performance gain could still be insignificant, see Amdahl's law. That's why you need to understand bottlenecks in your system before going around and start optimizing things, as it could be pointless or even counter productive.
- _dain_ 4y ago
  No, I don't actually agree with this at all. Sometimes -- a lot of the time actually -- you can have a pretty good idea ahead of time what will be fast and what will be slow. If you design the system before thinking about this, you can paint yourself into a corner and lock-in a fundamentally slow architecture, just like you can lock yourself into a fundamentally un-maintainable architecture. Like, when you're choosing a big data structure that is going to have lots of by-value lookups, you don't implement it as an array with O(n) lookup first and only move to a hashtable after benchmarking it. That would be absurd. You just use a hashtable with O(1) lookup right at the start. Because in the overwhelming majority of cases that's the right thing to do and it doesn't need any justification. And the reason you can do that is because you're an engineer and you know a damned thing about the domain you're working in. Structural engineers don't build a skyscraper out of paper mache first, and then rebuild it in concrete when it collapses in a stiff breeze. They just build it out of concrete the first time around. What so many people thoughtlessly call "premature optimization" isn't "premature" at all, it's just "knowing about the problem and knowing how the computer works". When you deliberately ignore this, what you're actually doing is "premature pessimization". Coding as if you don't know the difference between cache and RAM in 2023 is like coding as if you don't know the difference between RAM and disk in 1973. It's negligence. You know better! And look at what the Clean Code people want you to do instead. All these rules are geared towards future extensibility. Is that not a form of premature optimization? But optimizing for extensibility, not speed. And you end up with codebases littered with abstract interfaces that only have (and will only ever have) one implementation. But you pay for that premature abstraction in both cognitive load and CPU load. You ain't gonna need it!