8 ms·

Fast software is a discipline, not a purpose

56 points by frostmatthew 9y ago

AstralStorm 9y ago
There is indeed a point where being fit and clean turns into an obsession. There is such a thing as premature optimization. Commonly the mistakes costing performance are caused in part by bad design though, typically arrived at by either not caring or not fixing a quick and dirty prototype with something reasonable. Or "brute force" bug fixing in concurrent code. If I got paid for every unnecessary or replaceable synchronized statement in Java, I'd be rich. Or a fat data copy made using a queue.
- tluyben2 9y ago
  > There is indeed a point where being fit and clean turns into an obsession. There is such a thing as premature optimization. There is also something as no optimization (at any point), and I see far more often in the wild. I think more devs should run their app on a cheap Android phone or cheap consumer entry laptop to see what is happening when it's not running on 32 gb i7 octa cores. I recently got some entry level machine from a friend to play around with; it has 4gb and flash drive; it is eerie how slow the thing is; this is what is sold in walmart etc at the entry price level, so fair to say, if you are B2C software dev, this is your audience. For fun, I tried to do some development on it; Atom/Vscode started both snappy, but when devving on it a bit, they became rapidly unusable (cpu 100% always and typing appearing seconds after you typed it); emacs was ok; vim did well. You cannot run Electron on that and yet, that's what many people deliver. And my suspicion is that most don't optimize when using that (if you are fast software minded, you wouldn't use it in the first place, not because it's inherently bad, but because there are too many moving parts you cannot influence). Please try it on these machines to know what you are doing to your users :)
jackjeff 9y ago
“Avoid multiple passes over the data when one would do.” Totally disagree. Unless performance is an issue (like I’m not dealing with a trivial number of elements), I would rather use functional programming approaches to sort data into shape (think map/filter/reduce). These approaches typically result in passing over the data multiple times, and often performing multiple copies, but it makes for readable and less error prone code. Doing the performant thing consists of writing a for loop (or multiple of them unless you unroll) and automatically result with a large custom body of code performing multiple transformations on the data. It’s just wasted mental effort when you write it and every time when you happen to read it again. Most collections/array I process have trivial number of elements (<100) and you get virtually no benefit from writing optimized code. I have also written a lot of performant code, and the lesson is always benchmark benchmark benchmark. The results do not always fit the mental model about what you think should be the fastest. In particuliar avoiding cache misses is far more important than you might think.
- AstralStorm 9y ago
  Readable and less error prone? As opposed as to proven to be correct for example? Recursive functional code is a pain to prove to be right and not blow up stack. Multiple passes (when interspersed with other accesses) can blow up cache. Even calling through a lambda is a cost compilers cannot easily optimize away unless you help them. (Even in C++.) You do get the benefit of optimizing all the "trivial data size" code when it is all the functions being written in such silly way. Difference of 10% in one function as opposed to across whole application. Benchmarking is often even harder to do right than writing good tests. (In fact is tied to it.) Instead of benchmarking, be a real computer scientist and prove bounds and memory allocations.
  - tigershark 9y ago
    How is it difficult to prove if a recursive call is tail-recursive or not? If it's tail-recursive and your language supports tail-call optimisation then you proved that there will be no stack blow-up.
    - AstralStorm 9y ago
      For simple calls, not. For something more involved? (Dependent functions, corecursion, partially stateful code.) Very which is why compilers fail to optimize it in general.
  - lmm 9y ago
    > Recursive functional code is a pain to prove to be right and not blow up stack. Much easier to prove recursion correct than to prove a loop always terminates, since the recursion makes the way the state changes between steps much more explicit. If you're really worried about this you can use Idris and require all your functions to be total.
- elcritch 9y ago
  Plus some functional languages can combine those steps at compile, giving you the best of both worlds. Rust for example elides away many lambda's in map operations. I don't have the link handy, but a post a few months back showed Rust and Haskell functional patterns could almost match their hard coded imperative style implementations and C/C++ cousins.
blux 9y ago
Regarding the point "Don’t use floating-point operations when integers will do." I agree that using integer arithmetic can result in cleaner, easier to reason about code when applicable. It then follows that because of these traits the software will faster. But integer arithmetic is not necessarily faster than floating point arithmetic in general; see for example: https://youtu.be/3K2LmnaLLF8?t=31m10s https://youtu.be/3K2LmnaLLF8?t=31m10s.
- AstralStorm 9y ago
  Only if you either have to pay the extra cost of handling overflows, underflows, saturation or normalization. (First three involve conditionals which may be mispredicted, last a division or multiplication.) Likewise correct rounding. (Multiple math ops and sometimes a conditional too.) Or when the platform has no integer vector math for your type of choice. Which is still somewhat common.
- gopalv 9y ago
  > But integer arithmetic is not necessarily faster than floating point arithmetic in general; GPUs was the first place I feel like floats got a much better pathway than any other data type - graphical pixel manipulations also can get away with much higher arithmetic errors, because it's all going to get rounded down to a pixel eventually. On the other hand, the only place where I've really worked heavily with fixed point was on a graphical interface toolkit (MIDP on ARM I was working with would've had to use softfp for floats, so fixed point was hugely superior).
  - AstralStorm 9y ago
    Come on, all reasonable ARM have NEON and VFPv3 (including cortex M) about the only place I saw something that didn't was an old internet router. Hardware completely unsuited to maths. Softfp does not preclude the use of floating point math at all. It means you pay an extra cost when passing floats as arguments. (Which means you should rather use a hardfp math library if possible.)
    - gopalv 9y ago
twic 9y ago
> But then, I dress cleanly every single day even if I stay at home. And you should too. Nope. It's good to know how to produce fast software, but also good to know when to do so. Lemire sounds like he has some psychological issues to work out.
michaelg7x 9y ago
Code bloat could easily be sorted out by forcing people to read the generated assembly of their creations.
- reikonomusha 9y ago
  This is easy and convenient in Common Lisp with the DISASSEMBLE built-in function.
foreigner 9y ago
I think of myself as a software craftsman. Just like a master carpenter, the things I make should be not only functional but beautiful as well. Of course we should "care" about our work. However there are other metrics besides performance that are worth caring about, e.g. readability, maintainability, etc...
- mercer 9y ago
  Meeting deadlines, getting paid, whether anyone will ever care about the code, again, etc. I also try to be a 'craftsman', and I try to pick work where that's an option, but I find that in practice, more often than not I have to care about various 'other metrics' at the detriment of craftsmanship. Although I suppose being skilled at managing these various metrics could in itself be considered craftsmanship. And perhaps that mindset helps a bit with not getting utterly depressed at some of the shit I'm forced to deliver...
mbrock 9y ago
For most people, "seeing inefficient code" is basically hallucinating. There's real truth to the mantra about premature optimization. I've seen people so many times worry about the performance of something, thinking that some code looks slow, when in actual reality it's a complete nonissue. Software development is all about compromises, just like any kind of real world engineering. Building a stage for a weekend festival is different from building a bridge over a river. We're always on a budget, so if I can work much faster and safer and lose some marginal efficiency, I'll do it—unless marginal efficiency is what I'm competing on, which it usually isn't.
delta1 9y ago
1. Make it work 2. Make it right 3. Make it fast "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." - Donald Knuth
- microcolonel 9y ago
  The problem is that these rules should apply in sequence to a given feature or flow, but they're instead applied in sequence to the whole product. So what happens is: they make everything "work", then they spend the next two decades pecking at making it right, and making it fast takes a back seat. Corollary: interaction latency, power consumption, and other performance characteristics should probably be part of "make it right", and "make it fast" might not even be a value at all.
- Cthulhu_ 9y ago
  The challenge there is that a lot of developers are pushed by their managers to go to the next feature after #1, postponing 2 and 3 for when there's time. Time to market is considered the most important thing. Of course, it accumulates after a year.
  - zimpenfish 9y ago
    > pushed by their managers to go to the next feature Understandable, though, since features are what get customers to pay - 99% of customers will not care about efficient code as long as they can achieve what they're after without screaming.
    - hasenj 9y ago
      Things that are full of bugs and are sluggishly slow are what makes customers scream their lounges out 99% of the time.
- 9y ago
majikandy 9y ago
I’ve spent the last 20 years not caring about performance. Not once has performance ever been a real issue on any project I’ve been on. Imagine how much time I could have wasted prematurely optimising.
- speedplane 9y ago
  You've never written any code to run in parallel? Or used a data structure or algorithm with O(logN) time instead of O(N) time?
  - majikandy 9y ago
    I probably have, without particularly prioritising it. Ok, perhaps I should have said "overly caring about performance". Of course I have used built in functions in languages that take advantage of this when searching or sorting - but no, I can't remember a time when I had to get into the nitty gritty of low level performance enhancement, unless it was in a coding challenge of course.
nnq 9y ago
Blah. You get a lot of performance "for free" by getting the overall architecture of a system right. By starting with optimizations, you usually hit the "limit of your brainpower" very soon, like before you know enough to architect things right, so you mess up the architecture (think like "shit, we ended up with this location dependent time conversion logic in the backend instead of on the clients, so now is un-cacheable, but it's too late to refactor across multiple teams now... shit"). Just avoid building un-optimizable systems (nowadays it's harder than ever to avoid this... all the tools and services lay traps for us), and then you can safely leave the optimizations for later... If you can't avoid building an un-optimizable system, all your clever early optimizations are worthless anyway. If you can, they can be postponed anyway. Now, what I'd really want to read would be an article/book about avoiding the traps that lead people to get stuck with un-optimizable systems! (Though I imagine it's in general an unsolvable problem if natural evolution didn't solve it either: at some point your body accumulated enough de-optimizations that it basically stops working or being fixable/optimizable, so you die, and some of your memes and genes carry on in "rewritten forks" aka "other newly born people".)
ssijak 9y ago
"When people train, they usually don’t try to actually run faster or lift heavier weights. As a relatively healthy computer science professor, how fast I run or how much I can lift is of no practical relevance. " This is really strange. If you don`t try to lift heavier weights you will not have progress. It is called progressive loading. I don`t wan`t to go into the details of the training, but if you do not track and improve your training (with whatever goal you have in mind, there are different kinds of progressions) you are the same as the programmer who write bad code, and knows it is bad, but does not care.
- zimpenfish 9y ago
  > If you don`t try to lift heavier weights you will not have progress. That depends on what you're aiming for - if you're aiming for tone rather than bulk, you'd go for more reps of the same weight rather than stepping up the weight with the same reps, wouldn't you? (Although I guess, technically, that does count as "heavier" since you're still moving more weight in your sets.)
  - tomtomau 9y ago
    The general consensus is there's not really such thing as 'tone' - usually that's just a matter of low bodyfat % which shows your muscles more promimently. What you're talking about somewhat though is balancing hypertrophy and strength goals - where someone may wish to increase their reps to say 8-20 which is demonstrated with studies to increase muscle size, but is poorer for progressing with strength (where 3-8 reps is more effective)
- s_m_t 9y ago
  I interpreted that as "When people train, their goal in of itself is not usually to run faster or lift heavier weights". (the following might diverge from the actual content of the article, the author's english doesn't seem to be that great and I don't think we'll gain anything by taking the article literally)The goal might be overall health, attractiveness, etc. However, that doesn't mean they won't take the training seriously if they think it will help their larger goal. If you saw someone at the gym dilly dallying and doing a seriously half-assed rotation you might think they don't really care about their end goal. In the same vein if you saw a programmer coding something extremely slow just because they couldn't be bothered to put in the tiny bit of extra effort to get to a baseline of performance you might think they don't take their job or future skillset very seriously. Always thinking about the performance of your implementation is a form of training just like lifting weights is. Even if you don't care about making this particular section of code fast, or you don't care whether this particular dumbbell is lifted up and down 10 times in a row, you do those things in service of a greater goal.
a_imho 9y ago
There is a reason we don’t tend to hire people who show up to work in dirty t-shirts. It is not that we particularly care about dirty t-shirts, it is that we want people who care about their work. I don't quite agree the cleanliness of clothing is a good proxy of one's craftsmanship or personality, even if t-shirts are acceptable garment at a particular workplace.
agnivade 9y ago
It maybe different in US. But in some Asian countries, most people care about the money at the end of the day. So yea, they don't care. But does it matter ? People are happy as long as they get paid at the end of the day. They don't care about the quality of code written. Not saying its a good thing or bad thing. Just the way it is.

56 points by frostmatthew 9y ago

There is indeed a point where being fit and clean turns into an obsession. There is such a thing as premature optimization. Commonly the mistakes costing performance are caused in part by bad design though, typically arrived at by either not caring or not fixing a quick and dirty prototype with something reasonable. Or "brute force" bug fixing in concurrent code. If I got paid for every unnecessary or replaceable synchronized statement in Java, I'd be rich. Or a fat data copy made using a queue.

tluyben2 9y ago
> There is indeed a point where being fit and clean turns into an obsession. There is such a thing as premature optimization. There is also something as no optimization (at any point), and I see far more often in the wild. I think more devs should run their app on a cheap Android phone or cheap consumer entry laptop to see what is happening when it's not running on 32 gb i7 octa cores. I recently got some entry level machine from a friend to play around with; it has 4gb and flash drive; it is eerie how slow the thing is; this is what is sold in walmart etc at the entry price level, so fair to say, if you are B2C software dev, this is your audience. For fun, I tried to do some development on it; Atom/Vscode started both snappy, but when devving on it a bit, they became rapidly unusable (cpu 100% always and typing appearing seconds after you typed it); emacs was ok; vim did well. You cannot run Electron on that and yet, that's what many people deliver. And my suspicion is that most don't optimize when using that (if you are fast software minded, you wouldn't use it in the first place, not because it's inherently bad, but because there are too many moving parts you cannot influence). Please try it on these machines to know what you are doing to your users :)

jackjeff 9y ago

“Avoid multiple passes over the data when one would do.” Totally disagree. Unless performance is an issue (like I’m not dealing with a trivial number of elements), I would rather use functional programming approaches to sort data into shape (think map/filter/reduce). These approaches typically result in passing over the data multiple times, and often performing multiple copies, but it makes for readable and less error prone code. Doing the performant thing consists of writing a for loop (or multiple of them unless you unroll) and automatically result with a large custom body of code performing multiple transformations on the data. It’s just wasted mental effort when you write it and every time when you happen to read it again. Most collections/array I process have trivial number of elements (<100) and you get virtually no benefit from writing optimized code. I have also written a lot of performant code, and the lesson is always benchmark benchmark benchmark. The results do not always fit the mental model about what you think should be the fastest. In particuliar avoiding cache misses is far more important than you might think.

AstralStorm 9y ago
Readable and less error prone? As opposed as to proven to be correct for example? Recursive functional code is a pain to prove to be right and not blow up stack. Multiple passes (when interspersed with other accesses) can blow up cache. Even calling through a lambda is a cost compilers cannot easily optimize away unless you help them. (Even in C++.) You do get the benefit of optimizing all the "trivial data size" code when it is all the functions being written in such silly way. Difference of 10% in one function as opposed to across whole application. Benchmarking is often even harder to do right than writing good tests. (In fact is tied to it.) Instead of benchmarking, be a real computer scientist and prove bounds and memory allocations.
- tigershark 9y ago
  How is it difficult to prove if a recursive call is tail-recursive or not? If it's tail-recursive and your language supports tail-call optimisation then you proved that there will be no stack blow-up.
  - AstralStorm 9y ago
    For simple calls, not. For something more involved? (Dependent functions, corecursion, partially stateful code.) Very which is why compilers fail to optimize it in general.
- lmm 9y ago
  > Recursive functional code is a pain to prove to be right and not blow up stack. Much easier to prove recursion correct than to prove a loop always terminates, since the recursion makes the way the state changes between steps much more explicit. If you're really worried about this you can use Idris and require all your functions to be total.
elcritch 9y ago
Plus some functional languages can combine those steps at compile, giving you the best of both worlds. Rust for example elides away many lambda's in map operations. I don't have the link handy, but a post a few months back showed Rust and Haskell functional patterns could almost match their hard coded imperative style implementations and C/C++ cousins.

blux 9y ago

Regarding the point "Don’t use floating-point operations when integers will do." I agree that using integer arithmetic can result in cleaner, easier to reason about code when applicable. It then follows that because of these traits the software will faster. But integer arithmetic is not necessarily faster than floating point arithmetic in general; see for example: https://youtu.be/3K2LmnaLLF8?t=31m10s https://youtu.be/3K2LmnaLLF8?t=31m10s.

AstralStorm 9y ago
Only if you either have to pay the extra cost of handling overflows, underflows, saturation or normalization. (First three involve conditionals which may be mispredicted, last a division or multiplication.) Likewise correct rounding. (Multiple math ops and sometimes a conditional too.) Or when the platform has no integer vector math for your type of choice. Which is still somewhat common.
gopalv 9y ago
> But integer arithmetic is not necessarily faster than floating point arithmetic in general; GPUs was the first place I feel like floats got a much better pathway than any other data type - graphical pixel manipulations also can get away with much higher arithmetic errors, because it's all going to get rounded down to a pixel eventually. On the other hand, the only place where I've really worked heavily with fixed point was on a graphical interface toolkit (MIDP on ARM I was working with would've had to use softfp for floats, so fixed point was hugely superior).
- AstralStorm 9y ago
  Come on, all reasonable ARM have NEON and VFPv3 (including cortex M) about the only place I saw something that didn't was an old internet router. Hardware completely unsuited to maths. Softfp does not preclude the use of floating point math at all. It means you pay an extra cost when passing floats as arguments. (Which means you should rather use a hardfp math library if possible.)
  - gopalv 9y ago

twic 9y ago

> But then, I dress cleanly every single day even if I stay at home. And you should too. Nope. It's good to know how to produce fast software, but also good to know when to do so. Lemire sounds like he has some psychological issues to work out.

michaelg7x 9y ago

Code bloat could easily be sorted out by forcing people to read the generated assembly of their creations.

reikonomusha 9y ago
This is easy and convenient in Common Lisp with the DISASSEMBLE built-in function.

foreigner 9y ago

I think of myself as a software craftsman. Just like a master carpenter, the things I make should be not only functional but beautiful as well. Of course we should "care" about our work. However there are other metrics besides performance that are worth caring about, e.g. readability, maintainability, etc...

mercer 9y ago
Meeting deadlines, getting paid, whether anyone will ever care about the code, again, etc. I also try to be a 'craftsman', and I try to pick work where that's an option, but I find that in practice, more often than not I have to care about various 'other metrics' at the detriment of craftsmanship. Although I suppose being skilled at managing these various metrics could in itself be considered craftsmanship. And perhaps that mindset helps a bit with not getting utterly depressed at some of the shit I'm forced to deliver...

mbrock 9y ago

For most people, "seeing inefficient code" is basically hallucinating. There's real truth to the mantra about premature optimization. I've seen people so many times worry about the performance of something, thinking that some code looks slow, when in actual reality it's a complete nonissue. Software development is all about compromises, just like any kind of real world engineering. Building a stage for a weekend festival is different from building a bridge over a river. We're always on a budget, so if I can work much faster and safer and lose some marginal efficiency, I'll do it—unless marginal efficiency is what I'm competing on, which it usually isn't.

delta1 9y ago

1. Make it work 2. Make it right 3. Make it fast "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." - Donald Knuth

microcolonel 9y ago
The problem is that these rules should apply in sequence to a given feature or flow, but they're instead applied in sequence to the whole product. So what happens is: they make everything "work", then they spend the next two decades pecking at making it right, and making it fast takes a back seat. Corollary: interaction latency, power consumption, and other performance characteristics should probably be part of "make it right", and "make it fast" might not even be a value at all.
Cthulhu_ 9y ago
The challenge there is that a lot of developers are pushed by their managers to go to the next feature after #1, postponing 2 and 3 for when there's time. Time to market is considered the most important thing. Of course, it accumulates after a year.
- zimpenfish 9y ago
  > pushed by their managers to go to the next feature Understandable, though, since features are what get customers to pay - 99% of customers will not care about efficient code as long as they can achieve what they're after without screaming.
  - hasenj 9y ago
    Things that are full of bugs and are sluggishly slow are what makes customers scream their lounges out 99% of the time.
9y ago

majikandy 9y ago

I’ve spent the last 20 years not caring about performance. Not once has performance ever been a real issue on any project I’ve been on. Imagine how much time I could have wasted prematurely optimising.

speedplane 9y ago
You've never written any code to run in parallel? Or used a data structure or algorithm with O(logN) time instead of O(N) time?
- majikandy 9y ago
  I probably have, without particularly prioritising it. Ok, perhaps I should have said "overly caring about performance". Of course I have used built in functions in languages that take advantage of this when searching or sorting - but no, I can't remember a time when I had to get into the nitty gritty of low level performance enhancement, unless it was in a coding challenge of course.

nnq 9y ago

Blah. You get a lot of performance "for free" by getting the overall architecture of a system right. By starting with optimizations, you usually hit the "limit of your brainpower" very soon, like before you know enough to architect things right, so you mess up the architecture (think like "shit, we ended up with this location dependent time conversion logic in the backend instead of on the clients, so now is un-cacheable, but it's too late to refactor across multiple teams now... shit"). Just avoid building un-optimizable systems (nowadays it's harder than ever to avoid this... all the tools and services lay traps for us), and then you can safely leave the optimizations for later... If you can't avoid building an un-optimizable system, all your clever early optimizations are worthless anyway. If you can, they can be postponed anyway. Now, what I'd really want to read would be an article/book about avoiding the traps that lead people to get stuck with un-optimizable systems! (Though I imagine it's in general an unsolvable problem if natural evolution didn't solve it either: at some point your body accumulated enough de-optimizations that it basically stops working or being fixable/optimizable, so you die, and some of your memes and genes carry on in "rewritten forks" aka "other newly born people".)

ssijak 9y ago

"When people train, they usually don’t try to actually run faster or lift heavier weights. As a relatively healthy computer science professor, how fast I run or how much I can lift is of no practical relevance. " This is really strange. If you don`t try to lift heavier weights you will not have progress. It is called progressive loading. I don`t wan`t to go into the details of the training, but if you do not track and improve your training (with whatever goal you have in mind, there are different kinds of progressions) you are the same as the programmer who write bad code, and knows it is bad, but does not care.

zimpenfish 9y ago
> If you don`t try to lift heavier weights you will not have progress. That depends on what you're aiming for - if you're aiming for tone rather than bulk, you'd go for more reps of the same weight rather than stepping up the weight with the same reps, wouldn't you? (Although I guess, technically, that does count as "heavier" since you're still moving more weight in your sets.)
- tomtomau 9y ago
  The general consensus is there's not really such thing as 'tone' - usually that's just a matter of low bodyfat % which shows your muscles more promimently. What you're talking about somewhat though is balancing hypertrophy and strength goals - where someone may wish to increase their reps to say 8-20 which is demonstrated with studies to increase muscle size, but is poorer for progressing with strength (where 3-8 reps is more effective)
s_m_t 9y ago
I interpreted that as "When people train, their goal in of itself is not usually to run faster or lift heavier weights". (the following might diverge from the actual content of the article, the author's english doesn't seem to be that great and I don't think we'll gain anything by taking the article literally)The goal might be overall health, attractiveness, etc. However, that doesn't mean they won't take the training seriously if they think it will help their larger goal. If you saw someone at the gym dilly dallying and doing a seriously half-assed rotation you might think they don't really care about their end goal. In the same vein if you saw a programmer coding something extremely slow just because they couldn't be bothered to put in the tiny bit of extra effort to get to a baseline of performance you might think they don't take their job or future skillset very seriously. Always thinking about the performance of your implementation is a form of training just like lifting weights is. Even if you don't care about making this particular section of code fast, or you don't care whether this particular dumbbell is lifted up and down 10 times in a row, you do those things in service of a greater goal.

a_imho 9y ago

There is a reason we don’t tend to hire people who show up to work in dirty t-shirts. It is not that we particularly care about dirty t-shirts, it is that we want people who care about their work. I don't quite agree the cleanliness of clothing is a good proxy of one's craftsmanship or personality, even if t-shirts are acceptable garment at a particular workplace.

agnivade 9y ago

It maybe different in US. But in some Asian countries, most people care about the money at the end of the day. So yea, they don't care. But does it matter ? People are happy as long as they get paid at the end of the day. They don't care about the quality of code written. Not saying its a good thing or bad thing. Just the way it is.