7 ms·

Reflections on Software Performance

154 points by cespare 7y ago

branko_d 7y ago
Yes, performance is a feature. You have to plan and architecture for it, and you can't just tack it on after the fact by profiling a few hot codepaths (though you should do that too). Performance can be different from "scalability" though. Sometimes, there is tension between the two.
- sokoloff 7y ago
  As someone who has probably wasted more time than optimal agonizing over performance (I used to be a game dev for console and PC), what you say is absolutely true, but I think engineers have a tendency to think about Facebook scale before they have triple-digits of users. That is usually a mistake.
  - jcelerier 7y ago
    > engineers have a tendency to think about Facebook scale before they have triple-digits of users. I don't think number of users is what matters except for web services. If you make for instance a photo or audio editing software, there will never be enough performance. It isn't acceptable to say to your users that your software works fine until, say, 8192*8192 images : if you want to compete against other software, you have to consistently be the fastest at every task that artists may throw at you (else you will get bad reviews on specialist & prosumer press / forums / blogs which can kill your business pretty efficiently... as it takes hundreds of people saying "it's fast" to offset the effect of a single press article saying "it's slow as shit" in art communities).
  - GlitchMr 7y ago
    A web application doesn't have to be scalable. Stack Overflow for instance could run on a single web server (source: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...), and this is a very popular website with Alexa rank of 39.
    - saagarjha 7y ago
      Hacker News runs on a single machine, apparently.
      - karatestomp 7y ago
        Your average "web scale" cloud system with Node and lambdas and VMs and distributed databases galore feels slow and clunky as hell before it's even under load, to those of us who remember "bad" old LAMP stacks running on a 1U server. Not that I want to go back to that, exactly, but our performance expectations have gotten really screwy. [EDIT] or, hell, take "Web 2.0". Piles of code and frameworks and shadow DOMs and shit all chasing and touting "performance" while full-page-loading low-JS sites like Craigslist and Basic HTML Gmail (or HN) leave them in the dust. Know what those are doing? Handing HTML to the browser and letting it render it. No JS render step, no fetching JSON then passing it through Redux and then making twenty function calls to eventually modify a shadow DOM to later apply to the real one. The browser is fast. Your JS is what's fucking slow and eating all my memory.
        adossi 7y ago
        I would argue JS is a lot faster than you think, and the sluggishness you feel is due to the massive number of files being downloaded. Dozens of JS libraries (think jQuery, Bootstrap, etc.), several CSS stylesheets, and a hundred images or more. If each one of those files is even a few kilobytes each, there is still a 10ms (or even 100ms) download time on each of them, and unfortunately its very common for these files to be downloaded sequentially. JS on its own is quite performant.
        FridgeSeal 7y ago
        Whilst JS might be somewhat fast, you know what’s even faster? Designing your application so that it doesn’t need it. If I never have to download, parse and execute the JS, I’m already way ahead. With better privacy to boot.
        7y ago
  - VBprogrammer 7y ago
    I don't even think that is the problem. In my career I've seen full blown arguments over loops which, as a worst case, could only ever contain a handful of items, or are only executed once per user action. That's not facebook scale or not, it's just worrying about the wrong things.
    - thrower123 7y ago
      Fundamentally, some people are allergic to actually profiling things and collecting data. I don't understand it, but there are lots of people who would rather spend hours talking over things in the theoretical sense, rather than spending a half hour coding it both ways and benchmarking it to get some actual data to make a decision on.
      - gpderetta 7y ago
        On the other hand you can't benchmark every single line of potentially problematic code and even if you could, syntetic tests are not reliable. If by experience you know that a certain solution can be problematic and there is a different solution which has reasonable implementation costs, you should do what your experience tells you especially if fixing it after the fact would have significantly higher implementation costs.
- Koshkin 7y ago
  Sometimes it is helpful to have two implementations - one being a “reference” implementation that may sacrifice performance in favor of the guaranteed correctness, and the other being a high-performance production-quality implementation which may have little in common with its reference counterpart except for the same business logic they both implement.
- zzzcpan 7y ago
  > You have to plan and architecture for it And even that is not enough. You also have to know how to plan and architecture for it, have a well developed mental model of what can get you there, which means you have to practice doing high performance things, follow research and high performance ideas and generally have a habit of building things that are fast. Few people actually do that.
  - smallstepforman 7y ago
    Very well said. I agree.
- bachmeier 7y ago
  > Performance can be different from "scalability" though. Sometimes, there is tension between the two. And extensibility. It's not necessarily fun trying to add a new feature to someone else's "highly optimized" code.
magicalhippo 7y ago
> What is perhaps less apparent is that having faster tools changes how users use a tool or perform a task. Important here is that for a user, "faster" means with respect to achieving the goal. At work we've created a module where, instead of punching line items by hand and augmenting the data by memory or web searches, the user can paste data from Excel (or import from OCR) and the system remembers mappings for the data augmentation. After a couple of initial runs for the mapping table to build our users can process thousands of lines in 10 minutes or less, a task that could take the better part of a day. It's not uncommon with some follow-up support after new customers start with this module, so I often get to follow the transformation from before to after. They also quickly get accustomed. We'll hear it quick if those 10 minutes grows to 20 from one build to another, not much thought is given to how 20 minutes is still a lot faster than they'd be able to punch those 8000 lines :)
simonw 7y ago
This piece is excellent. I really love how it challenges the "optimize last" philosophy by pointing out that performance is integral to how a tool will be used and designing it in as part of the architecture from the very start can produce a fundamentally different product, even if it appears to have the same features.
- jmull 7y ago
  I think premature optimization remains as bad as always. But you design for performance. The proper time to address it is at design time. That's not premature, that's the right moment. I wish we could reserve the word "optimization" for the kinds of things you can do after implementation to improve the performance without significantly changing the design. That is, let's continue to optimize last, but not try to make the word optimize mean address performance in general. That's not what the word means, after all.
  - deleted 7y ago
    [deleted]
  - gameswithgo 7y ago
    premature optimization is bad since it is a tautology. but does it ever happen? =)
    - zzzcpan 7y ago
      Best to call it unneeded optimization then, not premature.
    - jmull 7y ago
      I don't think you mean tautology.
      - gameswithgo 7y ago
        I definitely do, but maybe I am wrong! If the optimization wasn't bad, it wouldn't have been premature.
tluyben2 7y ago
> It seems increasingly common these days to not worry about performance at all, You don't even have to continue there. People, who should know better, assume that 'modern cloud stuff' will make this trivial. You just add some auto-scaling and it can handle anything. Until it grinds to a halt because it cannot scale beyond bottlenecks (relational database most likely) or the credit card is empty trying to pull in more resources beyond the ridiculous amount that were already being used for the (relatively) tiny amount of users. This will only get worse as people generally use the 'premature optimization' (delivering software for launch is not premature!) and 'people are more expensive than more servers' (no they are not with some actual traffic and O(n^2) performing crap) as excuse to not even try to understand this anymore. Same with storage space; with NoSQL, there are terabytes of data growing out of nowhere because 'we don't care as it works and it's 'fast' to market, again 'programmers are more expensive than more hardware!'). Just run a script to fire up 500 aws instances backed by Dynamo and fall asleep. I am not so worried about premature optimization ; I am more worried about never optimization. And at that; i'm really worried about my (mostly younger) colleagues simply not caring because they believe it's a waste of time.
- vlovich123 7y ago
  There's also something to be said for building better tooling in this area. Not everyone can achieve expertise in everything. Better tooling helps level the playing feel (& eventually outperform experts when the tooling becomes indispensable). You may think that's a cop-out, but consider something like coz[1]. Sqlite is managed and maintained by experts. There's significant capital behind investing engineering effort. However, better tooling still managed to locate 25% of performance improvement[2] & even 9% in memcached. Even experts have their limits & of course these tools require expertise so a tool like coz is still an expert-only tool. The successful evolution of the underlying concept for mass adoption will happen when it's possible to convert "expert speak" into something that can be easily and simply communicated outside CPU or compiler experts to meet the user on their knowledge level so they can dig in as deep as they need to/want to. [1] https://github.com/plasma-umass/coz https://github.com/plasma-umass/coz [2] https://arxiv.org/abs/1608.03676 https://arxiv.org/abs/1608.03676
  - 7y ago
ken 7y ago
> the SQLite 3.8.7 release, which was 50% faster than the previous release Nit: the link says it’s 10% faster than the previous release. It’s 50% faster than some arbitrary point in the past, perhaps the time when they began their CPU-based profile optimization.
alexeiz 7y ago
Nice and clean static layout. A rarity these days when blog post web pages tend to be overloaded with headers, footers, and various crappy interactive elements.
deleted 7y ago
[deleted]
bcrosby95 7y ago
I've heard that performance is a feature but I feel like that understates the effort involved in seeking performance for a piece of software. If you want to call it a feature, its closer to N features: 1 for each feature you have. If you have 10 features, and add performance, the effort involved isn't like having 11 features. It's like having 20 features. The effect is multiplicative. This is because performance is a cross cutting concern. Many times cross cutting concerns are easy to inject/share effort with. But not with performance. You can't just add an @OptimizeThis annotation to speed up your code. Performance tuning tends to be very specific to each chunk of code.
- gameswithgo 7y ago
  If everyone on the team makes it a habit of worrying about it, everyone gets better at it. It becomes a part of the review process - "this looks correct, but is there a faster way?" or "this looks very fast, but we could make it a LOT simpler and only a little slower, maybe we should."
PouyaL 7y ago
Great stuff. We need to work on the fact at the moment, though that happens at time goes by.
igouy 7y ago
> I’ve really strongly come to believe that… I’ve come to believe really strongly that…
lcfcjs2 7y ago
This guy hasn't worked in a start up. Products should be iterative. You can't say your application is slow because it was architected badly.
luord 7y ago
> And while the SQLite developers were able to do this work after the fact, the more 1% regressions you can avoid in the first place, the easier this work is. That mention of regressions seems, IMO, a slightly out of left field attempt at dismissing how the SQLite example shows that you can, in fact, "make it fast" later. Maybe he should've a picked a different example entirely because it undermined his point a little bit.[1] All in all, his entire thesis comes from talking about a typechecker, which is indeed a piece of software whose each component in general contributes to the performance of the whole. It isn't a set of disparage moving parts (at least, from what I remember of my time studying parsers in college), so it's very hard to optimize by sections because all components mostly feed off each other. Most software is not a typechecking tool, plenty (dare I say, most) of software does have specific bottlenecks. Though I do agree that, even if we aren't focusing on it right away, we should keep performance in mind from the beginning. If nothing else, making the application/system as modular as possible, so as to make it easier to replace the slowest moving parts. [1] Which is a good thing IMO, as it highlights how this is all about trade-offs. "Premature optimization is the root of all evil", "CPU time is always cheaper than an engineer’s time", etc., are, in fact, mostly true, at least when talking about consumer software/saas: it really doesn't matter how fast your application is because crafting fast software is slower than crafting slow software, and your very performant tool is not used by anyone because everyone is already using that other tool that is slower but came out first.

154 points by cespare 7y ago

Yes, performance is a feature. You have to plan and architecture for it, and you can't just tack it on after the fact by profiling a few hot codepaths (though you should do that too). Performance can be different from "scalability" though. Sometimes, there is tension between the two.

sokoloff 7y ago
As someone who has probably wasted more time than optimal agonizing over performance (I used to be a game dev for console and PC), what you say is absolutely true, but I think engineers have a tendency to think about Facebook scale before they have triple-digits of users. That is usually a mistake.
- jcelerier 7y ago
  > engineers have a tendency to think about Facebook scale before they have triple-digits of users. I don't think number of users is what matters except for web services. If you make for instance a photo or audio editing software, there will never be enough performance. It isn't acceptable to say to your users that your software works fine until, say, 8192*8192 images : if you want to compete against other software, you have to consistently be the fastest at every task that artists may throw at you (else you will get bad reviews on specialist & prosumer press / forums / blogs which can kill your business pretty efficiently... as it takes hundreds of people saying "it's fast" to offset the effect of a single press article saying "it's slow as shit" in art communities).
- GlitchMr 7y ago
  A web application doesn't have to be scalable. Stack Overflow for instance could run on a single web server (source: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...), and this is a very popular website with Alexa rank of 39.
  - saagarjha 7y ago
    Hacker News runs on a single machine, apparently.
    - karatestomp 7y ago
      Your average "web scale" cloud system with Node and lambdas and VMs and distributed databases galore feels slow and clunky as hell before it's even under load, to those of us who remember "bad" old LAMP stacks running on a 1U server. Not that I want to go back to that, exactly, but our performance expectations have gotten really screwy. [EDIT] or, hell, take "Web 2.0". Piles of code and frameworks and shadow DOMs and shit all chasing and touting "performance" while full-page-loading low-JS sites like Craigslist and Basic HTML Gmail (or HN) leave them in the dust. Know what those are doing? Handing HTML to the browser and letting it render it. No JS render step, no fetching JSON then passing it through Redux and then making twenty function calls to eventually modify a shadow DOM to later apply to the real one. The browser is fast. Your JS is what's fucking slow and eating all my memory.
      - adossi 7y ago
        I would argue JS is a lot faster than you think, and the sluggishness you feel is due to the massive number of files being downloaded. Dozens of JS libraries (think jQuery, Bootstrap, etc.), several CSS stylesheets, and a hundred images or more. If each one of those files is even a few kilobytes each, there is still a 10ms (or even 100ms) download time on each of them, and unfortunately its very common for these files to be downloaded sequentially. JS on its own is quite performant.
        FridgeSeal 7y ago
        Whilst JS might be somewhat fast, you know what’s even faster? Designing your application so that it doesn’t need it. If I never have to download, parse and execute the JS, I’m already way ahead. With better privacy to boot.
        7y ago
- VBprogrammer 7y ago
  I don't even think that is the problem. In my career I've seen full blown arguments over loops which, as a worst case, could only ever contain a handful of items, or are only executed once per user action. That's not facebook scale or not, it's just worrying about the wrong things.
  - thrower123 7y ago
    Fundamentally, some people are allergic to actually profiling things and collecting data. I don't understand it, but there are lots of people who would rather spend hours talking over things in the theoretical sense, rather than spending a half hour coding it both ways and benchmarking it to get some actual data to make a decision on.
    - gpderetta 7y ago
      On the other hand you can't benchmark every single line of potentially problematic code and even if you could, syntetic tests are not reliable. If by experience you know that a certain solution can be problematic and there is a different solution which has reasonable implementation costs, you should do what your experience tells you especially if fixing it after the fact would have significantly higher implementation costs.
Koshkin 7y ago
Sometimes it is helpful to have two implementations - one being a “reference” implementation that may sacrifice performance in favor of the guaranteed correctness, and the other being a high-performance production-quality implementation which may have little in common with its reference counterpart except for the same business logic they both implement.
zzzcpan 7y ago
> You have to plan and architecture for it And even that is not enough. You also have to know how to plan and architecture for it, have a well developed mental model of what can get you there, which means you have to practice doing high performance things, follow research and high performance ideas and generally have a habit of building things that are fast. Few people actually do that.
- smallstepforman 7y ago
  Very well said. I agree.
bachmeier 7y ago
> Performance can be different from "scalability" though. Sometimes, there is tension between the two. And extensibility. It's not necessarily fun trying to add a new feature to someone else's "highly optimized" code.

magicalhippo 7y ago

> What is perhaps less apparent is that having faster tools changes how users use a tool or perform a task. Important here is that for a user, "faster" means with respect to achieving the goal. At work we've created a module where, instead of punching line items by hand and augmenting the data by memory or web searches, the user can paste data from Excel (or import from OCR) and the system remembers mappings for the data augmentation. After a couple of initial runs for the mapping table to build our users can process thousands of lines in 10 minutes or less, a task that could take the better part of a day. It's not uncommon with some follow-up support after new customers start with this module, so I often get to follow the transformation from before to after. They also quickly get accustomed. We'll hear it quick if those 10 minutes grows to 20 from one build to another, not much thought is given to how 20 minutes is still a lot faster than they'd be able to punch those 8000 lines :)

simonw 7y ago

This piece is excellent. I really love how it challenges the "optimize last" philosophy by pointing out that performance is integral to how a tool will be used and designing it in as part of the architecture from the very start can produce a fundamentally different product, even if it appears to have the same features.

jmull 7y ago
I think premature optimization remains as bad as always. But you design for performance. The proper time to address it is at design time. That's not premature, that's the right moment. I wish we could reserve the word "optimization" for the kinds of things you can do after implementation to improve the performance without significantly changing the design. That is, let's continue to optimize last, but not try to make the word optimize mean address performance in general. That's not what the word means, after all.
- deleted 7y ago
  [deleted]
- gameswithgo 7y ago
  premature optimization is bad since it is a tautology. but does it ever happen? =)
  - zzzcpan 7y ago
    Best to call it unneeded optimization then, not premature.
  - jmull 7y ago
    I don't think you mean tautology.
    - gameswithgo 7y ago
      I definitely do, but maybe I am wrong! If the optimization wasn't bad, it wouldn't have been premature.

tluyben2 7y ago

> It seems increasingly common these days to not worry about performance at all, You don't even have to continue there. People, who should know better, assume that 'modern cloud stuff' will make this trivial. You just add some auto-scaling and it can handle anything. Until it grinds to a halt because it cannot scale beyond bottlenecks (relational database most likely) or the credit card is empty trying to pull in more resources beyond the ridiculous amount that were already being used for the (relatively) tiny amount of users. This will only get worse as people generally use the 'premature optimization' (delivering software for launch is not premature!) and 'people are more expensive than more servers' (no they are not with some actual traffic and O(n^2) performing crap) as excuse to not even try to understand this anymore. Same with storage space; with NoSQL, there are terabytes of data growing out of nowhere because 'we don't care as it works and it's 'fast' to market, again 'programmers are more expensive than more hardware!'). Just run a script to fire up 500 aws instances backed by Dynamo and fall asleep. I am not so worried about premature optimization ; I am more worried about never optimization. And at that; i'm really worried about my (mostly younger) colleagues simply not caring because they believe it's a waste of time.

vlovich123 7y ago
There's also something to be said for building better tooling in this area. Not everyone can achieve expertise in everything. Better tooling helps level the playing feel (& eventually outperform experts when the tooling becomes indispensable). You may think that's a cop-out, but consider something like coz[1]. Sqlite is managed and maintained by experts. There's significant capital behind investing engineering effort. However, better tooling still managed to locate 25% of performance improvement[2] & even 9% in memcached. Even experts have their limits & of course these tools require expertise so a tool like coz is still an expert-only tool. The successful evolution of the underlying concept for mass adoption will happen when it's possible to convert "expert speak" into something that can be easily and simply communicated outside CPU or compiler experts to meet the user on their knowledge level so they can dig in as deep as they need to/want to. [1] https://github.com/plasma-umass/coz https://github.com/plasma-umass/coz [2] https://arxiv.org/abs/1608.03676 https://arxiv.org/abs/1608.03676
- 7y ago

ken 7y ago

> the SQLite 3.8.7 release, which was 50% faster than the previous release Nit: the link says it’s 10% faster than the previous release. It’s 50% faster than some arbitrary point in the past, perhaps the time when they began their CPU-based profile optimization.

alexeiz 7y ago

Nice and clean static layout. A rarity these days when blog post web pages tend to be overloaded with headers, footers, and various crappy interactive elements.

deleted 7y ago

[deleted]

bcrosby95 7y ago

I've heard that performance is a feature but I feel like that understates the effort involved in seeking performance for a piece of software. If you want to call it a feature, its closer to N features: 1 for each feature you have. If you have 10 features, and add performance, the effort involved isn't like having 11 features. It's like having 20 features. The effect is multiplicative. This is because performance is a cross cutting concern. Many times cross cutting concerns are easy to inject/share effort with. But not with performance. You can't just add an @OptimizeThis annotation to speed up your code. Performance tuning tends to be very specific to each chunk of code.

gameswithgo 7y ago
If everyone on the team makes it a habit of worrying about it, everyone gets better at it. It becomes a part of the review process - "this looks correct, but is there a faster way?" or "this looks very fast, but we could make it a LOT simpler and only a little slower, maybe we should."

PouyaL 7y ago

Great stuff. We need to work on the fact at the moment, though that happens at time goes by.

igouy 7y ago

> I’ve really strongly come to believe that… I’ve come to believe really strongly that…

lcfcjs2 7y ago

This guy hasn't worked in a start up. Products should be iterative. You can't say your application is slow because it was architected badly.

luord 7y ago

> And while the SQLite developers were able to do this work after the fact, the more 1% regressions you can avoid in the first place, the easier this work is. That mention of regressions seems, IMO, a slightly out of left field attempt at dismissing how the SQLite example shows that you can, in fact, "make it fast" later. Maybe he should've a picked a different example entirely because it undermined his point a little bit.[1] All in all, his entire thesis comes from talking about a typechecker, which is indeed a piece of software whose each component in general contributes to the performance of the whole. It isn't a set of disparage moving parts (at least, from what I remember of my time studying parsers in college), so it's very hard to optimize by sections because all components mostly feed off each other. Most software is not a typechecking tool, plenty (dare I say, most) of software does have specific bottlenecks. Though I do agree that, even if we aren't focusing on it right away, we should keep performance in mind from the beginning. If nothing else, making the application/system as modular as possible, so as to make it easier to replace the slowest moving parts. [1] Which is a good thing IMO, as it highlights how this is all about trade-offs. "Premature optimization is the root of all evil", "CPU time is always cheaper than an engineer’s time", etc., are, in fact, mostly true, at least when talking about consumer software/saas: it really doesn't matter how fast your application is because crafting fast software is slower than crafting slow software, and your very performant tool is not used by anyone because everyone is already using that other tool that is slower but came out first.