3 ms·
When starting a project, I prefer to have a "performance budget'. For example "The application must start in under one second". Then during development you ca
by londons_explore 7y ago
When starting a project, I prefer to have a "performance budget'.
For example "The application must start in under one second".
Then during development you can divide up the budget - 400 milliseconds for loading dependencies, 200 milliseconds for rendering the UI, etc.
Then you can decide if any given optimization is needed or necessary to meet your goals.
- londons_explore 7y agoMy latest project has a performance budget of (on average) 150 milliseconds per click, and it's a web-app. It's going to be a tough goal to meet, considering typical web apps today take 5000ms to load!
- touisteur 7y agoThat's what's often done in mission-critical software with critical latency elements. Problem is, you have to make sure those values are directly traced to either specific customer requirements, whole system spec or normative reference. It's too easy for systems engineers to come up with hard-to-reach numbers with no grounding on the Real customer need... You also have to specify and measure exactly the correct element. Sometimes the latency budget is split in not clearly-cut parts, and on events or categories of events that are not clearly defined. Saying 'all requests should give a complete result in < 20ms' just isn't realistic and creates a world of hurt. What kind of request? Always, always? In what load conditions? What about error conditions? Fail-over conditions? Hot/cold start?... It's not simple like specifying a clear-cut feature. Performance budget should often not be all black and white. Especially when trying to push the envelope of what modern PC hardware (with OoO execution, complex memory hierarchies, multicore CPU/GPU rapid-busy-bused hybrid archs) can do. It should probably be a risk law, and at least a critical spec, and thoroughly tested and regression-tested as such. And here, sometimes premature optimization will be necessary. If someone gives you hard latency requirements, you'll have to say, very quick, 'I'll need a soft/hard real-time OS it won't do extreme high-throughput'. You'll have to bench... Premature /micro/-optimization is a problem, sure. Make it work, make it right, make it fast. But global optimization must start at the system design level. How much money do you have? What can you do for $X?
- wruza 7y agoHow do you account for n like in O(n)?
- saagarjha 7y agoIn the case you can’t avoid running O(n) code on a code path that affects user experience, pick a large n that would be a reasonable value for a user to have.
- chaboud 7y agoThe problem with this way of thinking is that those budgets can lead to pathological behaviors. Start time is a great example. Let's say you have 1 second to start, but everything you need to do takes 1.2 seconds. Okay... We'll defer some work and show a dummy screen to be "started" in less than 1 second, except now we're loading/drawing the dummy screen, so we're usable in 1.4 seconds, and after a double-start flash, so let's smooth that out with a transition to the interactive UI and we're usable in 2.2 seconds, but we "started" the app in 0.4. Yay? It may sound contrived, but I've seen this very response to strict start KPI's before. People end up optimizing the micro-goal (KPI) rather than the macro-goal (better experience). The flip side problem is, when you have multiple people responsible for parts of the work, people will stop optimizing when they hit their budgeted allotment, rather than working to better the whole. A Dev with a 400ms budget might sleep for 390ms to buy himself three years of squeezing out optimizations... More likely, he'll stop when he gets to 350ms, even though he could have gotten to 250ms without too much effort.
- spinningslate 7y agoSo much this. I'm an advocate of performance metrics being _measured_ and tracked over time, but very careful about setting performance _targets_. It's the latter that tends to cause the dysfunctional behaviour - it's Goodhart's law [0] in action. When it is required then focusing on user value can help: e.g. "<x> seconds to user login being available" rather than "...page loaded". There's still the question of absolute numbers rather than statistical distributions, but that's another topic. [0] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- thrower123 7y agoReminds me of the apocryphal story of the game that was trying to get under it's memory budget to be able to run on one of the early consoles, and after scrimping and saving and crunching, they were still over. Until one of the senior engineers commented out an unused buffer that allocated a couple kb "just in case"
- pieterr 7y ago