4 ms·

The problem with this way of thinking is that those budgets can lead to pathological behaviors. Start time is a great example. Let's say you have 1 second to

The problem with this way of thinking is that those budgets can lead to pathological behaviors. Start time is a great example. Let's say you have 1 second to start, but everything you need to do takes 1.2 seconds. Okay... We'll defer some work and show a dummy screen to be "started" in less than 1 second, except now we're loading/drawing the dummy screen, so we're usable in 1.4 seconds, and after a double-start flash, so let's smooth that out with a transition to the interactive UI and we're usable in 2.2 seconds, but we "started" the app in 0.4. Yay? It may sound contrived, but I've seen this very response to strict start KPI's before. People end up optimizing the micro-goal (KPI) rather than the macro-goal (better experience). The flip side problem is, when you have multiple people responsible for parts of the work, people will stop optimizing when they hit their budgeted allotment, rather than working to better the whole. A Dev with a 400ms budget might sleep for 390ms to buy himself three years of squeezing out optimizations... More likely, he'll stop when he gets to 350ms, even though he could have gotten to 250ms without too much effort.

spinningslate 7y ago
So much this. I'm an advocate of performance metrics being _measured_ and tracked over time, but very careful about setting performance _targets_. It's the latter that tends to cause the dysfunctional behaviour - it's Goodhart's law [0] in action. When it is required then focusing on user value can help: e.g. "<x> seconds to user login being available" rather than "...page loaded". There's still the question of absolute numbers rather than statistical distributions, but that's another topic. [0] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
thrower123 7y ago
Reminds me of the apocryphal story of the game that was trying to get under it's memory budget to be able to run on one of the early consoles, and after scrimping and saving and crunching, they were still over. Until one of the senior engineers commented out an unused buffer that allocated a couple kb "just in case"
- pieterr 7y ago
  This one? https://news.ycombinator.com/item?id=776296 https://news.ycombinator.com/item?id=776296
  - thrower123 7y ago
    That's the one, although I've seen many different variations on the theme.
einpoklum 7y ago
> Yay If the startup screen allows you to, say, launch an "Open File/Project/Folder" dialog or load one of the recently-worked-on items, but won't let you do anything meaningful - then yes, Yay. It's quite realistic to load that kind of UI in a lot less time than the "meat" of an app is usable. I would take 0.4 seconds for that and 2.2 for the whole app over 1.2 for the whole app - any day of the week.

spinningslate 7y ago

So much this. I'm an advocate of performance metrics being _measured_ and tracked over time, but very careful about setting performance _targets_. It's the latter that tends to cause the dysfunctional behaviour - it's Goodhart's law [0] in action. When it is required then focusing on user value can help: e.g. "<x> seconds to user login being available" rather than "...page loaded". There's still the question of absolute numbers rather than statistical distributions, but that's another topic. [0] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

thrower123 7y ago

Reminds me of the apocryphal story of the game that was trying to get under it's memory budget to be able to run on one of the early consoles, and after scrimping and saving and crunching, they were still over. Until one of the senior engineers commented out an unused buffer that allocated a couple kb "just in case"

pieterr 7y ago
This one? https://news.ycombinator.com/item?id=776296 https://news.ycombinator.com/item?id=776296
- thrower123 7y ago
  That's the one, although I've seen many different variations on the theme.

einpoklum 7y ago

> Yay If the startup screen allows you to, say, launch an "Open File/Project/Folder" dialog or load one of the recently-worked-on items, but won't let you do anything meaningful - then yes, Yay. It's quite realistic to load that kind of UI in a lot less time than the "meat" of an app is usable. I would take 0.4 seconds for that and 2.2 for the whole app over 1.2 for the whole app - any day of the week.