4 ms·

For what it's worth, Jeff Dean of Google has famously emphasized 95th and 99th percentile performance in preference to average or median performance over the ye

by kmavm 16y ago

For what it's worth, Jeff Dean of Google has famously emphasized 95th and 99th percentile performance in preference to average or median performance over the years. The realization that edge cases more powerfully determine user perception of performance than average cases is a deep one, and is not original to Facebook or Google; Dean and colleagues were making some of the same points when they were at DEC's CRC pre-Google.

davnola 16y ago
Do you know of any research supporting that? It sounds plausible to me but a quick search turned up nothing.
- jonnathanson 16y ago
  There is some social psychology and operations research around this -- not necessarily as it relates to human/machine interactions, but certainly as it relates to such quasi-analogous situations as call centers, lines, speed vs. variability of service at McDonald's, and so forth. The premise being practiced by Facebook takes a lot of cues from behavioral science data that is generalizable to many different domains. McDonald's, to give a famous example, proved analytically a few years back that -- beyond a certain threshold of commonly expected service times -- their customers would rather get semi-slow service on a consistent basis than highly variable service. In a perfect world, of course, average wait times are as short as possible in addition to a minimization of variance. But when you're at the level of acceptability, there are diminishing returns on speed increases and increasing returns on reduced variance.
  - lenn0x 16y ago
    Do you have a link to this research or study from McDonalds?
    - jonnathanson 16y ago
      Alas, no, it was from an analytics consulting engagement a buddy of mine did a few years ago. But I am reasonably sure the outcome of the data is public knowledge and available somewhere. I wish I knew where.
- kmavm 16y ago
  Alas, my understanding of this point is entirely folkloric. Self-experimentation, and working on improving real interactive systems, agrees with it. Please update if you find anything more comprehensive.

by kmavm 16y ago

davnola 16y ago

Do you know of any research supporting that? It sounds plausible to me but a quick search turned up nothing.

jonnathanson 16y ago
There is some social psychology and operations research around this -- not necessarily as it relates to human/machine interactions, but certainly as it relates to such quasi-analogous situations as call centers, lines, speed vs. variability of service at McDonald's, and so forth. The premise being practiced by Facebook takes a lot of cues from behavioral science data that is generalizable to many different domains. McDonald's, to give a famous example, proved analytically a few years back that -- beyond a certain threshold of commonly expected service times -- their customers would rather get semi-slow service on a consistent basis than highly variable service. In a perfect world, of course, average wait times are as short as possible in addition to a minimization of variance. But when you're at the level of acceptability, there are diminishing returns on speed increases and increasing returns on reduced variance.
- lenn0x 16y ago
  Do you have a link to this research or study from McDonalds?
  - jonnathanson 16y ago
    Alas, no, it was from an analytics consulting engagement a buddy of mine did a few years ago. But I am reasonably sure the outcome of the data is public knowledge and available somewhere. I wish I knew where.
kmavm 16y ago
Alas, my understanding of this point is entirely folkloric. Self-experimentation, and working on improving real interactive systems, agrees with it. Please update if you find anything more comprehensive.