4 ms·

Only looking at the 95th percentile is pretty disappointing. Enough page loads happen that higher percentile experiences aren't exactly rare.

by xxpor 3y ago

charcircuit 3y ago
It's not like real users aren't hitting loading times in those higher percentiles. They really should be showing p100.
- persnickety 3y ago
  p100 would include those where the uplink is so slow that the page takes minutes to render completely. Those users are useful to see, but the browser is much less of a bottleneck for them, and the data should be broken out into a separate bin.
  - sfink 3y ago
    It would also include users who happened to close their laptop lids and suspend for an hour, right during pageload.
    - astrange 3y ago
      That's if it's measured by wallclock time, but they should be using a monotonic timer that ignores sleep and clock changes.
sfink 3y ago
You mean only displaying and discussing the 95th percentile in a blog post? There's no reason to assume that what is brought up in a blog post is going to match what engineers are looking at. In this case, I'm a developer at Mozilla, and I would say that I agree that it's worthwhile to look at the 99th percentile as well. And the median (50th percentile). And other platforms. And segregate it by website, but we don't collect that, or by country, but although I think we might be collecting that we avoid correlating too many things before discarding the non-aggregate data. There are too many German users to worry about identifying one by knowing they're German, but there's aren't that many German users with >100 tabs running on Windows 7 on slower hardware, etc. I wouldn't want to pile up any more data than necessary in a blog post, though. The point would get buried. Man Bites Dog Whose Litter Mate Once Skipped A Veterinary Visit Because Owner Was Out Of Town At A Wedding Between Two People Whose Names Start With D (Yes, we do consider many different percentiles when making decisions. We kind of have to come up with arguments for what matters, given a change we made or are contemplating. Some things improve the 50th and regress the 95th, for example, and that's a useful clue. Telemetry tracks half a dozen different percentiles.)
- xxpor 3y ago
  I'm sure internally folks are looking at more stuff, but blog post wise, it makes it look like you're intentionally omitting it (see https://youtu.be/lJ8ydIuPFeU?t=1239 https://youtu.be/lJ8ydIuPFeU?t=1239)
- teo_zero 3y ago
  But what is the reason behind the choice to only show the 95th percentile and not the mean or median, that undoubtedly are more understandable by a vast audience?

Only looking at the 95th percentile is pretty disappointing. Enough page loads happen that higher percentile experiences aren't exactly rare.

by xxpor 3y ago

Only looking at the 95th percentile is pretty disappointing. Enough page loads happen that higher percentile experiences aren't exactly rare.

charcircuit 3y ago

It's not like real users aren't hitting loading times in those higher percentiles. They really should be showing p100.

persnickety 3y ago
p100 would include those where the uplink is so slow that the page takes minutes to render completely. Those users are useful to see, but the browser is much less of a bottleneck for them, and the data should be broken out into a separate bin.
- sfink 3y ago
  It would also include users who happened to close their laptop lids and suspend for an hour, right during pageload.
  - astrange 3y ago
    That's if it's measured by wallclock time, but they should be using a monotonic timer that ignores sleep and clock changes.

sfink 3y ago

You mean only displaying and discussing the 95th percentile in a blog post? There's no reason to assume that what is brought up in a blog post is going to match what engineers are looking at. In this case, I'm a developer at Mozilla, and I would say that I agree that it's worthwhile to look at the 99th percentile as well. And the median (50th percentile). And other platforms. And segregate it by website, but we don't collect that, or by country, but although I think we might be collecting that we avoid correlating too many things before discarding the non-aggregate data. There are too many German users to worry about identifying one by knowing they're German, but there's aren't that many German users with >100 tabs running on Windows 7 on slower hardware, etc. I wouldn't want to pile up any more data than necessary in a blog post, though. The point would get buried. Man Bites Dog Whose Litter Mate Once Skipped A Veterinary Visit Because Owner Was Out Of Town At A Wedding Between Two People Whose Names Start With D (Yes, we do consider many different percentiles when making decisions. We kind of have to come up with arguments for what matters, given a change we made or are contemplating. Some things improve the 50th and regress the 95th, for example, and that's a useful clue. Telemetry tracks half a dozen different percentiles.)

xxpor 3y ago
I'm sure internally folks are looking at more stuff, but blog post wise, it makes it look like you're intentionally omitting it (see https://youtu.be/lJ8ydIuPFeU?t=1239 https://youtu.be/lJ8ydIuPFeU?t=1239)
teo_zero 3y ago
But what is the reason behind the choice to only show the 95th percentile and not the mean or median, that undoubtedly are more understandable by a vast audience?