4 ms·
This is an exceptionally important concept when building web services. I had tacitly understood this but the lead engineer at Blekko really helped me internaliz
by ChuckMcM 10y ago
This is an exceptionally important concept when building web services. I had tacitly understood this but the lead engineer at Blekko really helped me internalize what it meant. My first test question now to someone who asserts they understand the performance of their web based product is what is the 99% quantile latency. Answers start with "huh?" (ok this person hasn't thought to hard about the problem), "the average is ..." (this person measures latency but doesn't really understand that you can't infer very much useful information from the average), "the median is ..." (this person at least gets that latency can be all over the map and so they are trying to identify the midpoint), and "its xx mS". Which is someone who realizes that by starting at the 99th percentile value they can reason about what how the service is working in its entirety.
- nickpsecurity 10y ago"Which is someone who realizes that by starting at the 99th percentile value they can reason about what how the service is working in its entirety." I'm still at "huh?" but things are clicking more. Would you elaborate on that statement?
- barrkel 10y agoI read it as an estimate of worst-case latency, which will be a sum of many different latencies at different layers of the service: application, network, database, disk, memory, synchronization, CPU contention, VM contention - the whole lot.
- vikiomega9 10y agoDoesn't this depend on how "tuned" your subcomponents are? If I had a system that's supposedly heavily optimized, I would simply start seeing the worst-case latency on a higher percentile, 99.9th and so forth. Gil Tene has an excellent talk[1] on this. [1] https://youtu.be/lJ8ydIuPFeU https://youtu.be/lJ8ydIuPFeU
- nkurz 10y agoMaybe, but even this still feels incomplete. Missing from this (and ironically missing from the linked article given that it's part of the blog's title) is the notion of concurrency. Whether a xx mS 99th percentile latency is good or bad depends very strongly on how many parallel requests are in flight. In a single-threaded situation where a slow response holds up all other requests, a few slow responses might be a disaster, but if it just means that an individual user has to wait half a second longer, it might be a non-issue. I agree with the blog author's point that characterizing the entire latency distribution is more valuable than looking at an individual statistic, but it's difficult for me to understand how this blog post can say avoid talking about the equally important degree of parallelism. I'm probably a broken record, but I think Little's Law is one of the most under-utilized concepts in system analysis and design. http://web.mit.edu/sgraves/www/papers/Little's%20Law-Published.pdf http://web.mit.edu/sgraves/www/papers/Little's%20Law-Publish....
- infinite8s 10y agoActually concurrency makes it worse. Say your site has an median latency of < second but a p99 of 10 seconds for any particular request, but your site needs to make 100 requests (not unheard of with modern sites) for any particular page. This means that now almost 99% of your users will be affected by the p99 latency. This is usually a big issue because most systems are designed with median or mean latency in mind, so the p99 is usually orders of magnitude greater.
- devbug 10y agoMathmatically, 1-(1-0.99)^100 = ~1.
- dmit 10y agoNot quite. What you calculated is the probability that visitors will not hit the p99 case on all their requests. In other words, the chance of getting a p99 response on all 100 requests is infinitesimal. The chance of getting at least one p99 response out of a hundred is 1 - 0.99^100 ≈ 63%. https://latencytipoftheday.blogspot.com/2014/06/latencytipoftheday-most-page-loads.html https://latencytipoftheday.blogspot.com/2014/06/latencytipof...
- neeleshs 10y agoIndeed. Error rates, latency distribution,max latencies, percentiles (as a special case of distribution) and throughput give a good picture of how a system is working