3 ms·

I may have missed it, but in many scenarios, the reason p95 etc matters isn't that it is 5% of cases ('of course 3g users are slow') but each user may issue man

I may have missed it, but in many scenarios, the reason p95 etc matters isn't that it is 5% of cases ('of course 3g users are slow') but each user may issue many requests. Ex: If serving 20 assets over a session, most users will be hit w a great p50 and bad p95. Trouble shootong at the session-of-indiv-requests level is tough, so being able to zoom in/out is the power of correlation IDs and observability stacks (vs this kind of monitoring view afaict).

bobbiechen 7y ago
And similarly, within a microservice architecture the tail latency is amplified through all its downstream requests or if there is fan-out: Consider a system where each server typically responds in 10ms but with a 99th-percentile latency of one second. If a user request is handled on just one such server, one user request in 100 will be slow (one second). The figure here outlines how service-level latency in this hypothetical scenario is affected by very modest fractions of latency outliers. If a user request must collect responses from 100 such servers in parallel, then 63% of user requests will take more than one second (marked “x” in the figure). "The tail at scale", Jeffrey Dean and Luiz André Barroso: https://dl.acm.org/doi/abs/10.1145/2408776.2408794?download=true https://dl.acm.org/doi/abs/10.1145/2408776.2408794?download=...