And similarly, within a microservice architecture the tail latency is amplified through all its downstream requests or if there is fan-out: Consider a syst

And similarly, within a microservice architecture the tail latency is amplified through all its downstream requests or if there is fan-out: Consider a system where each server typically responds in 10ms but with a 99th-percentile latency of one second. If a user request is handled on just one such server, one user request in 100 will be slow (one second). The figure here outlines how service-level latency in this hypothetical scenario is affected by very modest fractions of latency outliers. If a user request must collect responses from 100 such servers in parallel, then 63% of user requests will take more than one second (marked “x” in the figure). "The tail at scale", Jeffrey Dean and Luiz André Barroso: https://dl.acm.org/doi/abs/10.1145/2408776.2408794?download=true https://dl.acm.org/doi/abs/10.1145/2408776.2408794?download=...