Gil Tene has an entire talk about this point called "How NOT to measure latency" (https://www.youtube.com/watch?v=lJ8ydIuPFeU https://www.youtube.com/watch?v=lJ

Gil Tene has an entire talk about this point called "How NOT to measure latency" (https://www.youtube.com/watch?v=lJ8ydIuPFeU https://www.youtube.com/watch?v=lJ8ydIuPFeU). In many circumstances, a web service (either server or client) has large numbers of dependencies, so a user only has to hit 95th-percentile latency for one dependency to have their overall latency significantly hurt. In other words, users may encounter the "rare" bad case on a majority of requests. Thus, even higher percentiles should be tightly controlled in order to truly keep a web service's overall latency under control.