3 ms·
I honestly think most teams would be better off measuring max on the latency curve. It at least isn’t subject to the many transform errors introduced by most me
by kasey_junk 7y ago
I honestly think most teams would be better off measuring max on the latency curve. It at least isn’t subject to the many transform errors introduced by most metric pipelines & it is easy to explain to people without getting into why 95 vs 99 vs 99.9.
- lonelappde 7y agoMax is always infinity/timeout/highly variable isn't it? And it doesn't tell you when you just made all your requests 1s slower.
- kasey_junk 7y agoIt’s not infinite, it better be timeout but frequently isn’t and yes there is variance to it but usually (handwave) not any more variant than the tail percentile that teams choose. I’ve had the experience multiple times where simply shifting a graph to max shows that timeouts/load shedding is t working, then teams get to the point where they are hitting timeouts way more than they thought. Only after working through those issues do you get to actually improving latency. The upside of simplicity in the number is only overtaken by the downside when you start chasing real time constraints in systems that don’t need it.
- kqr 7y agoIt is highly variable, and in my experience aiming to reduce that variability is one of the most reliable ways to increase total system performance, which is frequently dominated by max latencies. In other words: max latency often indicates some degenerate case (assignable causes) that you want to work out of your system to improve its performance.