This skips over a solution that helps both users, make each request faster. That may seem like an obvious statement but the “solution” they propose is a complic

This skips over a solution that helps both users, make each request faster. That may seem like an obvious statement but the “solution” they propose is a complicated partitioning approach. This presumes you’ve already made each request as fast as reasonable. In my experience that’s rarely true. This also doesn’t get into the real challenge, which is the trade offs for tail latency for median (or more typically 99%) latency. Most of the time the things you do to flatten tail latency increase latency lower in the curve.