3 ms·
I have found in many distributed systems the latency of top level requests follows a log-normal distribution, which sounds like what you are describing. But tun
by spullara 9y ago
I have found in many distributed systems the latency of top level requests follows a log-normal distribution, which sounds like what you are describing. But tuning when you launch backup requests you can trade off overhead for reduced tail latencies. After experimenting with it at Twitter I then found a Jeff Dean paper from Google describing it as well. Haven't been able to find that paper again though. Here is his presentation where he discusses backup requests:
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44875.pdf https://static.googleusercontent.com/media/research.google.c...
- hnaccy 9y ago>Non-intuitive: remove capacity under load to improve latency (?!) Does he elaborate on this?
- kolpa 9y agoIt could mean * one of your servers is unhealthy, remove it from the pool. * reject some requests to make the rest faster.
- spullara 9y agoThe most common case I can think of is when you have local caches and randomly distribute traffic amongst your servers. The hit rate on the cache improves as you reduce the number of machines. If the cache expires before the same machine is hit again, you get no benefit from the cache at all. Even in the case where you are hitting the cache you are doing more original requests as you add more servers.
- swsieber 9y agoRemoving capacity probably doesn't include removing capacity at all levels of the stack - perhaps it only means reducing the number of frontends, not db servers. That'd reduce load on the db server, allowing it to serve faster.