3 ms·
I know its not main point of the article, but the "Bigger Holding Space" results are surprising to me. The idle time barely changes at all, and even the respons
by zokier 4y ago
I know its not main point of the article, but the "Bigger Holding Space" results are surprising to me. The idle time barely changes at all, and even the response time change is very moderate (<10% increase), while rejection rate drops three orders of magnitude(!). I suppose this is somehow an artifact on the particular distributions chosen in the example? Also intuitively tail-end latency (like p99) would suffer more when you add queues, I guess figuring that out analytically is more difficult..
- kqr 4y agoMost queues are nearly empty most of the time. (Consequently, most queues have oversized limits.) The only time I've experienced queues grow beyond "nearly empty" are either 1. When there is high request-to-request variability. E.g. one request triggers a garbage collection or something pathological in the database or another shared resource, that blocks all other requests from processing. This can quickly build up a serious queue. 2. When the system is designed too weak for peak loads (e.g. in the afternoon for online stores) with the expectation that once the peak subsides, things will be quiet enough to catch up with the overload. This model does not deal with case 1 (since the exponential distribution has comparatively little variance) and is supposed to represent a system where 2 isn't a factor either. ---- In terms of tail latency: I agree! That might end up being a future article because it would have been too much for this one.