3 ms·
With fewer servers, requests would not take 33ms, they would, on average, take longer. As load increases, the probability that requests are queued while CPUs ar
by barrkel 12y ago
With fewer servers, requests would not take 33ms, they would, on average, take longer. As load increases, the probability that requests are queued while CPUs are busy with other requests increases. The number of requests being processed and not blocked on I/O can't exceed the physical parallelism available.
You don't need to be CPU-bound before this starts happening because requests don't arrive in evenly spaced intervals. Decreasing CPU load, even on a moderately loaded, I/O bound system, can still reduce latency.