It sounds like you're talking about a CPU-heavy workload. In this context often it's an IO-bound workload; the service makes upstream requests to a database and

It sounds like you're talking about a CPU-heavy workload. In this context often it's an IO-bound workload; the service makes upstream requests to a database and other services. Most of the request lifetime is waiting for responses. To get reasonably many requests started and waiting on those responses, either you need to write in async/callback oriented style, or you need a mechanism to make many thousands of naive, blocking routines coexist efficiently.