"The solution is to queue because this removes the latency" - what? Queueing when your workers aren't at capacity surely adds latency. Agreed that routers need

"The solution is to queue because this removes the latency" - what? Queueing when your workers aren't at capacity surely adds latency. Agreed that routers need to be aware of whether workers are busy. This might be really simple if the dispatcher receives the worker's reply and forwards it, or you could have the worker explicitly signal. Why would you have a single global dispatch? While it is "optimum" if the dispatch were instant and infallible, presumably you'd have random assignment to any of a handful of independent dispatchers, each perhaps with dedicated workers, perhaps not. (also agreed that fault tolerance is tricky if your operations aren't idempotent, and costly in any case)