Overload is as much about establishing a baseline. Even under "perfect" routing, the problem is that you need to know what load your system can cope with. You

Overload is as much about establishing a baseline. Even under "perfect" routing, the problem is that you need to know what load your system can cope with. You need queueing to a certain extent since a queue will absorb spikes in the load and smooth out the sudden arrival of a large number of requests in a short time. But excessive queueing leads to latency. One of the "intelligent" routing problems is we have to consider something more than queue length. We need to know, prior to running, how expensive a query is going to be. Otherwise we may end up queueing someone after the expensive query while a neighboring dyno could serve the request quickly. But you generally can't do this. Under load, such a system would "amplify" expensive queries: all queries after the expensive one in queue will be expensive as well. This is why I would advise people to move to a model where concurrency happens "in the process" as well. It is actually easier to dequeue the work off the routing layer as fast as possible and then interleave expensive and cheap work in the dyno.