Another approach than capping the queue explicitly is to record time and cap latency by treating the queue as full if the front of the queue is "too old". If th

Another approach than capping the queue explicitly is to record time and cap latency by treating the queue as full if the front of the queue is "too old". If the time to process the queue is easily predictable this is roughly the same, but when it's not it'll make the queue length adapt dynamically depending on maximum acceptable latency. To your latter example, there are load balancers with support for this pattern: serving up a "please wait" page to new users to prevent unacceptable latency for users already using the site. Frankly more sites ought to do that.