What you're describing is queueing, but the solution you're really looking for is back pressure. Any queue will eventually fill and when you fill it, things fal

What you're describing is queueing, but the solution you're really looking for is back pressure. Any queue will eventually fill and when you fill it, things fall out. So rather than just dropping requests or letting the requests swamp your box, what you want to do is intelligently reject requests. On the web, this means sending 503 responses. In a more intelligent client-server model, this might be the client doing an exponential back-off retry with jitter. A plain-old retry can swamp the server and just kill everything. An exponential back-off retry will allow the backend to achieve a level of service that isn't fast, but also isn't unavailable. Hopefully in this case the haproxy is used intelligently and simply returns cached responses to GET requests without query strings. You don't want to hold onto connections or pass them off to the dynamic layer behind the proxy if you're overloaded. You can also respond to the client with a page with an auto-refresh, implementing an exponential back-off retry.