> HAProxy is able to keep a connection queue and feed a maximum of (for example) 4 concurrent requests to the backend(s), thus providing back-pressure to incomi

> HAProxy is able to keep a connection queue and feed a maximum of (for example) 4 concurrent requests to the backend(s), thus providing back-pressure to incoming requests. NGINX can do this as well, either with the "max_conns" parameter for upstreams, or (trickier, but perhaps more effective when the upstream is async) in combination with rate limiting: limit_req_zone $server_name zone=root:10m rate=100r/s; limit_req_status 429; location / { limit_req zone=root burst=100 delay=4; }