6 ms·
Unfortunately HAProxy doesn't buffer requests*, which is necessary for a production deployment of gunicorn. And for anybody using AWS, ALB doesn't buffer reque
by luhn 5y ago
Unfortunately HAProxy doesn't buffer requests*, which is necessary for a production deployment of gunicorn. And for anybody using AWS, ALB doesn't buffer requests either. Because of this I'm actually running both HAProxy and nginx in front of my gunicorn instances—nginx in front for request buffering and HAProxy behind that for queuing.
If anybody is interested, I've packaged both as Docker containers:
HAProxy queuing/load shedding: https://hub.docker.com/r/luhn/spillway https://hub.docker.com/r/luhn/spillway
nginx request buffering: https://hub.docker.com/r/luhn/gunicorn-proxy https://hub.docker.com/r/luhn/gunicorn-proxy
* It does have an http_buffer_request option, but this only buffers the first 8kB (?) of the request.
- Twirrim 5y agoCouldn't Apache httpd just do all of that for you? mod_buffer provides request buffering, and mod_proxy_balancer provides load balancing capabilities.
- luhn 5y agoCan Apache do request queuing?
- Twirrim 5y agoYes, under MPM, you have a listen backlog. If there's a spare worker available to process a request, they'll be picked off the backlog. https://httpd.apache.org/docs/2.4/mod/mpm_common.html#listenbacklog https://httpd.apache.org/docs/2.4/mod/mpm_common.html#listen...
- luhn 5y agoI don't think that's what I'm looking for, that's queuing at the front of the pipe but I need it queuing at the end of the pipe. Apache should be buffering and queuing lots of connections (with a timeout) and sending them single-file in gunicorn. This lays out what I'm trying to achieve: https://aws.amazon.com/builders-library/using-load-shedding-to-avoid-overload/ https://aws.amazon.com/builders-library/using-load-shedding-...