James, if you are using supervisord for managing backends, you could take servers out of rotation with help of (shamelessly advertised) ngx_supervisord+upstream

James, if you are using supervisord for managing backends, you could take servers out of rotation with help of (shamelessly advertised) ngx_supervisord+upstream_fair combo. Side-channel for changing upstreams on the fly (with no entires in nginx configuration) is on the roadmap for ngx_supervisord-2.0, but sadly I don't see this happening in the foreseeable future... Also, could you explain what do you mean by non-trivial health checks?