4 ms·

Using nginx as a load balancer

68 points by mickeyben 17y ago

Nothing new, all is in the documentation. I just tried to make the things simpler for noobs as I was few hours ago on this subject.

justinsb 17y ago
Everyone knowledgable about nginx (that I've talked to) says not to use it as a load balancer though. Best practice seems to be to front haproxy with nginx. I've personally encountered some pretty serious issues when a back-end server isn't responding (nginx hanging); the _balancing_ aspect is non-existent, though there's a third party nginx module for it (upstream_fair); but it generally doesn't seem like there's motivation to encourage/support using nginx as a load balancer. It's a pity, because nginx does SSL much better than haproxy does. We're still using nginx but we monitor it _very_ closely.
- piotrSikora 17y ago
  Because of the way in which nginx is implemented, it is simply impossible for nginx to hang because of dead / not responding backend. In such case nginx will send either 502 Bad Gateway or 504 Gateway Time-out error response.
  - justinsb 17y ago
    Are you volunteering to fix the bug if I can reproduce it? That would be very kind of you :-)
    - piotrSikora 17y ago
      If you can reproduce this with recent version (either 0.7.60+ or 0.8.x, not 0.6.x from please-dont-use-me-anymore repository), then I can promise you that I will at least look into this issue. You can mail me off-site with the way to reproduce this (check my profile for e-mail address).
- andrewvc 17y ago
  Agreed wrt to SSL. I use Engine Yard Cloud which uses HAProxy + nginx, which breaks X-Forwarded-For for SSL requests.
  - catch23 17y ago
    they should use stunnel modified to work with x-forward headers instead of nginx. nginx adds little benefit if you're only using it for the ssl part of request handling.
    - andrewvc 17y ago
      Agreed. I asked about that in October, they said: "We still do not have any ETA on when that will be available. That being said I do not think we will have this ready anytime in the next couple of months." I'm generally happy with EY Cloud, but it's still got some rough edges like this that should have been ironed out before they went out of beta.
- mickeyben 17y ago
  Thanks for this comment, I didn't know about it. After your comment I google for more infos and find that : http://affectioncode.wordpress.com/2008/06/28/another-comparison-of-haproxy-and-nginx/ http://affectioncode.wordpress.com/2008/06/28/another-compar... http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer... I'll have to make a try
- jbyers 17y ago
  We use nginx 0.7.x as a load balancer and have no complaints. A pair balance 25M+ requests a day. It's certainly not a full-featured load balancer -- nothing like the visibility and control you get with haproxy or similar -- but for us it's never been a problem. Our backends handle up to 100 concurrent requests and we use straight round-robin across a large pool. In ~2 years, nginx has never hung, mis-balanced requests, or skipped a beat. One node failed once because of human error during a binary upgrade, and I was pissed -- I thought I could keep the thing running forever. Many of the horror stories I used to see on the nginx lists were in cases where each backend could only handle one request at a time, which I think was the genesis of upstream_fair. I suspect this is a big reason for the bad load balancing reputation. What's missing is easily taking servers out of rotation and non-trivial health checks. Both can be done with config file changes, and nginx's graceful restart mechanism is very robust. But there's no side-channel for changing upstreams on the fly. nginx + haproxy is a fine choice if you need more control, and I've seen haproxy on both sides. Haven't used SSL support in haproxy, but it's good in nginx.
  - whyleyc 17y ago
    Out of interest how beefy are the servers you are using to run nginx ? We're looking at it for a deployment and I'm wondering if you had any tips about whether it's a particularly RAM or CPU-hungry beast ?
    - jbyers 17y ago
      We have single-processor 4GB AMD 1216 and 1354s running nginx. Neither one goes above 20% CPU while doing a few hundred req/s. If we were doing a lot more SSL the picture might change, but nginx seems very efficient.
      - whyleyc 17y ago
        Thanks - very helpful to know. One follow-on question - you mention you use 2 machines, is this in a failover capacity or do you use another piece of hardware/software to balance in front if nginx ?
  - 17y ago
theli0nheart 17y ago
Just use HAProxy.
noss 17y ago
While on the topic of nginx, is anyone using nginx to split the groups for performing a/b testing? It seems like a sensible place to randomly place visitors in groups with cookies.
deleted 17y ago
[deleted]

68 points by mickeyben 17y ago

Nothing new, all is in the documentation. I just tried to make the things simpler for noobs as I was few hours ago on this subject.

justinsb 17y ago

Everyone knowledgable about nginx (that I've talked to) says not to use it as a load balancer though. Best practice seems to be to front haproxy with nginx. I've personally encountered some pretty serious issues when a back-end server isn't responding (nginx hanging); the _balancing_ aspect is non-existent, though there's a third party nginx module for it (upstream_fair); but it generally doesn't seem like there's motivation to encourage/support using nginx as a load balancer. It's a pity, because nginx does SSL much better than haproxy does. We're still using nginx but we monitor it _very_ closely.

piotrSikora 17y ago
Because of the way in which nginx is implemented, it is simply impossible for nginx to hang because of dead / not responding backend. In such case nginx will send either 502 Bad Gateway or 504 Gateway Time-out error response.
- justinsb 17y ago
  Are you volunteering to fix the bug if I can reproduce it? That would be very kind of you :-)
  - piotrSikora 17y ago
    If you can reproduce this with recent version (either 0.7.60+ or 0.8.x, not 0.6.x from please-dont-use-me-anymore repository), then I can promise you that I will at least look into this issue. You can mail me off-site with the way to reproduce this (check my profile for e-mail address).
andrewvc 17y ago
Agreed wrt to SSL. I use Engine Yard Cloud which uses HAProxy + nginx, which breaks X-Forwarded-For for SSL requests.
- catch23 17y ago
  they should use stunnel modified to work with x-forward headers instead of nginx. nginx adds little benefit if you're only using it for the ssl part of request handling.
  - andrewvc 17y ago
    Agreed. I asked about that in October, they said: "We still do not have any ETA on when that will be available. That being said I do not think we will have this ready anytime in the next couple of months." I'm generally happy with EY Cloud, but it's still got some rough edges like this that should have been ironed out before they went out of beta.
mickeyben 17y ago
Thanks for this comment, I didn't know about it. After your comment I google for more infos and find that : http://affectioncode.wordpress.com/2008/06/28/another-comparison-of-haproxy-and-nginx/ http://affectioncode.wordpress.com/2008/06/28/another-compar... http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer... I'll have to make a try
jbyers 17y ago
We use nginx 0.7.x as a load balancer and have no complaints. A pair balance 25M+ requests a day. It's certainly not a full-featured load balancer -- nothing like the visibility and control you get with haproxy or similar -- but for us it's never been a problem. Our backends handle up to 100 concurrent requests and we use straight round-robin across a large pool. In ~2 years, nginx has never hung, mis-balanced requests, or skipped a beat. One node failed once because of human error during a binary upgrade, and I was pissed -- I thought I could keep the thing running forever. Many of the horror stories I used to see on the nginx lists were in cases where each backend could only handle one request at a time, which I think was the genesis of upstream_fair. I suspect this is a big reason for the bad load balancing reputation. What's missing is easily taking servers out of rotation and non-trivial health checks. Both can be done with config file changes, and nginx's graceful restart mechanism is very robust. But there's no side-channel for changing upstreams on the fly. nginx + haproxy is a fine choice if you need more control, and I've seen haproxy on both sides. Haven't used SSL support in haproxy, but it's good in nginx.
- whyleyc 17y ago
  Out of interest how beefy are the servers you are using to run nginx ? We're looking at it for a deployment and I'm wondering if you had any tips about whether it's a particularly RAM or CPU-hungry beast ?
  - jbyers 17y ago
    We have single-processor 4GB AMD 1216 and 1354s running nginx. Neither one goes above 20% CPU while doing a few hundred req/s. If we were doing a lot more SSL the picture might change, but nginx seems very efficient.
    - whyleyc 17y ago
      Thanks - very helpful to know. One follow-on question - you mention you use 2 machines, is this in a failover capacity or do you use another piece of hardware/software to balance in front if nginx ?
- 17y ago

theli0nheart 17y ago

Just use HAProxy.

noss 17y ago

While on the topic of nginx, is anyone using nginx to split the groups for performing a/b testing? It seems like a sensible place to randomly place visitors in groups with cookies.

deleted 17y ago

[deleted]