11 ms·

Nginx Load Balancing Basics

172 points by jimaek 14y ago

wtracy 14y ago
What does the syntax server unix:/tmp/backend; do? Does that serve from the local filesystem without going through another http daemon?
- jimaek 14y ago
  It was used just as an example that you can use anything as an upstream. Its just a unix socket that can be listened by php-fpm for example.
- dignan 14y ago
  It uses a unix socket. It's also a bit faster than using a TCP socket to proxy (at least in my limited testing), so I prefer it.
adrianpike 14y ago
Another awesome thing - as of 1.3.1/1.2.2, nginx can do least connections load balancing, which is better if your upstream response time isn't very consistent.
ccarpenterg 14y ago
It would be nice to have a library for adding server/s to an upstream on the fly. Something like: nginx addserver upstream-name 127.0.0.1:8025 EDIT: Added upstream name
- deleted 14y ago
  [deleted]
- druiid 14y ago
  That is what puppet or a similar configuration management solution is for (or doing it by hand I suppose). There's not really a realistic scenario I can think of where you'd be wanting a built-in nginx function to service this requirement.
  - conradev 14y ago
    For those interested, the "application_nginx" cookbook for Chef does just this. It can configure upstream servers based on role very easily.
    - druiid 14y ago
      In puppet it's pretty easy as well. The puppet module modification I made to puppet-nginx allows you to do resource collection for a group of upstream servers (and thus add to a group of upstream locations transparently).
- justincormack 14y ago
  If you really want to do it dynamically, you could read the upstreams from Redis with a combination of Nginx modules. Or use Puppet and restart, as the other poster suggested, as it won't break connections in progress.
  - jvoorhis 14y ago
    Storing and reading the configuration from Redis will be slower at runtime, but should scale more easily for a large number of hosts and be much more responsive than using Puppet or Chef. I recommend choosing either solution based on the number of hosts and rate of change.
  - jvoorhis 14y ago
    You can also use a custom DNS resolver with Nginx, and point it to a tool like DNSMasq or PowerDNS [1]. [1] http://wiki.nginx.org/HttpCoreModule#resolver http://wiki.nginx.org/HttpCoreModule#resolver
- mattparlane 14y ago
  See my comment about Hipache -- it pulls config from Redis on the fly.
moepstar 14y ago
That's really helpful & nicely written - shouldn't that be added to the NGINX Wiki in some way (either as a complete how-to-do-this or just as a link)?
- jimaek 14y ago
  Thank you! I do not feel right about adding it myself. But maybe somebody eventually will do it
bigdubs 14y ago
Something that is pretty important, and is missing from this guide, is to make sure you add headers indicating what the original IP address for the requests were (either in x-forwarded-for or x-real-ip or something else common.) Can do this in the root location "/" with: proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; Also good to remember to put another header in for the forwarded protocol (if you're terminating an ssl tunnel at the balancer.)
- tszming 14y ago
  Agreed, and also the host header proxy_set_header Host $host;
- WALoeIII 14y ago
  For backend nginx instances (we use nginx to balance application servers, and nginx right in front of Unicorn on those application servers) use the Real IP module to have logs transparently show the original request IP not the load balancer's IP. http://wiki.nginx.org/HttpRealipModule http://wiki.nginx.org/HttpRealipModule
- elliotanderson 14y ago
  On a security note - your application code should only trust those headers (X-Forwarded-For, X-Real-Ip, etc) for IP lookup if you control the load balancer and strip it from incoming requests. There is nothing to stop a malicious client adding the header themselves and if you rely on IP lookup (i.e. Dev Mode active for 127.0.0.1) for access control you can leave yourself wide open. While I can't find the article at the moment, Stack Overflow accidentally gave admin level access to the site because of this over sight.
  - lorewarden 14y ago
    In my experience if a client adds their own X-Forwarded-For header trying to spoof their IP, nginx simply prepends it to the X-Forwarded-For header like "1.2.3.4, 33.33.33.1", where 1.2.3.4 is the address the client supplied in their spoofing attempt, and 33.33.33.1 is the actual IP address forwarded by nginx. So you can choose to trust only the rightmost one, if there are several entries in the list.
cmer 14y ago
I'm really on the fence between Haproxy or Nginx. I have used Haproxy successfully in the past, but I'm tempted by the simplicity of Nginx, especially now that it supports SPDY. Would like to hear people's thoughts on using Nginx in "real life" for load balancing rather than Haproxy.
- Firehed 14y ago
  I can't compare it to HAProxy, but nginx load balancing was probably the simplest and most reliable part of our web infrastructure, and did exactly what we wanted and needed. Never played around with SPDY, but I liked the various options regarding server weighting, SSL termination, and like you mention the ease of configuration. It wasn't too fancy, but solved a problem and solved it well. Unfortunately we had to switch off of it due to PCI compliance concerns[1], but I'd use it again in a heartbeat. [1] not because there were actual issues, but because other solutions were fully audited out of the box. I'm hardly surprised that we've had more issues with those solutions than we ever had with nginx, including the time when we barely knew how to configure the thing. One of the unavoidable hazards of PCI Level 1 :( We still use it for the actual web requests quite happily.
  - cmer 14y ago
    Gives me confidence in going with nginx for LB. Was the lack of a web ui (like haproxy has) ever a concern? How did you keep track of dead servers behind the LB?
    - Firehed 14y ago
      We didn't have enough servers behind it to really deal with dead servers, to be honest. Seemed to detect a failed server and route around it quickly enough, and we have monitoring per server in place to go in and reboot the thing or whatever. The configs are pretty straightforward, but might get a little nuts if you're dealing with hundreds of servers behind the thing. I don't have to wear a sysadmin hat too frequently (thank god) but when I did it was pretty easy to deal with. Huge fan of the fact that reloading the config would perform a configtest automatically before trying to apply the new settings. I don't know why all software doesn't do this.
chr15 14y ago
Can anyone compare Nginx and HAProxy, and also provide an example of using both in an infrastructure?
- druiid 14y ago
  Here or in an article? We use both in our infrastructure. Generally I would say that if you're proxying web connections and need caching or the ability to do lots of complicated rewriting on the proxy side, use nginx. If you're proxying database, mail or similar... haproxy. If you don't need any caching or similar, either nginx or haproxy depending on your application.
- scottbruin 14y ago
  I'm not terribly experienced with Nginx, but HAProxy was (and is) a load balancer first where Nginx is a server with load balancing abilities (same as Apache can, though it gets less love these days). HAProxy has pretty powerful HTTP support and capabilities, however, so I'm not sure I buy the other argument in this thread. HAProxy also allows you to modify balanced nodes while the server is running, and has fantastic logging once you get used to looking at it.
mattparlane 14y ago
Another alternative is Hipache: https://github.com/dotcloud/hipache https://github.com/dotcloud/hipache It's based on NodeJS, and it's really good. I've been using it in front of three web servers serving around 800 small-to-medium business websites for the last six months and it's been fantastic. It pull configuration data from Redis so you can easily do things like automating deployments etc.
ftwinnovations 14y ago
Does anyone know of a quality basics guide for HAProxy?
- scottbruin 14y ago
  I actually think the basic docs are very good for going over theory. There's a more readable version of them on Google Code: http://code.google.com/p/haproxy-docs/wiki/HAProxy http://code.google.com/p/haproxy-docs/wiki/HAProxy Beyond that, it depends how you're using (HTTP load balancing, TCP only, etc) Got any specific questions? We've been running it in production for over a year.
pootch 14y ago
You dont load balance with nginx you use HAProxy.

172 points by jimaek 14y ago

What does the syntax server unix:/tmp/backend; do? Does that serve from the local filesystem without going through another http daemon?

jimaek 14y ago
It was used just as an example that you can use anything as an upstream. Its just a unix socket that can be listened by php-fpm for example.
dignan 14y ago
It uses a unix socket. It's also a bit faster than using a TCP socket to proxy (at least in my limited testing), so I prefer it.

adrianpike 14y ago

Another awesome thing - as of 1.3.1/1.2.2, nginx can do least connections load balancing, which is better if your upstream response time isn't very consistent.

ccarpenterg 14y ago

It would be nice to have a library for adding server/s to an upstream on the fly. Something like: nginx addserver upstream-name 127.0.0.1:8025 EDIT: Added upstream name

deleted 14y ago
[deleted]
druiid 14y ago
That is what puppet or a similar configuration management solution is for (or doing it by hand I suppose). There's not really a realistic scenario I can think of where you'd be wanting a built-in nginx function to service this requirement.
- conradev 14y ago
  For those interested, the "application_nginx" cookbook for Chef does just this. It can configure upstream servers based on role very easily.
  - druiid 14y ago
    In puppet it's pretty easy as well. The puppet module modification I made to puppet-nginx allows you to do resource collection for a group of upstream servers (and thus add to a group of upstream locations transparently).
justincormack 14y ago
If you really want to do it dynamically, you could read the upstreams from Redis with a combination of Nginx modules. Or use Puppet and restart, as the other poster suggested, as it won't break connections in progress.
- jvoorhis 14y ago
  Storing and reading the configuration from Redis will be slower at runtime, but should scale more easily for a large number of hosts and be much more responsive than using Puppet or Chef. I recommend choosing either solution based on the number of hosts and rate of change.
- jvoorhis 14y ago
  You can also use a custom DNS resolver with Nginx, and point it to a tool like DNSMasq or PowerDNS [1]. [1] http://wiki.nginx.org/HttpCoreModule#resolver http://wiki.nginx.org/HttpCoreModule#resolver
mattparlane 14y ago
See my comment about Hipache -- it pulls config from Redis on the fly.

moepstar 14y ago

That's really helpful & nicely written - shouldn't that be added to the NGINX Wiki in some way (either as a complete how-to-do-this or just as a link)?

jimaek 14y ago
Thank you! I do not feel right about adding it myself. But maybe somebody eventually will do it

bigdubs 14y ago

Something that is pretty important, and is missing from this guide, is to make sure you add headers indicating what the original IP address for the requests were (either in x-forwarded-for or x-real-ip or something else common.) Can do this in the root location "/" with: proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; Also good to remember to put another header in for the forwarded protocol (if you're terminating an ssl tunnel at the balancer.)

tszming 14y ago
Agreed, and also the host header proxy_set_header Host $host;
WALoeIII 14y ago
For backend nginx instances (we use nginx to balance application servers, and nginx right in front of Unicorn on those application servers) use the Real IP module to have logs transparently show the original request IP not the load balancer's IP. http://wiki.nginx.org/HttpRealipModule http://wiki.nginx.org/HttpRealipModule
elliotanderson 14y ago
On a security note - your application code should only trust those headers (X-Forwarded-For, X-Real-Ip, etc) for IP lookup if you control the load balancer and strip it from incoming requests. There is nothing to stop a malicious client adding the header themselves and if you rely on IP lookup (i.e. Dev Mode active for 127.0.0.1) for access control you can leave yourself wide open. While I can't find the article at the moment, Stack Overflow accidentally gave admin level access to the site because of this over sight.
- lorewarden 14y ago
  In my experience if a client adds their own X-Forwarded-For header trying to spoof their IP, nginx simply prepends it to the X-Forwarded-For header like "1.2.3.4, 33.33.33.1", where 1.2.3.4 is the address the client supplied in their spoofing attempt, and 33.33.33.1 is the actual IP address forwarded by nginx. So you can choose to trust only the rightmost one, if there are several entries in the list.

cmer 14y ago

I'm really on the fence between Haproxy or Nginx. I have used Haproxy successfully in the past, but I'm tempted by the simplicity of Nginx, especially now that it supports SPDY. Would like to hear people's thoughts on using Nginx in "real life" for load balancing rather than Haproxy.

Firehed 14y ago
I can't compare it to HAProxy, but nginx load balancing was probably the simplest and most reliable part of our web infrastructure, and did exactly what we wanted and needed. Never played around with SPDY, but I liked the various options regarding server weighting, SSL termination, and like you mention the ease of configuration. It wasn't too fancy, but solved a problem and solved it well. Unfortunately we had to switch off of it due to PCI compliance concerns[1], but I'd use it again in a heartbeat. [1] not because there were actual issues, but because other solutions were fully audited out of the box. I'm hardly surprised that we've had more issues with those solutions than we ever had with nginx, including the time when we barely knew how to configure the thing. One of the unavoidable hazards of PCI Level 1 :( We still use it for the actual web requests quite happily.
- cmer 14y ago
  Gives me confidence in going with nginx for LB. Was the lack of a web ui (like haproxy has) ever a concern? How did you keep track of dead servers behind the LB?
  - Firehed 14y ago
    We didn't have enough servers behind it to really deal with dead servers, to be honest. Seemed to detect a failed server and route around it quickly enough, and we have monitoring per server in place to go in and reboot the thing or whatever. The configs are pretty straightforward, but might get a little nuts if you're dealing with hundreds of servers behind the thing. I don't have to wear a sysadmin hat too frequently (thank god) but when I did it was pretty easy to deal with. Huge fan of the fact that reloading the config would perform a configtest automatically before trying to apply the new settings. I don't know why all software doesn't do this.

chr15 14y ago

Can anyone compare Nginx and HAProxy, and also provide an example of using both in an infrastructure?

druiid 14y ago
Here or in an article? We use both in our infrastructure. Generally I would say that if you're proxying web connections and need caching or the ability to do lots of complicated rewriting on the proxy side, use nginx. If you're proxying database, mail or similar... haproxy. If you don't need any caching or similar, either nginx or haproxy depending on your application.
scottbruin 14y ago
I'm not terribly experienced with Nginx, but HAProxy was (and is) a load balancer first where Nginx is a server with load balancing abilities (same as Apache can, though it gets less love these days). HAProxy has pretty powerful HTTP support and capabilities, however, so I'm not sure I buy the other argument in this thread. HAProxy also allows you to modify balanced nodes while the server is running, and has fantastic logging once you get used to looking at it.

mattparlane 14y ago

Another alternative is Hipache: https://github.com/dotcloud/hipache https://github.com/dotcloud/hipache It's based on NodeJS, and it's really good. I've been using it in front of three web servers serving around 800 small-to-medium business websites for the last six months and it's been fantastic. It pull configuration data from Redis so you can easily do things like automating deployments etc.

ftwinnovations 14y ago

Does anyone know of a quality basics guide for HAProxy?

scottbruin 14y ago
I actually think the basic docs are very good for going over theory. There's a more readable version of them on Google Code: http://code.google.com/p/haproxy-docs/wiki/HAProxy http://code.google.com/p/haproxy-docs/wiki/HAProxy Beyond that, it depends how you're using (HTTP load balancing, TCP only, etc) Got any specific questions? We've been running it in production for over a year.

pootch 14y ago

You dont load balance with nginx you use HAProxy.