5 ms·
Consul is great, but couldn't you just simply use NGINX Plus (paid) and utilize the upstream_conf[1], state[2] and healthcheck[3] premium features? Sorry for t
by nodesocket 10y ago
Consul is great, but couldn't you just simply use NGINX Plus (paid) and utilize the upstream_conf[1], state[2] and healthcheck[3] premium features?
Sorry for the long code block below, but it is a full working example that has automatic health checks (adds and removes backends automatically) and a dynamic backend configuration that is persisted to a state file.
http {
proxy_next_upstream error timeout http_502 http_504;
match statusok {
status 200;
body ~ "200 OK";
}
upstream backend {
state /etc/nginx/backend-state/backends;
zone backend 256k;
keepalive 20;
}
}
server {
# dynamic upstream
location = /upstream_conf {
upstream_conf;
allow 127.0.0.1;
deny all;
}
location / {
proxy_connect_timeout 15s;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://backend;
}
# health check
location /internal-health-check {
internal;
proxy_connect_timeout 15s;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://backend;
health_check interval=3s fails=3 passes=5 uri=/health match=statusok;
}
}
[1] - https://nginx.org/en/docs/http/ngx_http_upstream_conf_module.html https://nginx.org/en/docs/http/ngx_http_upstream_conf_module...
[2] - http://nginx.org/en/docs/http/ngx_http_upstream_module.html#state http://nginx.org/en/docs/http/ngx_http_upstream_module.html#...
[3] - http://nginx.org/en/docs/http/ngx_http_upstream_module.html#health_check http://nginx.org/en/docs/http/ngx_http_upstream_module.html#...
- josegonzalez 10y agoYou still need to populate those back ends somehow, and a simple http healthchecks doesn't show the entire picture around whether or not you should forward requests to a service.
- nodesocket 10y agoYou populate backends using the dynamic upstream_conf which saves state to a state file (see [1]). # add a backend curl http://127.0.0.1/upstream_conf?add=&upstream=backend&server=10.0.0.1:8000 # remove a backend curl http://127.0.0.1/upstream_conf?remove=&upstream=backend&id=1 [1] - https://www.nginx.com/blog/dynamic-reconfiguration-with-nginx-plus/ https://www.nginx.com/blog/dynamic-reconfiguration-with-ngin...
- josegonzalez 10y agoSo now you have another bit in your deploy pipeline that needs to construct curl requests vs declaring your services against a registrar. Can you share these registrations across servers? I'd hate to have to poll for all my web servers to register a new API process, and I'm at the small scale (~200 servers).
- nodesocket 10y agoPut your NGINX state file on a shared disk (NFS) between all load balancers (something like AWS Cloud File Storage).
- josegonzalez 10y agoAt my company, every server has its own internal load balancer (previously haproxy, now nginx). While definitely possible, I don't think sharing a single file across hundreds of boxes is a "best practice". That said, this certainly would work for companies with a ton of money to burn (nginx+ licenses aren't cheap) or who are at smaller scales. If it works, ship it!
- nodesocket 10y agoYou really shouldn't need more than a handful of load balancers (may one or two in a few different availability zones for HA). You can even pay Amazon and NGINX hourly using the NGINX Plus AMI[1]. [1] - https://aws.amazon.com/marketplace/pp/B00A04GAG4/ https://aws.amazon.com/marketplace/pp/B00A04GAG4/
- snug 10y agoIt would actually be just as useful to use consul with the upstream resolv available on nginx (paid) A demo of this can be found here[0] 0: https://github.com/nginxinc/NGINX-Demos/tree/master/consul-dns-srv-demo https://github.com/nginxinc/NGINX-Demos/tree/master/consul-d...