4 ms·

I spent a long time fighting this battle with nginx. I eventually came to the conclusion that while it was technically possible, it just wasn't worth the troub

by bmurphy1976 11y ago

I spent a long time fighting this battle with nginx. I eventually came to the conclusion that while it was technically possible, it just wasn't worth the trouble (and yes, I tried various alternatives such as Tengine and OpenResty). What I settled on was the following: Ngninx for SSL termination in front of an haproxy load balancer. I wrote a simple Python script (92 lines of code) that spits out a new haproxy configuration and gracefully reloads haproxy whenever the DNS changes. That has (after fixing a bug or two in my code) been FAR more reliable than any other solution to this problem we've tried. Bonus: haproxy is a superior load balancer with better runtime metrics and health checking options than nginx.

culo 11y ago
did you look at Kong? https://github.com/mashape/kong https://github.com/mashape/kong
adamonduty 11y ago
What part of the battle couldn't you solve? Doesn't the author's setting of the resolver directive work for you, or is there something I'm missing? Also, the author doesn't mention it, but you can also set a custom cache timeout with the valid=Xs option. http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver http://nginx.org/en/docs/http/ngx_http_core_module.html#reso...
- bmurphy1976 11y ago
  It was not that reliable for us. We still had unexplained occurrences of IP caching and it's easy to fat finger the configuration and lose the resolving functionality. Anyway, as I mentioned haproxy has vastly superior load balancing capabilities. Nginx still has a lot of work to do to match haproxy in this area.
thu 11y ago
Any reason you don't use HAProxy for SSL termination too?
- bmurphy1976 11y ago
  There are a lot of reasons. Simpler configuration, mutual SSL, URL rewriting and the like. That stuff would really suck if we only used haproxy.
  - STRML 11y ago
    HAProxy 1.5 does a pretty great job with that stuff now. I agree that the configuration syntax can be difficult but it is very powerful. There are some limitations of course, and I can see why you might want to use nginx for some routing. One of the more bizarre tricks I've had to use for more complex redirects (like non-www to www while properly injecting HSTS headers) involved sending the request to a backend that sent to a single frontend via a local port [1]. Hopefully that kludge will be fixed in 1.6. 1. http://blog.haproxy.com/2015/06/09/haproxy-and-http-strict-transport-security-hsts-header-in-http-redirects/ http://blog.haproxy.com/2015/06/09/haproxy-and-http-strict-t...
    - bmurphy1976 11y ago
      I know its capable, but you have to understand we are working in a team environment where the expertise is primarily with nginx. It's also quite a lot easier to introduce nginx to a dev who has never used it before. Not every decision can or should be made solely based on technical capabilities. Bizarre tricks are not something I am a fan of deploying to production and using as the foundation for our service. That is hardly a compelling argument. ;)

culo 11y ago

did you look at Kong? https://github.com/mashape/kong https://github.com/mashape/kong

adamonduty 11y ago

What part of the battle couldn't you solve? Doesn't the author's setting of the resolver directive work for you, or is there something I'm missing? Also, the author doesn't mention it, but you can also set a custom cache timeout with the valid=Xs option. http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver http://nginx.org/en/docs/http/ngx_http_core_module.html#reso...

bmurphy1976 11y ago
It was not that reliable for us. We still had unexplained occurrences of IP caching and it's easy to fat finger the configuration and lose the resolving functionality. Anyway, as I mentioned haproxy has vastly superior load balancing capabilities. Nginx still has a lot of work to do to match haproxy in this area.

thu 11y ago

Any reason you don't use HAProxy for SSL termination too?

bmurphy1976 11y ago
There are a lot of reasons. Simpler configuration, mutual SSL, URL rewriting and the like. That stuff would really suck if we only used haproxy.
- STRML 11y ago
  HAProxy 1.5 does a pretty great job with that stuff now. I agree that the configuration syntax can be difficult but it is very powerful. There are some limitations of course, and I can see why you might want to use nginx for some routing. One of the more bizarre tricks I've had to use for more complex redirects (like non-www to www while properly injecting HSTS headers) involved sending the request to a backend that sent to a single frontend via a local port [1]. Hopefully that kludge will be fixed in 1.6. 1. http://blog.haproxy.com/2015/06/09/haproxy-and-http-strict-transport-security-hsts-header-in-http-redirects/ http://blog.haproxy.com/2015/06/09/haproxy-and-http-strict-t...
  - bmurphy1976 11y ago
    I know its capable, but you have to understand we are working in a team environment where the expertise is primarily with nginx. It's also quite a lot easier to introduce nginx to a dev who has never used it before. Not every decision can or should be made solely based on technical capabilities. Bizarre tricks are not something I am a fan of deploying to production and using as the foundation for our service. That is hardly a compelling argument. ;)