6 ms·

SSL session caching in nginx

76 points by brkcmd 15y ago

tobylane 15y ago
I had to check several times I hadn't accidentally hit zoom a few times. Sounds useful, but how many visitors do you need to have for this to be worth doing?
- WALoeIII 15y ago
  The number of visitors you have is irrelevant. If you have any visitors you are doing them a huge disservice if you do not have SSL session caching. You would use external SSL caching like this if you have more than one SSL termination point (typically a webserver like nginx/Apache) behind a load-balancer.
thecoffman 15y ago
Very cool post - not only is it informative, but you've taken the "oh-so-rare" extra step of actually coding up a solution to what you're talking about - rather than taking the easier approach and just telling others what they're doing wrong but presenting no practical alternatives. Kudos!
nwmcsween 15y ago
You'll usually run out of entropy before cpu usage becomes relevant with SSL processing, I've seen old versions of apache hang with little or no entropy to process SSL connections. I recommend some sort of RNG or a poor mans software version such as http://www.issihosts.com/haveged/ http://www.issihosts.com/haveged/
- epi0Bauqu 15y ago
  This comment really opened my eyes -- thank you! I am still a bit confused though. I think I was running out of entropy, but it seems nginx should use /dev/urandom, which supposedly doesn't block -- just becomes less random when entropy runs out. So is nginx set up to block when entropy is depleted and that is why this happens?
mentat 15y ago
Crypto hardware acceleration is commodity. I believe that Broadcom sold the chips for <$5 over 5 years ago. To say they don't scale horizontally doesn't make sense either. If they're integrated with PCIe then you can get a lot of crypto processing in a single chasis. (Disclosure: Used to make crypto accelerators and load balancers) There surely are reasons not to integrate at the load balancer, but they're not because the load balancer will melt down.
sugarcode 15y ago
One downside of this approach (without some funky iptables/networking-fu) is that you loose the source IP from the original request. Adding headers like X-Forwarded-For only works after the request has been decrypted, so all the traffic will appear to source from the load balancer, which can present its own issues. IMO (and I believe Google agrees - http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html http://www.imperialviolet.org/2010/06/25/overclocking-ssl.ht...) the advantages of terminating SSL at the load balancer outweigh the horizontal scalability of this approach, at least in most cases.
- X-Istence 15y ago
  Transparent load balancers exist and don't have the issues you are talking about.
  - sugarcode 15y ago
    Hardware or software? There's some hacks with TPROXY/HAproxy I've seen that would do the transparent proxy but the setup seems like more trouble than it's worth.
    - X-Istence 15y ago
      OpenBSD PF for example can do round-robin load balancing.
    - ominous_prime 15y ago
      IPVS is built-in to the Linux kernel, and HA projects like keepalived have ipvsadm integration. Tproxy works fine, and has been in the kernel since 2.6.30. In most load-balancing cases, losing the remote IP address isn't that big a deal (you have to deal with NAT too), and a full proxy like haproxy has it's benefits.
thomas-st 15y ago
Why does session affinity not solve the problem of session caching? The author says it's "a whole other world of pain and suffering" but doesn't explain why.
- jadc 15y ago
  Seconded. It would be great if the author could expand on this point.
  - jpetazzo 15y ago
    I guess that it's not easy to implement session affinity on the SSL level. You cannot access information like cookies or other HTTP headers (since they are inside the SSL payload that you're trying to handle). So you have to use the little information you have: source IP address and port, and session data. The idea would be to make sure that a given client is always sent to the same SSL handler. We could imagine having two layers of load balancers: - first layer would use source IP address and/or session data to determine to which server of the second layer the connection should be forwarded; - second layer would receive the connection and to the proper SSL handling. I believe that this would work, but it seems that it would require a custom "half-implementation" of SSL on the first layer of load balancers. I don't know if there is any provision for that in OpenSSL or GNUTLS. Also, since there are already hooks to do session caching in most SSL-enabled servers, using those hooks to plug in a memcached backend seems to be less "disruptive" (read "easier to understand, implement and debug").
    - ominous_prime 15y ago
      You can balance ssl traffic at a TCP level using the client's IP to set affinity. The only real catch is if you have a large base of clients behind a single NAT, but in most cases traffic will balance out pretty well. Sometimes though, you don't want affinity at all. If you don't care what backend server takes the request, you can balance the load more efficiently, and more easily rotate servers in and out of service.
- ominous_prime 15y ago
mmaunder 15y ago
Matt this is awesome. Nginx is becoming the standard for front end load balancing for many high traffic sites and this helps. We only have one nginx front end load balancer (even though we do over 4000 http req/s) but we'll be migrating to a cluster soon so I'll give this a whirl.

76 points by brkcmd 15y ago

I had to check several times I hadn't accidentally hit zoom a few times. Sounds useful, but how many visitors do you need to have for this to be worth doing?

WALoeIII 15y ago
The number of visitors you have is irrelevant. If you have any visitors you are doing them a huge disservice if you do not have SSL session caching. You would use external SSL caching like this if you have more than one SSL termination point (typically a webserver like nginx/Apache) behind a load-balancer.

thecoffman 15y ago

Very cool post - not only is it informative, but you've taken the "oh-so-rare" extra step of actually coding up a solution to what you're talking about - rather than taking the easier approach and just telling others what they're doing wrong but presenting no practical alternatives. Kudos!

nwmcsween 15y ago

You'll usually run out of entropy before cpu usage becomes relevant with SSL processing, I've seen old versions of apache hang with little or no entropy to process SSL connections. I recommend some sort of RNG or a poor mans software version such as http://www.issihosts.com/haveged/ http://www.issihosts.com/haveged/

epi0Bauqu 15y ago
This comment really opened my eyes -- thank you! I am still a bit confused though. I think I was running out of entropy, but it seems nginx should use /dev/urandom, which supposedly doesn't block -- just becomes less random when entropy runs out. So is nginx set up to block when entropy is depleted and that is why this happens?

mentat 15y ago

Crypto hardware acceleration is commodity. I believe that Broadcom sold the chips for <$5 over 5 years ago. To say they don't scale horizontally doesn't make sense either. If they're integrated with PCIe then you can get a lot of crypto processing in a single chasis. (Disclosure: Used to make crypto accelerators and load balancers) There surely are reasons not to integrate at the load balancer, but they're not because the load balancer will melt down.

sugarcode 15y ago

One downside of this approach (without some funky iptables/networking-fu) is that you loose the source IP from the original request. Adding headers like X-Forwarded-For only works after the request has been decrypted, so all the traffic will appear to source from the load balancer, which can present its own issues. IMO (and I believe Google agrees - http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html http://www.imperialviolet.org/2010/06/25/overclocking-ssl.ht...) the advantages of terminating SSL at the load balancer outweigh the horizontal scalability of this approach, at least in most cases.

X-Istence 15y ago
Transparent load balancers exist and don't have the issues you are talking about.
- sugarcode 15y ago
  Hardware or software? There's some hacks with TPROXY/HAproxy I've seen that would do the transparent proxy but the setup seems like more trouble than it's worth.
  - X-Istence 15y ago
    OpenBSD PF for example can do round-robin load balancing.
  - ominous_prime 15y ago
    IPVS is built-in to the Linux kernel, and HA projects like keepalived have ipvsadm integration. Tproxy works fine, and has been in the kernel since 2.6.30. In most load-balancing cases, losing the remote IP address isn't that big a deal (you have to deal with NAT too), and a full proxy like haproxy has it's benefits.

thomas-st 15y ago

Why does session affinity not solve the problem of session caching? The author says it's "a whole other world of pain and suffering" but doesn't explain why.

jadc 15y ago
Seconded. It would be great if the author could expand on this point.
- jpetazzo 15y ago
  I guess that it's not easy to implement session affinity on the SSL level. You cannot access information like cookies or other HTTP headers (since they are inside the SSL payload that you're trying to handle). So you have to use the little information you have: source IP address and port, and session data. The idea would be to make sure that a given client is always sent to the same SSL handler. We could imagine having two layers of load balancers: - first layer would use source IP address and/or session data to determine to which server of the second layer the connection should be forwarded; - second layer would receive the connection and to the proper SSL handling. I believe that this would work, but it seems that it would require a custom "half-implementation" of SSL on the first layer of load balancers. I don't know if there is any provision for that in OpenSSL or GNUTLS. Also, since there are already hooks to do session caching in most SSL-enabled servers, using those hooks to plug in a memcached backend seems to be less "disruptive" (read "easier to understand, implement and debug").
  - ominous_prime 15y ago
    You can balance ssl traffic at a TCP level using the client's IP to set affinity. The only real catch is if you have a large base of clients behind a single NAT, but in most cases traffic will balance out pretty well. Sometimes though, you don't want affinity at all. If you don't care what backend server takes the request, you can balance the load more efficiently, and more easily rotate servers in and out of service.
ominous_prime 15y ago

mmaunder 15y ago

Matt this is awesome. Nginx is becoming the standard for front end load balancing for many high traffic sites and this helps. We only have one nginx front end load balancer (even though we do over 4000 http req/s) but we'll be migrating to a cluster soon so I'll give this a whirl.