3 ms·

2k sockets on a test bed vs 2k real user request in production is very different. I doubt you ran a top 1000 Alexa site on your laptop. Today we need to deal wi

by z3t4 5y ago

2k sockets on a test bed vs 2k real user request in production is very different. I doubt you ran a top 1000 Alexa site on your laptop. Today we need to deal with SSL which eats from the performance budget.

jiggawatts 5y ago
> SSL which eats from the performance budget. That was a short-lived thing, and has now become a myth perpetuated by companies like Citrix and F5 that sell "SSL offload" appliances for $$$. Have you benchmarked the overhead of TLS? In my experience, a single CPU core can easily put out multiple gigabytes of AES-256 (tens of gigabits). This benchmark shows 3 GB/s (24 Gbps) for recent AMD CPUs, and nearly 40 Gbps per core for an Intel CPU: https://calomel.org/aesni_ssl_performance.html https://calomel.org/aesni_ssl_performance.html A multi-core server is very unlikely to have more than a 1-5% overheard due to TLS. Even connection set up is a minor overhead with elliptic curve certificates. This is thanks to the AES offload instructions, which are present in all server CPUs made any time in the last 5-7 years or so. As long as the modern Galois Counter Mode (GCM) is used with AES, performance should be great. Meanwhile, Citrix ADC v13 with a hardware "SSL offload card" actually slows down connections! I had a very hard time getting more than 600 Mbps through one. It seems to be the way the ASIC offload chip is architected: it seems to use a large number of slow cores, a bit like a GPU. This means that any one TLS stream will have its bandwidth capped!
- tpetry 5y ago
  The problem with these benchmarks is they measure the bandwidth you can push through an established tls connection. Try to build 2000 new tls connections a seconds (yes many are still active and dont need to be restarted) that is what is the really slow part. Not sending the data over already established channels.
  - lossolo 5y ago
    This exactly. Most CPU time in our Haproxy is spent on crypto/new TLS sessions handling thousands of new connections per second.

by z3t4 5y ago

jiggawatts 5y ago

> SSL which eats from the performance budget. That was a short-lived thing, and has now become a myth perpetuated by companies like Citrix and F5 that sell "SSL offload" appliances for $$$. Have you benchmarked the overhead of TLS? In my experience, a single CPU core can easily put out multiple gigabytes of AES-256 (tens of gigabits). This benchmark shows 3 GB/s (24 Gbps) for recent AMD CPUs, and nearly 40 Gbps per core for an Intel CPU: https://calomel.org/aesni_ssl_performance.html https://calomel.org/aesni_ssl_performance.html A multi-core server is very unlikely to have more than a 1-5% overheard due to TLS. Even connection set up is a minor overhead with elliptic curve certificates. This is thanks to the AES offload instructions, which are present in all server CPUs made any time in the last 5-7 years or so. As long as the modern Galois Counter Mode (GCM) is used with AES, performance should be great. Meanwhile, Citrix ADC v13 with a hardware "SSL offload card" actually slows down connections! I had a very hard time getting more than 600 Mbps through one. It seems to be the way the ASIC offload chip is architected: it seems to use a large number of slow cores, a bit like a GPU. This means that any one TLS stream will have its bandwidth capped!

tpetry 5y ago
The problem with these benchmarks is they measure the bandwidth you can push through an established tls connection. Try to build 2000 new tls connections a seconds (yes many are still active and dont need to be restarted) that is what is the really slow part. Not sending the data over already established channels.
- lossolo 5y ago
  This exactly. Most CPU time in our Haproxy is spent on crypto/new TLS sessions handling thousands of new connections per second.