4 ms·

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?

rovr138 6y ago
Where is SSL the bottleneck? Wondering if terminating earlier and just relying on HTTP after would help.
- wahern 6y ago
  Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. Session caching can be an important optimization that can significantly reduce those costs, but scaling session caching has its own problems. But IME real-world bottlenecks have more to do with overall architecture. People tend to heavily focus on technical details, such as concurrency architecture--the how. But the biggest opportunities for improved performance usually involve functional aspects of an architecture--the what. (Note that these aren't fixed categories; they're relative positions. A technical detail often becomes a functional model as development progresses.) 12 RPS is a long way from implicating SSL. If you get to the point where SSL is an identifiable bottleneck, you've either made a series of tremendously good decisions or exceptionally poor decisions.
  - toast0 6y ago
    > Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. I don't think that's right. Cloudflare's blog [1] says they can do about 9.5x the handshakes/sec with ECDSA at 256-bits vs RSA at 2048. Verification for ECDSA signatures are somewhat slower, but it's usually an acceptable tradeoff to make clients do a bit more work so that servers do a lot less. I agree though, at 12 RPS, TLS isn't the bottleneck. [1] https://blog.cloudflare.com/ecdsa-the-digital-signature-algorithm-of-a-better-internet/ https://blog.cloudflare.com/ecdsa-the-digital-signature-algo...
    - wahern 6y ago
      Ah, you're right. 1) I got it backwards, verification is much faster with RSA, so RSA is better for clients). 2) I was testing libressl (macOS, OpenBSD), where signing speeds between rsa2048 and ecdsap256 are nearly identical (Core i5, M1, AMD GX-412T), whereas with OpenSSL (AMD EPYC) ecdsap256 is faster (30x advantage to ecdsap256 actually, as compared to 4x verification advantage to RSA). Though, the magnitudes here seem to be sensitive to optimization effort.
kijin 6y ago
HTTP/1.1 Keep-Alive helps a lot. HTTP/2 is even better. You only do the SSL handshake once.
- notafraudster 6y ago
  Flipping the HTTP/2 switch was amazing for me -- I have a page that for reasons needs to load ~500 small images. Initially I was worried about having to figure out a sprite-based method to compile the images (which is not ideal because there are many permutations of which 500 images), but when I turned on HTTP/2, the overhead just disappeared. The images load instantly. I'm nowhere near the multi-k RPS metrics as above, but it was night and day for me even for individual requests.
  - Ueland 6y ago
    Now web-developers just need to realize that much of the value in using CDN's is gone. Earlier years it made sense to spread content over multiple domains, like a CDN domain for static content. Now due to HTTP2, as much as possible should be served on the same domain to get the full HTTP2 effect.
gwittel 6y ago
SSL handshaking is definitely a bottleneck. I can’t speak for the benchmarks but with SSL typically this scales via using persistent connections (keep alive), and SSL session resumption (much lower cost to setup vs a full handshake). In the end you want to measure perf with and without SSL so you can identify the real bottleneck. Otherwise you might just benchmark your SSL implementations handshake performance and not what you really want. As an example, I found that Kubernetes nginx-ingress can’t cache SSL sessions on the upstream side (nginx to your app). So request bursts can really hurt your application unless your pool has enough open connections to handle the burst (keep alive and keep alive requests). Without benchmarking my app in different ways I wouldn’t have figured this out as easily and might have just assumed my app was slow.
ev1 6y ago
HTTP/3 is 0-RTT. You only need to connect once, also UDP

rovr138 6y ago

Where is SSL the bottleneck? Wondering if terminating earlier and just relying on HTTP after would help.

wahern 6y ago
Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. Session caching can be an important optimization that can significantly reduce those costs, but scaling session caching has its own problems. But IME real-world bottlenecks have more to do with overall architecture. People tend to heavily focus on technical details, such as concurrency architecture--the how. But the biggest opportunities for improved performance usually involve functional aspects of an architecture--the what. (Note that these aren't fixed categories; they're relative positions. A technical detail often becomes a functional model as development progresses.) 12 RPS is a long way from implicating SSL. If you get to the point where SSL is an identifiable bottleneck, you've either made a series of tremendously good decisions or exceptionally poor decisions.
- toast0 6y ago
  > Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. I don't think that's right. Cloudflare's blog [1] says they can do about 9.5x the handshakes/sec with ECDSA at 256-bits vs RSA at 2048. Verification for ECDSA signatures are somewhat slower, but it's usually an acceptable tradeoff to make clients do a bit more work so that servers do a lot less. I agree though, at 12 RPS, TLS isn't the bottleneck. [1] https://blog.cloudflare.com/ecdsa-the-digital-signature-algorithm-of-a-better-internet/ https://blog.cloudflare.com/ecdsa-the-digital-signature-algo...
  - wahern 6y ago
    Ah, you're right. 1) I got it backwards, verification is much faster with RSA, so RSA is better for clients). 2) I was testing libressl (macOS, OpenBSD), where signing speeds between rsa2048 and ecdsap256 are nearly identical (Core i5, M1, AMD GX-412T), whereas with OpenSSL (AMD EPYC) ecdsap256 is faster (30x advantage to ecdsap256 actually, as compared to 4x verification advantage to RSA). Though, the magnitudes here seem to be sensitive to optimization effort.

kijin 6y ago

HTTP/1.1 Keep-Alive helps a lot. HTTP/2 is even better. You only do the SSL handshake once.

notafraudster 6y ago
Flipping the HTTP/2 switch was amazing for me -- I have a page that for reasons needs to load ~500 small images. Initially I was worried about having to figure out a sprite-based method to compile the images (which is not ideal because there are many permutations of which 500 images), but when I turned on HTTP/2, the overhead just disappeared. The images load instantly. I'm nowhere near the multi-k RPS metrics as above, but it was night and day for me even for individual requests.
- Ueland 6y ago
  Now web-developers just need to realize that much of the value in using CDN's is gone. Earlier years it made sense to spread content over multiple domains, like a CDN domain for static content. Now due to HTTP2, as much as possible should be served on the same domain to get the full HTTP2 effect.

gwittel 6y ago

SSL handshaking is definitely a bottleneck. I can’t speak for the benchmarks but with SSL typically this scales via using persistent connections (keep alive), and SSL session resumption (much lower cost to setup vs a full handshake). In the end you want to measure perf with and without SSL so you can identify the real bottleneck. Otherwise you might just benchmark your SSL implementations handshake performance and not what you really want. As an example, I found that Kubernetes nginx-ingress can’t cache SSL sessions on the upstream side (nginx to your app). So request bursts can really hurt your application unless your pool has enough open connections to handle the burst (keep alive and keep alive requests). Without benchmarking my app in different ways I wouldn’t have figured this out as easily and might have just assumed my app was slow.

ev1 6y ago

HTTP/3 is 0-RTT. You only need to connect once, also UDP