8 ms·

2 Performance

240 points by migueldemoura 6y ago

tomxor 6y ago
> With HTTP/2, any interruption (packet loss) in the TCP connection blocks all streams (Head of line blocking). This issue is really noticeable on my crappy home mobile internet when loading web pages, in combination with the timeout being absurdly long for reasons I don't understand.
- Flimm 6y ago
  Same here. My Internet was so bad the other day that loading a web article that was already in my cache would hang indefinitely. I enabled "Offline mode" in my browser, and it loaded the article instantly. On macOS, launching Firefox or Chrome would just hang indefinitely (without displaying a window) when in lie-fi, presumably checking for updates or something. I know there's a push to get software to support an offline mode, but I wish there was a similar push to improve software when in lie-fi.
- bsdubernerd 6y ago
  This is a major set back introduced with HTTP/2 and I'm not sure why this is not mentioned often. Under firefox you can set "network.http.sdpy.enable" to false to switch back to HTTP/1. The improvement I have with HTTP2 is hardly noticeable, but HOL blocking is very tangible as soon as you have occasional random packet loss.
  - tomxor 6y ago
    > Under firefox you can set "network.http.sdpy.enable" to false to switch back to HTTP/1. Thanks! I might have to try that out, currently relegated to tunneling everything over sshuttle which of course makes things slower but reliable since it decompiles the TCP packets making them appear to work flawlessly to HTTP... HTTP1 may be a faster solution. [edit] Trying this now. Correction on parent comment for Firefox: goto: about:config search: network.http.spdy.enabled.http2
- jedisct1 6y ago
  So why are we doing DNS over HTTP?
- rubatuga 6y ago
  Is this necessarily true though? I know that TCP acks and seqs contain info about the packets that have already been seen, so if only one packet is missing, the client will tell the server almost immediately, at which point the onus is on the server to resend quickly. This would be at the Linux network layer however. Found an article explaining the concept, called selective acknowledgment: https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Selective_acknowledgments https://en.wikipedia.org/wiki/Transmission_Control_Protocol#... Would reducing the retransmission delay be sufficient? Or simply letting the browser open two connections to a standard HTTP port?
mpweiher 6y ago
TLDR: still slightly slower, but "very excited"
jrochkind1 6y ago
So, as far as the results: In their synthetic benchmarks, they find negligible to no improvement: > For a small test page of 15KB, HTTP/3 takes an average of 443ms to load compared to 458ms for HTTP/2. However, once we increase the page size to 1MB that advantage disappears: HTTP/3 is just slightly slower than HTTP/2 on our network today, taking 2.33s to load versus 2.30s And in their closer to real world benchmarks, they find no improvement, instead some negligible degradation. > As you can see, HTTP/3 performance still trails HTTP/2 performance, by about 1-4% on average in North America and similar results are seen in Europe, Asia and South America. We suspect this could be due to the difference in congestion algorithms: HTTP/2 on BBR v1 vs. HTTP/3 on CUBIC. In the future, we’ll work to support the same congestion algorithm on both to get a more accurate apples-to-apples comparison. As a developer of web apps, I will personally continue to not think that much about HTTP/3. Perhaps in the future network/systems engineers will have figured out how to make it bear fruit? I don't know, but it seems to me of unclear wisdom to count on it.
- jrochkind1 6y ago
  Their own conclusion seems odd to me: > Overall, we’re very excited to be allowed to help push this standard forward. Our implementation is holding up well, offering better performance in some cases and at worst similar to HTTP/2. As the standard finalizes, we’re looking forward to seeing browsers add support for HTTP/3 in mainstream versions. I feel like the conclusion should be "Hypothetical advantages of HTTP/3 still not realized," but they are "We're excited to be working on this", with no mention of... why. Like, why isn't HTTP/3 resulting in expected advantages; what might be changed to change this; what are you going to do to try to realize actual advantages to 'push the standard forward'? It seems like a standard for the sake of saying you have done something and have a standard, if there aren't any realized advantages, no?
  - eastdakota 6y ago
    Because Quiche, our HTTP/3 library, only supports CUBIC for congestion control, not more modern algorithms like BBR. Even without modern congestion control, HTTP/3 performs close to as fast as HTTP/2. We expect it’s performance will improve significantly when we implement BBR and other enhancements already in more mature protocols.
the_duke 6y ago
This does not mention if the tests also simulated and measured packet loss. With a good network connection with little packet loss, I wouldn't expect much benefit to /3. Especially since all the server and client implementations are immature and in user space without kernel support. The benefits should show up with (poor) mobile connections.
- matdehaast 6y ago
  Thanks for pointing this out! I really wish the blog would explain that better. /3 will really shine where connection has degradation. For me the most exciting part is the seamless network switching potential of /3 on mobile devices
- zurn 6y ago
  IME cellular connections don't exhibit much packet loss. The lower level layer of the mobile network usually ensures that the packets are usually eventually delivered, they just take a while. This makes sense since internet protocols are designed to interpret packet loss as congestion and will slow down transmission when it occurs.
pgjones 6y ago
It is possible to compare HTTP/3 to HTTP/2 & HTTP/1 using Python, as Hypercorn (via aioquic for HTTP/3) supports all three. When I compared late last year I found HTTP/3 to be noticeably slower, https://pgjones.dev/blog/early-look-at-http3-2019/ https://pgjones.dev/blog/early-look-at-http3-2019/ however my test was much less comprehensive than the one here.
londons_explore 6y ago
A major benefit of HTTP/3 is the ability to transparently switch from one network connection to another without restarting requests. You could be midway through a gaming session over websocket, and walk away from your wifi, and you shouldn't notice a glitch. Nearly nothing else offers that ability, and it's very annoying, especially in offices with hundreds of wifi access points - I should be able to walk down the corridor on a video call without glitchiness! MPTCP (developed mostly by Apple) offers the same, but Google and Microsoft are holding it back, for some unknown reason.
- ComputerGuru 6y ago
  This is called WiFi handoff and any enterprise AP deployment worth a damn should have this sorted out, albeit in a proprietary manner. The WiFi standard already has a client establishing a connection to a new AP before giving up the old one at the actual “physical” transport layer, these proprietary extensions exchange existing connection state information over the wired backbone between APs when a client is attempting to move from one to the other so that it can theoretically be a “seamless” experience. In theory, anyway.
  - Androider 6y ago
    Presumably HTTP/3 could do WiFi-> 4G -> WiFi -> 5G -> WiFi... hand-offs as you're moving around.
    - ldng 6y ago
      Genuinely asking, is it actually working or is that like the promises of multiplexing in HTTP/2 that don't really work IRL ?
      - vbezhenar 6y ago
        What's wrong with HTTP/2 multiplexing?
      - Matthias247 6y ago
WhatIsDukkha 6y ago
So I can't find the reference but I believe there was a paper a few months back claiming that there were big issues with fairness (as I understand the word) with other protocols. The gist of it was that Quic tends to just flat out choke out TCP running on the same network paths? Anyone know about this? There is some mention of BBRv2 improving fairness but not the outside academic paper I was looking for - https://datatracker.ietf.org/meeting/106/materials/slides-106-iccrg-update-on-bbrv2 https://datatracker.ietf.org/meeting/106/materials/slides-10...
cletus 6y ago
So in a former life I worked on Google Fiber and, among other things, wrote a pure JS Speedtest (before Ookla had one alhtough there's might've been in beta by then). It's still there (http://speed.googlefiber.net http://speed.googlefiber.net). This was necessary because Google Fiber installers use Chromebooks to verify installations and Chromebooks don't support Flash. This is a surprisingly difficult problem, especially given the constraints of using pure JS. Some issues that spring to mind included: - The User-Agent is meaningless on iPhones, basically because Steve Jobs got sick of leaking new models in Apache logs. There are other ways of figuring this out but it's a huge pain. - Send too much traffic and you can crash the browser, particularly on mobile devices; - To maximize throughput it became necessary to use a range of ports and simultaneously communicate on all of them. This in turn could be an issue with firewalls; - Run the test too long and performance in many cases would start to degrade; - Send too much traffic and you could understate the connection speed; - Sending larger blobs tended to be better for measuring throughput but too large could degrade performance or crash the browser. Of course, what "too large" was varied by device; - HTTPS was abysmal for raw throughput on all but the beefiest of computers; - To get the best results you needed to turn off a bunch of stuff like Nagel's algorithm and any implicit gzip compression; - You'd have to send random data to avoid caching even with careful HTTP headers that should've disabled caching. And so on. Perhaps the most vexing issue that I was never able to pin down was with Chrome on Linux. In certain circumstances (and I never figured out what exactly they were other than high throughput), Chrome on Linux would write the blobs it downloaded to /tmp (default behaviour) and never release them until you refreshed the Webpage. And no there were no dangling references. The only clue this was happening was that Chrome would start spitting weird error messages to the console and those errors couldn't be trapped. So pure JS could actually do a lot and I actually spent a fair amount of effort to get this to accurately show speeds up to 10G (I got up to 8.5G down and ~7G up on Chrome on a MBP). But getting back to the article at hand, what you tend to find is how terribly TCP does with latency. A small increase in latency would have a devastating effect on reported speeds. Anyone from Australia should be intimately familiar with this as it's clear (at least to me) that many if not most services are never tested on or designed for high-latency networks. 300ms RTT vs <80ms can be the difference between a relatively snappy SPA and something that is utterly unusable due to serial loads and excessive round trips. So looking at this article, the first thing I searched for was the word "latency" and I didn't find it. Now sure the idea of a CDN like Cloudfare is to have a POP close to most customers but that just isn't always possible. Plus you hit things not in the CDN. Even DNS latency matters here where pople have shown meaningful improvements in Web performance by just having a hot cache of likely DNS lookups. The degradation in throughput in TCP that comes from latency is well-known academically. It just doesn't seem to be known about, given attention to or otherwise catered for in user-facing services. Will HTTP/3 help with this? I have no idea. But I'd like to know before someone dismisses it as having minimal improvements or, worse, as degrading performance.
sholladay 6y ago
In Node.js (curious to hear about other ecosystems), HTTP/2 hasn't even caught on yet. Sure, it's technically supported by Node core and various frameworks, but hardly anyone is really using it. Most of the benefits that HTTP/2 brings to the table require a new model that doesn't map cleanly to the traditional request/response lifecycle. It seems harder to program applications using HTTP/2 because of that. Perhaps some of it is what we are used to and the burden of learning something new, but I don't think that's the whole story. I wonder if future HTTP versions will address this in some way or if it is going to continue to be the new normal. It will be interesting to see what the adoption curve looks like for HTTP/3 and onward. I'm still building everything on HTTP/1.1 (RFC 7230) and have no plans to change that any time soon, even though I can appreciate the features that are available in the newer versions.
- Androider 6y ago
  Turns out it's not really an issue in practice, since you rarely serve naked Node.js to the Internet. If you put something like a load balancer (ELB) or reverse proxy (Nginx) in front of your service which speaks HTTP/2, you already get 95% of the benefits. I expect HTTP/3 to likewise just be a toggle offered by AWS/GCP/Azure/NGinx etc. in the future, and your users will see an immediate benefit.
  - steveklabnik 6y ago
    Cloudflare includes such a toggle for HTTP/3, though to be honest I forget if it's still in a closed beta or more generally available.
- Matthias247 6y ago
  > Most of the benefits that HTTP/2 brings to the table require a new model that doesn't map cleanly to the traditional request/response lifecycle This is not true. The only HTTP/2 feature that doesn't fit into the traditional HTTP semantics is PUSH. And even that is the request/response model - the only difference is that the request is injected also from the server side and not being received from the client. We just pretend we would have received such a request from the client, send the response towards the client, and hope the client won't reject it.
flyinprogrammer 6y ago
When you're ready for an actual improvement check out https://rsocket.io/ https://rsocket.io/
underdeserver 6y ago
I wonder how many different artifacts Cloudflare is serving on this test page. Maybe a real test is the difference grouped by the number of files served on a single page load.
ryanthedev 6y ago
So http3 will be using UDP? Makes sense. Will we see more performance tuning when it comes to MTU sizes?
elsif1 6y ago
I'm curious as to how good the bandwidth estimation is. That's something that can certainly be improved from TCP, but it's also something that has a lot of corner cases and is not usually done super well in UDP protocols (e.g. WebRTC)
KenanSulayman 6y ago
The USP of h3 isn't peak performance, it's 95th percentile latencies.

240 points by migueldemoura 6y ago

> With HTTP/2, any interruption (packet loss) in the TCP connection blocks all streams (Head of line blocking). This issue is really noticeable on my crappy home mobile internet when loading web pages, in combination with the timeout being absurdly long for reasons I don't understand.

Flimm 6y ago
Same here. My Internet was so bad the other day that loading a web article that was already in my cache would hang indefinitely. I enabled "Offline mode" in my browser, and it loaded the article instantly. On macOS, launching Firefox or Chrome would just hang indefinitely (without displaying a window) when in lie-fi, presumably checking for updates or something. I know there's a push to get software to support an offline mode, but I wish there was a similar push to improve software when in lie-fi.
bsdubernerd 6y ago
This is a major set back introduced with HTTP/2 and I'm not sure why this is not mentioned often. Under firefox you can set "network.http.sdpy.enable" to false to switch back to HTTP/1. The improvement I have with HTTP2 is hardly noticeable, but HOL blocking is very tangible as soon as you have occasional random packet loss.
- tomxor 6y ago
  > Under firefox you can set "network.http.sdpy.enable" to false to switch back to HTTP/1. Thanks! I might have to try that out, currently relegated to tunneling everything over sshuttle which of course makes things slower but reliable since it decompiles the TCP packets making them appear to work flawlessly to HTTP... HTTP1 may be a faster solution. [edit] Trying this now. Correction on parent comment for Firefox: goto: about:config search: network.http.spdy.enabled.http2
jedisct1 6y ago
So why are we doing DNS over HTTP?
rubatuga 6y ago
Is this necessarily true though? I know that TCP acks and seqs contain info about the packets that have already been seen, so if only one packet is missing, the client will tell the server almost immediately, at which point the onus is on the server to resend quickly. This would be at the Linux network layer however. Found an article explaining the concept, called selective acknowledgment: https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Selective_acknowledgments https://en.wikipedia.org/wiki/Transmission_Control_Protocol#... Would reducing the retransmission delay be sufficient? Or simply letting the browser open two connections to a standard HTTP port?

mpweiher 6y ago

TLDR: still slightly slower, but "very excited"

jrochkind1 6y ago

So, as far as the results: In their synthetic benchmarks, they find negligible to no improvement: > For a small test page of 15KB, HTTP/3 takes an average of 443ms to load compared to 458ms for HTTP/2. However, once we increase the page size to 1MB that advantage disappears: HTTP/3 is just slightly slower than HTTP/2 on our network today, taking 2.33s to load versus 2.30s And in their closer to real world benchmarks, they find no improvement, instead some negligible degradation. > As you can see, HTTP/3 performance still trails HTTP/2 performance, by about 1-4% on average in North America and similar results are seen in Europe, Asia and South America. We suspect this could be due to the difference in congestion algorithms: HTTP/2 on BBR v1 vs. HTTP/3 on CUBIC. In the future, we’ll work to support the same congestion algorithm on both to get a more accurate apples-to-apples comparison. As a developer of web apps, I will personally continue to not think that much about HTTP/3. Perhaps in the future network/systems engineers will have figured out how to make it bear fruit? I don't know, but it seems to me of unclear wisdom to count on it.

jrochkind1 6y ago
Their own conclusion seems odd to me: > Overall, we’re very excited to be allowed to help push this standard forward. Our implementation is holding up well, offering better performance in some cases and at worst similar to HTTP/2. As the standard finalizes, we’re looking forward to seeing browsers add support for HTTP/3 in mainstream versions. I feel like the conclusion should be "Hypothetical advantages of HTTP/3 still not realized," but they are "We're excited to be working on this", with no mention of... why. Like, why isn't HTTP/3 resulting in expected advantages; what might be changed to change this; what are you going to do to try to realize actual advantages to 'push the standard forward'? It seems like a standard for the sake of saying you have done something and have a standard, if there aren't any realized advantages, no?
- eastdakota 6y ago
  Because Quiche, our HTTP/3 library, only supports CUBIC for congestion control, not more modern algorithms like BBR. Even without modern congestion control, HTTP/3 performs close to as fast as HTTP/2. We expect it’s performance will improve significantly when we implement BBR and other enhancements already in more mature protocols.

the_duke 6y ago

This does not mention if the tests also simulated and measured packet loss. With a good network connection with little packet loss, I wouldn't expect much benefit to /3. Especially since all the server and client implementations are immature and in user space without kernel support. The benefits should show up with (poor) mobile connections.

matdehaast 6y ago
Thanks for pointing this out! I really wish the blog would explain that better. /3 will really shine where connection has degradation. For me the most exciting part is the seamless network switching potential of /3 on mobile devices
zurn 6y ago
IME cellular connections don't exhibit much packet loss. The lower level layer of the mobile network usually ensures that the packets are usually eventually delivered, they just take a while. This makes sense since internet protocols are designed to interpret packet loss as congestion and will slow down transmission when it occurs.

pgjones 6y ago

It is possible to compare HTTP/3 to HTTP/2 & HTTP/1 using Python, as Hypercorn (via aioquic for HTTP/3) supports all three. When I compared late last year I found HTTP/3 to be noticeably slower, https://pgjones.dev/blog/early-look-at-http3-2019/ https://pgjones.dev/blog/early-look-at-http3-2019/ however my test was much less comprehensive than the one here.

londons_explore 6y ago

A major benefit of HTTP/3 is the ability to transparently switch from one network connection to another without restarting requests. You could be midway through a gaming session over websocket, and walk away from your wifi, and you shouldn't notice a glitch. Nearly nothing else offers that ability, and it's very annoying, especially in offices with hundreds of wifi access points - I should be able to walk down the corridor on a video call without glitchiness! MPTCP (developed mostly by Apple) offers the same, but Google and Microsoft are holding it back, for some unknown reason.

ComputerGuru 6y ago
This is called WiFi handoff and any enterprise AP deployment worth a damn should have this sorted out, albeit in a proprietary manner. The WiFi standard already has a client establishing a connection to a new AP before giving up the old one at the actual “physical” transport layer, these proprietary extensions exchange existing connection state information over the wired backbone between APs when a client is attempting to move from one to the other so that it can theoretically be a “seamless” experience. In theory, anyway.
- Androider 6y ago
  Presumably HTTP/3 could do WiFi-> 4G -> WiFi -> 5G -> WiFi... hand-offs as you're moving around.
  - ldng 6y ago
    Genuinely asking, is it actually working or is that like the promises of multiplexing in HTTP/2 that don't really work IRL ?
    - vbezhenar 6y ago
      What's wrong with HTTP/2 multiplexing?
    - Matthias247 6y ago

WhatIsDukkha 6y ago

So I can't find the reference but I believe there was a paper a few months back claiming that there were big issues with fairness (as I understand the word) with other protocols. The gist of it was that Quic tends to just flat out choke out TCP running on the same network paths? Anyone know about this? There is some mention of BBRv2 improving fairness but not the outside academic paper I was looking for - https://datatracker.ietf.org/meeting/106/materials/slides-106-iccrg-update-on-bbrv2 https://datatracker.ietf.org/meeting/106/materials/slides-10...

cletus 6y ago

So in a former life I worked on Google Fiber and, among other things, wrote a pure JS Speedtest (before Ookla had one alhtough there's might've been in beta by then). It's still there (http://speed.googlefiber.net http://speed.googlefiber.net). This was necessary because Google Fiber installers use Chromebooks to verify installations and Chromebooks don't support Flash. This is a surprisingly difficult problem, especially given the constraints of using pure JS. Some issues that spring to mind included: - The User-Agent is meaningless on iPhones, basically because Steve Jobs got sick of leaking new models in Apache logs. There are other ways of figuring this out but it's a huge pain. - Send too much traffic and you can crash the browser, particularly on mobile devices; - To maximize throughput it became necessary to use a range of ports and simultaneously communicate on all of them. This in turn could be an issue with firewalls; - Run the test too long and performance in many cases would start to degrade; - Send too much traffic and you could understate the connection speed; - Sending larger blobs tended to be better for measuring throughput but too large could degrade performance or crash the browser. Of course, what "too large" was varied by device; - HTTPS was abysmal for raw throughput on all but the beefiest of computers; - To get the best results you needed to turn off a bunch of stuff like Nagel's algorithm and any implicit gzip compression; - You'd have to send random data to avoid caching even with careful HTTP headers that should've disabled caching. And so on. Perhaps the most vexing issue that I was never able to pin down was with Chrome on Linux. In certain circumstances (and I never figured out what exactly they were other than high throughput), Chrome on Linux would write the blobs it downloaded to /tmp (default behaviour) and never release them until you refreshed the Webpage. And no there were no dangling references. The only clue this was happening was that Chrome would start spitting weird error messages to the console and those errors couldn't be trapped. So pure JS could actually do a lot and I actually spent a fair amount of effort to get this to accurately show speeds up to 10G (I got up to 8.5G down and ~7G up on Chrome on a MBP). But getting back to the article at hand, what you tend to find is how terribly TCP does with latency. A small increase in latency would have a devastating effect on reported speeds. Anyone from Australia should be intimately familiar with this as it's clear (at least to me) that many if not most services are never tested on or designed for high-latency networks. 300ms RTT vs <80ms can be the difference between a relatively snappy SPA and something that is utterly unusable due to serial loads and excessive round trips. So looking at this article, the first thing I searched for was the word "latency" and I didn't find it. Now sure the idea of a CDN like Cloudfare is to have a POP close to most customers but that just isn't always possible. Plus you hit things not in the CDN. Even DNS latency matters here where pople have shown meaningful improvements in Web performance by just having a hot cache of likely DNS lookups. The degradation in throughput in TCP that comes from latency is well-known academically. It just doesn't seem to be known about, given attention to or otherwise catered for in user-facing services. Will HTTP/3 help with this? I have no idea. But I'd like to know before someone dismisses it as having minimal improvements or, worse, as degrading performance.

sholladay 6y ago

In Node.js (curious to hear about other ecosystems), HTTP/2 hasn't even caught on yet. Sure, it's technically supported by Node core and various frameworks, but hardly anyone is really using it. Most of the benefits that HTTP/2 brings to the table require a new model that doesn't map cleanly to the traditional request/response lifecycle. It seems harder to program applications using HTTP/2 because of that. Perhaps some of it is what we are used to and the burden of learning something new, but I don't think that's the whole story. I wonder if future HTTP versions will address this in some way or if it is going to continue to be the new normal. It will be interesting to see what the adoption curve looks like for HTTP/3 and onward. I'm still building everything on HTTP/1.1 (RFC 7230) and have no plans to change that any time soon, even though I can appreciate the features that are available in the newer versions.

Androider 6y ago
Turns out it's not really an issue in practice, since you rarely serve naked Node.js to the Internet. If you put something like a load balancer (ELB) or reverse proxy (Nginx) in front of your service which speaks HTTP/2, you already get 95% of the benefits. I expect HTTP/3 to likewise just be a toggle offered by AWS/GCP/Azure/NGinx etc. in the future, and your users will see an immediate benefit.
- steveklabnik 6y ago
  Cloudflare includes such a toggle for HTTP/3, though to be honest I forget if it's still in a closed beta or more generally available.
Matthias247 6y ago
> Most of the benefits that HTTP/2 brings to the table require a new model that doesn't map cleanly to the traditional request/response lifecycle This is not true. The only HTTP/2 feature that doesn't fit into the traditional HTTP semantics is PUSH. And even that is the request/response model - the only difference is that the request is injected also from the server side and not being received from the client. We just pretend we would have received such a request from the client, send the response towards the client, and hope the client won't reject it.

flyinprogrammer 6y ago

When you're ready for an actual improvement check out https://rsocket.io/ https://rsocket.io/

underdeserver 6y ago

I wonder how many different artifacts Cloudflare is serving on this test page. Maybe a real test is the difference grouped by the number of files served on a single page load.

ryanthedev 6y ago

So http3 will be using UDP? Makes sense. Will we see more performance tuning when it comes to MTU sizes?

elsif1 6y ago

I'm curious as to how good the bandwidth estimation is. That's something that can certainly be improved from TCP, but it's also something that has a lot of corner cases and is not usually done super well in UDP protocols (e.g. WebRTC)

KenanSulayman 6y ago

The USP of h3 isn't peak performance, it's 95th percentile latencies.