5 ms·

You can open multiple tcp connections in parallel to virtually eliminate the head of line blocking problem for the vast majority of experiences. The problem ha

You can open multiple tcp connections in parallel to virtually eliminate the head of line blocking problem for the vast majority of experiences. The problem has been exaggerated for the sake of justifying http2/3 but in practical terms it barely affects the average user experience. 80% optimal was good enough. On the other hand it reduces cost by a few percentage points for the vanishingly few web hosts large enough such that that would have a meaningful financial impact.

vhiremath4 6y ago
There is a cost to opening a new connection and having to re-incur multiple roundtrips in order to complete a new TCP and TLS handshake. This is a meaningful overhead savings for folks with a network setup that has high last-mile latency.
- CyberRabbi 6y ago
  As has been demonstrated by my original post, clearly saving a few rtt is irrelevant here. Anyway your point is halfway-irrelevant since if you initiate all connections in parallel you only suffer the rtt latency of a single connection. The only people saving here are the people running servers that serve world-scale numbers of requests. The average user notices nearly no difference.
  - alexchamberlain 6y ago
    The world-scale companies are also the companies writing the standards.
    - CyberRabbi 6y ago
      Funny that
  - vhiremath4 6y ago
    I'm not sure I fully follow here, but I would like to be able to. Created a video response: https://www.loom.com/share/4539869f41cf47c5a154a342308584de https://www.loom.com/share/4539869f41cf47c5a154a342308584de
    - CyberRabbi 6y ago
      Hehe I engage with people like that all the time and I think I’ve learned a lot. In my humble opinion the people who are easily offended aren’t really worth discussing difficult problems with. Actually I find that I learn a lot more from people who don’t get hung up when their ego happens to unintentionally get bruised. I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push.
      - vhiremath4 6y ago
        My ego is not bruised. As I said: > I'm not sure I fully follow here, but I would like to be able to What I'm calling out is that you may have fallen into the trap (as I did years back) that you need to be an asshole to drive towards solving hard problems. You don't, and it actually blocked me from creating the psychological safety necessary to get good ideas out of people who otherwise wouldn't feel comfortable speaking up. Genuinely good/supporting vibes from my direction to you - do with it what you will, but I do believe your ego is showing here more than mine. We can agree to disagree since I believe each of us will over-index on our past life experiences vs. those of some random person on Hacker News. > I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push. I think I get what you're saying, but honestly the way you're wording your points is a bit confusing. I am going to chalk this up to "I'm just too dumb" for sake of laziness at this point.
        CyberRabbi 6y ago
        I don’t believe that you need to be an asshole to drive towards solving hard problems. Not in the least. That is your projection. Calling your point irrelevant is not a personal attack on you, it was intended to be a neutral objective statement. Smart people often make irrelevant points, in fact the smartest people often make the most irrelevant points when trying to drive at the truth. I don’t believe you are dumb but I do have reason to believe you can interpret a personal attack where there is none.
  - jayd16 6y ago
    What if you're not running requests in parallel and instead want a pre-warmed tls connection? Something like gRPC could be implemented as a websocket protocol but you lose all the http semantics for each internal request. You can't expect a load balancer to handle your custom socket protocol but with a standard like http/2 nginx is able to do it.
    - CyberRabbi 6y ago
      That sounds like a TLS problem and not an http problem. Fortunately TLS has already solved that problem: https://blog.cloudflare.com/introducing-0-rtt/ https://blog.cloudflare.com/introducing-0-rtt/ Though http keepalive seems to already solve your use-case, without TLS 1.3’s connection rtt optimizations, if you indeed aren’t running requests in parallel.
      - jayd16 6y ago
        >Specifically, only GET requests with no query parameters are answered over 0-RTT. Sorry but no. Not solved. They also have a newer post about 0-RTT and how QUIC makes it even faster. https://blog.cloudflare.com/even-faster-connection-establishment-with-quic-0-rtt-resumption/ https://blog.cloudflare.com/even-faster-connection-establish...
        CyberRabbi 6y ago
        >Specifically, only GET requests with no query parameters are answered over 0-RTT. That only applies to cloudflare’s implementation of 0-rtt. That isn’t a general restriction. Additionally 0-rtt isn’t the only rtt optimization in tls 1.3. Either way your original point was moot since http keepalive is sufficient to avoid tls connection costs for serial requests.
  - youngtaff 6y ago
    Those connections will all go though TCP startup phase where throughput is limited by the congestion window size. And because they're unaware of each other it's likely they'll eventually start competing with each other and cause packet loss
    - CyberRabbi 6y ago
      In the vast majority of cases, neither of those things practically matter. The multiple connections are for intended for concurrency, not parallelism. Web assets are not large enough for slow start to impact overall download speed by any large margin, nor sustain downloads long enough that the connections start to compete. Requests are bursty.
      - youngtaff 6y ago
        It's quite a few years old –https://web.archive.org/web/20170724120441/https://insouciant.org/tech/network-congestion-and-web-browsing/ https://web.archive.org/web/20170724120441/https://insoucian... – but I still see this in some packet captures
        CyberRabbi 6y ago
        Yea technically there is a potential benefit but the point is that it’s so marginal that the majority of users won’t ever notice it. You’re saving 1-5ms off of a page load.
    - saurik 6y ago
      ...but multiple TCP connections (within reason: too many causes a problem, but not just a handful) actually almost always out-perform a single one for numerous reasons (including the congestion controller having more leverage from looking like multiple users, being able to hit multiple backend machines from dns load balancing, and working around various per-connection bottlenecks); this is in fact the primary trick of "download helpers": they download multiple independent segments of the file on parallel connections.
TwoBit 6y ago
Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?
- CyberRabbi 6y ago
  I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.
  - jlokier 6y ago
    It's actually not trivial to optimize dynamic prioritisation across multiple connections. To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence. But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps. It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial. With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots. With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine. Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.
    - CyberRabbi 6y ago
raxxorrax 6y ago
But you would have to implement it on a layer that maybe should be exempt from dealing with network loading. I agree that the benefits only fit certain use cases, but it is still nice to have it and it is already widely adapted.
lxgr 6y ago
> vanishingly few web hosts large enough such that that would have a meaningful financial impact By that logic, electricity wasted on inefficient power adapters and devices in standby mode is irrelevant: The marginal savings per household are cents per year. If it doesn't hurt me as an end user (and I don't see how HTTP/2 does), what's the objection here? Nor is any smaller service provider forced to adopt it server-side.
- CyberRabbi 6y ago
  The premise of my argument is that the end user is barely affected either way. There is potential harm to the web ecosystem by making the stack more complex and I am questioning the rationale for tolerating that harm is since these fundamental changes are being mostly driven by the specific use cases of large corporation, not the needs of the greater community. You’re right no one is forced to adopt this new web stack and for that I am grateful. Given recent events I will likely not adopt google’s web stack.

vhiremath4 6y ago

There is a cost to opening a new connection and having to re-incur multiple roundtrips in order to complete a new TCP and TLS handshake. This is a meaningful overhead savings for folks with a network setup that has high last-mile latency.

CyberRabbi 6y ago
As has been demonstrated by my original post, clearly saving a few rtt is irrelevant here. Anyway your point is halfway-irrelevant since if you initiate all connections in parallel you only suffer the rtt latency of a single connection. The only people saving here are the people running servers that serve world-scale numbers of requests. The average user notices nearly no difference.
- alexchamberlain 6y ago
  The world-scale companies are also the companies writing the standards.
  - CyberRabbi 6y ago
    Funny that
- vhiremath4 6y ago
  I'm not sure I fully follow here, but I would like to be able to. Created a video response: https://www.loom.com/share/4539869f41cf47c5a154a342308584de https://www.loom.com/share/4539869f41cf47c5a154a342308584de
  - CyberRabbi 6y ago
    Hehe I engage with people like that all the time and I think I’ve learned a lot. In my humble opinion the people who are easily offended aren’t really worth discussing difficult problems with. Actually I find that I learn a lot more from people who don’t get hung up when their ego happens to unintentionally get bruised. I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push.
    - vhiremath4 6y ago
      My ego is not bruised. As I said: > I'm not sure I fully follow here, but I would like to be able to What I'm calling out is that you may have fallen into the trap (as I did years back) that you need to be an asshole to drive towards solving hard problems. You don't, and it actually blocked me from creating the psychological safety necessary to get good ideas out of people who otherwise wouldn't feel comfortable speaking up. Genuinely good/supporting vibes from my direction to you - do with it what you will, but I do believe your ego is showing here more than mine. We can agree to disagree since I believe each of us will over-index on our past life experiences vs. those of some random person on Hacker News. > I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push. I think I get what you're saying, but honestly the way you're wording your points is a bit confusing. I am going to chalk this up to "I'm just too dumb" for sake of laziness at this point.
      - CyberRabbi 6y ago
        I don’t believe that you need to be an asshole to drive towards solving hard problems. Not in the least. That is your projection. Calling your point irrelevant is not a personal attack on you, it was intended to be a neutral objective statement. Smart people often make irrelevant points, in fact the smartest people often make the most irrelevant points when trying to drive at the truth. I don’t believe you are dumb but I do have reason to believe you can interpret a personal attack where there is none.
- jayd16 6y ago
  What if you're not running requests in parallel and instead want a pre-warmed tls connection? Something like gRPC could be implemented as a websocket protocol but you lose all the http semantics for each internal request. You can't expect a load balancer to handle your custom socket protocol but with a standard like http/2 nginx is able to do it.
  - CyberRabbi 6y ago
    That sounds like a TLS problem and not an http problem. Fortunately TLS has already solved that problem: https://blog.cloudflare.com/introducing-0-rtt/ https://blog.cloudflare.com/introducing-0-rtt/ Though http keepalive seems to already solve your use-case, without TLS 1.3’s connection rtt optimizations, if you indeed aren’t running requests in parallel.
    - jayd16 6y ago
      >Specifically, only GET requests with no query parameters are answered over 0-RTT. Sorry but no. Not solved. They also have a newer post about 0-RTT and how QUIC makes it even faster. https://blog.cloudflare.com/even-faster-connection-establishment-with-quic-0-rtt-resumption/ https://blog.cloudflare.com/even-faster-connection-establish...
      - CyberRabbi 6y ago
        >Specifically, only GET requests with no query parameters are answered over 0-RTT. That only applies to cloudflare’s implementation of 0-rtt. That isn’t a general restriction. Additionally 0-rtt isn’t the only rtt optimization in tls 1.3. Either way your original point was moot since http keepalive is sufficient to avoid tls connection costs for serial requests.
- youngtaff 6y ago
  Those connections will all go though TCP startup phase where throughput is limited by the congestion window size. And because they're unaware of each other it's likely they'll eventually start competing with each other and cause packet loss
  - CyberRabbi 6y ago
    In the vast majority of cases, neither of those things practically matter. The multiple connections are for intended for concurrency, not parallelism. Web assets are not large enough for slow start to impact overall download speed by any large margin, nor sustain downloads long enough that the connections start to compete. Requests are bursty.
    - youngtaff 6y ago
      It's quite a few years old –https://web.archive.org/web/20170724120441/https://insouciant.org/tech/network-congestion-and-web-browsing/ https://web.archive.org/web/20170724120441/https://insoucian... – but I still see this in some packet captures
      - CyberRabbi 6y ago
        Yea technically there is a potential benefit but the point is that it’s so marginal that the majority of users won’t ever notice it. You’re saving 1-5ms off of a page load.
  - saurik 6y ago
    ...but multiple TCP connections (within reason: too many causes a problem, but not just a handful) actually almost always out-perform a single one for numerous reasons (including the congestion controller having more leverage from looking like multiple users, being able to hit multiple backend machines from dns load balancing, and working around various per-connection bottlenecks); this is in fact the primary trick of "download helpers": they download multiple independent segments of the file on parallel connections.

TwoBit 6y ago

Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?

CyberRabbi 6y ago
I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.
- jlokier 6y ago
  It's actually not trivial to optimize dynamic prioritisation across multiple connections. To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence. But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps. It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial. With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots. With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine. Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.
  - CyberRabbi 6y ago

raxxorrax 6y ago

But you would have to implement it on a layer that maybe should be exempt from dealing with network loading. I agree that the benefits only fit certain use cases, but it is still nice to have it and it is already widely adapted.

lxgr 6y ago

> vanishingly few web hosts large enough such that that would have a meaningful financial impact By that logic, electricity wasted on inefficient power adapters and devices in standby mode is irrelevant: The marginal savings per household are cents per year. If it doesn't hurt me as an end user (and I don't see how HTTP/2 does), what's the objection here? Nor is any smaller service provider forced to adopt it server-side.

CyberRabbi 6y ago
The premise of my argument is that the end user is barely affected either way. There is potential harm to the web ecosystem by making the stack more complex and I am questioning the rationale for tolerating that harm is since these fundamental changes are being mostly driven by the specific use cases of large corporation, not the needs of the greater community. You’re right no one is forced to adopt this new web stack and for that I am grateful. Given recent events I will likely not adopt google’s web stack.