8 ms·

Weird that all these man hours were spent revising the entire web stack for the sake of squeezing out the last bit of unnecessary latency via quic / http3 just

Weird that all these man hours were spent revising the entire web stack for the sake of squeezing out the last bit of unnecessary latency via quic / http3 just for them to leave out server push. It’s like going from 80% to 99% just to leave off the last 1%. Without going all the way with server push, the logic behind the transition from http1.1/tls/tcp to http3 seems a bit dubious to me. If they aren’t going to go all the way, things already worked well enough IMO. Seems like google is just arbitrarily deciding what they want the web stack to look like.

AgentME 6y ago
HTTP/2 and HTTP/3 are both very useful even without server push. HTTP/2 fixed the issues of HTTP pipelining and made it so multiple requests could share one connection without issues like head-of-line blocking, and HTTP/3 improves things further so that a missed packet in one request doesn't slow down simultaneous requests. Server push was never a critical feature of them. The measured improvements from 2 and 3 were almost always without server push.
- CyberRabbi 6y ago
  You can open multiple tcp connections in parallel to virtually eliminate the head of line blocking problem for the vast majority of experiences. The problem has been exaggerated for the sake of justifying http2/3 but in practical terms it barely affects the average user experience. 80% optimal was good enough. On the other hand it reduces cost by a few percentage points for the vanishingly few web hosts large enough such that that would have a meaningful financial impact.
  - vhiremath4 6y ago
    There is a cost to opening a new connection and having to re-incur multiple roundtrips in order to complete a new TCP and TLS handshake. This is a meaningful overhead savings for folks with a network setup that has high last-mile latency.
    - CyberRabbi 6y ago
      As has been demonstrated by my original post, clearly saving a few rtt is irrelevant here. Anyway your point is halfway-irrelevant since if you initiate all connections in parallel you only suffer the rtt latency of a single connection. The only people saving here are the people running servers that serve world-scale numbers of requests. The average user notices nearly no difference.
      - alexchamberlain 6y ago
        The world-scale companies are also the companies writing the standards.
        CyberRabbi 6y ago
        Funny that
      - vhiremath4 6y ago
        I'm not sure I fully follow here, but I would like to be able to. Created a video response: https://www.loom.com/share/4539869f41cf47c5a154a342308584de https://www.loom.com/share/4539869f41cf47c5a154a342308584de
        CyberRabbi 6y ago
        Hehe I engage with people like that all the time and I think I’ve learned a lot. In my humble opinion the people who are easily offended aren’t really worth discussing difficult problems with. Actually I find that I learn a lot more from people who don’t get hung up when their ego happens to unintentionally get bruised. I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push.
        vhiremath4 6y ago
        My ego is not bruised. As I said: > I'm not sure I fully follow here, but I would like to be able to What I'm calling out is that you may have fallen into the trap (as I did years back) that you need to be an asshole to drive towards solving hard problems. You don't, and it actually blocked me from creating the psychological safety necessary to get good ideas out of people who otherwise wouldn't feel comfortable speaking up. Genuinely good/supporting vibes from my direction to you - do with it what you will, but I do believe your ego is showing here more than mine. We can agree to disagree since I believe each of us will over-index on our past life experiences vs. those of some random person on Hacker News. > I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push. I think I get what you're saying, but honestly the way you're wording your points is a bit confusing. I am going to chalk this up to "I'm just too dumb" for sake of laziness at this point.
  - TwoBit 6y ago
    Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?
    - CyberRabbi 6y ago
      I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.
      - jlokier 6y ago
        It's actually not trivial to optimize dynamic prioritisation across multiple connections. To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence. But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps. It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial. With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots. With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine. Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.
        CyberRabbi 6y ago
  - raxxorrax 6y ago
    But you would have to implement it on a layer that maybe should be exempt from dealing with network loading. I agree that the benefits only fit certain use cases, but it is still nice to have it and it is already widely adapted.
  - lxgr 6y ago
    > vanishingly few web hosts large enough such that that would have a meaningful financial impact By that logic, electricity wasted on inefficient power adapters and devices in standby mode is irrelevant: The marginal savings per household are cents per year. If it doesn't hurt me as an end user (and I don't see how HTTP/2 does), what's the objection here? Nor is any smaller service provider forced to adopt it server-side.
    - CyberRabbi 6y ago
      The premise of my argument is that the end user is barely affected either way. There is potential harm to the web ecosystem by making the stack more complex and I am questioning the rationale for tolerating that harm is since these fundamental changes are being mostly driven by the specific use cases of large corporation, not the needs of the greater community. You’re right no one is forced to adopt this new web stack and for that I am grateful. Given recent events I will likely not adopt google’s web stack.
- littlecranky67 6y ago
  > so that a missed packet in one request doesn't slow down simultaneous requests This is already true for HTTP/1.1. The browser uses 6 parallel TCP connections to fetch resources, and a packet los in one of the TCP connections wouldn't stall the other ones. QUIC only fixes this for HTTP/2.0 which uses a single TCP connection with multiplexed HTTP streams; a single packet loss on the TCP level would stall all multiplexed streams on an OS level.
  - M2Ys4U 6y ago
    But HTTP/1.1 using multiple TCP streams means that, where there are a lot of small requests, there's a lot of bandwidth left on the table because the TCP window never grows.
    - littlecranky67 6y ago
      Late with a reply, but want to note that this is not true. Most OSes scale tcp congestion control windows (snd_cwnd) based on the target IP, that is any increase of the window in any tcp connection to the same host increases the window for all open connections. Even if you close all connections, the kernel caches the last known window size for this host and re-uses this on subsequent connections for a given timeframe.
tannhaeuser 6y ago
Indeed, and I won't be surprised to hear that the other advertised benefit of http/2 and beyond (avoid head of line blocking) turns out to be marginal at best, and just as difficult to take advantage of. I've got to ask, what was all the buzz about then, and why was it necessary to "improve" a simple text protocol into a giant ball of binary protocols replacing and circumventing regular TCP/IP. In combination with subverting web standardization, I can only assume it was a deliberate effort to deprive the web of its open nature, by making it so fscking complicated that only Google is capable to implement it. Same with Kubernetes, which turns out just as the one ring to bind them all, and enslave humanity into "the cloud". I have a suspicion a coming generation will not look favorably to Google tech in a post-SV era, who managed to subvert RFC and just about every other org we had to keep the net sane.
- CyberRabbi 6y ago
  It saves google a very large amount of money if every chrome browser opens http3 connections to google.com on startup instead of the layered stack. You don’t see much of an effort to deprecate the layered stack and in my opinion that’s because standardizing http3 is enough to lower their costs, what other server hosts do is irrelevant to their bottom line.
- chrismorgan 6y ago
  You’re getting the narrative quite wrong here and assuming a great deal of bad faith that simply doesn’t exist. People often seem to think Google presented SPDY and QUIC to IETF as faits accomplis, and they were just adopted as HTTP/2 and HTTP/3 because Google said so. This is not how the standardisation processes work. Rather: (1) many involved parties recognised that something like this would be of value; (2) Google developed something, because they happened to be one of the parties that cared the most about it; (3) Google gave it to IETF; (4) all the relevant stakeholders joined in on improving it until there was consensus and practical experience that it really was worth it; and (5) it was finalised and published. What ends up being standardised normally has some quite major differences from what was initially proposed. IETF QUIC is definitely quite different from gQUIC, as HTTP/2 is from SPDY. Standardisation within IETF brings diverse parties together to improve things based upon their experience and expertise. Certainly there will be dissenters, because there are trade-offs everywhere (e.g. the Varnish author was lukewarm about HTTP/2, reckoning it wasn’t worth it and that more radical changes should be made to HTTP semantics), but the end result will be better than what was initially presented, and there must be broad consensus that what is to be published is better than what preceded it (in this case HTTP/1.1). At Fastmail I observed a fair bit of the process of the standardisation of JMAP at IETF, and it benefited enormously from the process, changing shape quite significantly in some areas from what Fastmail initially presented. The end result of HTTP/2 is definitely harder to implement than HTTP/1 (though it’s still not too bad—I implemented it in the draft days and didn’t have any real trouble, there was mostly just more to implement than with HTTP/1), but of its operational parameters, it’s better in every way than HTTP/1. Turns out that a protocol being plain-text really just isn’t useful, so long as the semantics are conveyed—literally the only people that need to care about the wire protocol are the people making tools that speak it (that is, HTTP libraries). There’s really just one issue with HTTP/2: it makes it possible to use a single TCP connection where HTTP/1 probably used up to six, but this leads to TCP head-of-line blocking issues becoming more serious. And make no mistake, TCP head-of-line blocking is a real issue on high-loss networks like the outskirts of wi-fi and cellular networks. HTTP/2’s multiplexing solved real problems, and HTTP/3’s HOLB-fixing solves the last real problem with HTTP/2. Google didn’t browbeat people into doing their will; rather, they presented a draft, and then everyone worked together to improve upon that draft, and they all (or at the least, almost all) agreed that the end result was a good improvement. > a deliberate effort to deprive the web of its own nature by making it so complicated that only Google is capable to implement it. … are you aware of how many HTTP/3 implementations there are already? In Rust alone, there are at least three fairly mature implementations: Quinn by various people, Quiche by Cloudflare, Neqo by Mozilla; as well as a handful more not-so-mature implementations. Look, I’m not fond of Google, and I do think they abuse their position in many and various places, but this is not one of them.
LoSboccacc 6y ago
> google is just arbitrarily deciding what they want the web stack to look like well yes they do try but not arbitrarily, with them running as much infra they do and margin being as they are, a bit here and there on the stack can dramatically influence their bottom line.

AgentME 6y ago

HTTP/2 and HTTP/3 are both very useful even without server push. HTTP/2 fixed the issues of HTTP pipelining and made it so multiple requests could share one connection without issues like head-of-line blocking, and HTTP/3 improves things further so that a missed packet in one request doesn't slow down simultaneous requests. Server push was never a critical feature of them. The measured improvements from 2 and 3 were almost always without server push.

CyberRabbi 6y ago
You can open multiple tcp connections in parallel to virtually eliminate the head of line blocking problem for the vast majority of experiences. The problem has been exaggerated for the sake of justifying http2/3 but in practical terms it barely affects the average user experience. 80% optimal was good enough. On the other hand it reduces cost by a few percentage points for the vanishingly few web hosts large enough such that that would have a meaningful financial impact.
- vhiremath4 6y ago
  There is a cost to opening a new connection and having to re-incur multiple roundtrips in order to complete a new TCP and TLS handshake. This is a meaningful overhead savings for folks with a network setup that has high last-mile latency.
  - CyberRabbi 6y ago
    As has been demonstrated by my original post, clearly saving a few rtt is irrelevant here. Anyway your point is halfway-irrelevant since if you initiate all connections in parallel you only suffer the rtt latency of a single connection. The only people saving here are the people running servers that serve world-scale numbers of requests. The average user notices nearly no difference.
    - alexchamberlain 6y ago
      The world-scale companies are also the companies writing the standards.
      - CyberRabbi 6y ago
        Funny that
    - vhiremath4 6y ago
      I'm not sure I fully follow here, but I would like to be able to. Created a video response: https://www.loom.com/share/4539869f41cf47c5a154a342308584de https://www.loom.com/share/4539869f41cf47c5a154a342308584de
      - CyberRabbi 6y ago
        Hehe I engage with people like that all the time and I think I’ve learned a lot. In my humble opinion the people who are easily offended aren’t really worth discussing difficult problems with. Actually I find that I learn a lot more from people who don’t get hung up when their ego happens to unintentionally get bruised. I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push.
        vhiremath4 6y ago
        My ego is not bruised. As I said: > I'm not sure I fully follow here, but I would like to be able to What I'm calling out is that you may have fallen into the trap (as I did years back) that you need to be an asshole to drive towards solving hard problems. You don't, and it actually blocked me from creating the psychological safety necessary to get good ideas out of people who otherwise wouldn't feel comfortable speaking up. Genuinely good/supporting vibes from my direction to you - do with it what you will, but I do believe your ego is showing here more than mine. We can agree to disagree since I believe each of us will over-index on our past life experiences vs. those of some random person on Hacker News. > I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push. I think I get what you're saying, but honestly the way you're wording your points is a bit confusing. I am going to chalk this up to "I'm just too dumb" for sake of laziness at this point.
- TwoBit 6y ago
  Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?
  - CyberRabbi 6y ago
    I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.
    - jlokier 6y ago
      It's actually not trivial to optimize dynamic prioritisation across multiple connections. To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence. But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps. It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial. With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots. With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine. Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.
      - CyberRabbi 6y ago
- raxxorrax 6y ago
  But you would have to implement it on a layer that maybe should be exempt from dealing with network loading. I agree that the benefits only fit certain use cases, but it is still nice to have it and it is already widely adapted.
- lxgr 6y ago
  > vanishingly few web hosts large enough such that that would have a meaningful financial impact By that logic, electricity wasted on inefficient power adapters and devices in standby mode is irrelevant: The marginal savings per household are cents per year. If it doesn't hurt me as an end user (and I don't see how HTTP/2 does), what's the objection here? Nor is any smaller service provider forced to adopt it server-side.
  - CyberRabbi 6y ago
    The premise of my argument is that the end user is barely affected either way. There is potential harm to the web ecosystem by making the stack more complex and I am questioning the rationale for tolerating that harm is since these fundamental changes are being mostly driven by the specific use cases of large corporation, not the needs of the greater community. You’re right no one is forced to adopt this new web stack and for that I am grateful. Given recent events I will likely not adopt google’s web stack.
littlecranky67 6y ago
> so that a missed packet in one request doesn't slow down simultaneous requests This is already true for HTTP/1.1. The browser uses 6 parallel TCP connections to fetch resources, and a packet los in one of the TCP connections wouldn't stall the other ones. QUIC only fixes this for HTTP/2.0 which uses a single TCP connection with multiplexed HTTP streams; a single packet loss on the TCP level would stall all multiplexed streams on an OS level.
- M2Ys4U 6y ago
  But HTTP/1.1 using multiple TCP streams means that, where there are a lot of small requests, there's a lot of bandwidth left on the table because the TCP window never grows.
  - littlecranky67 6y ago
    Late with a reply, but want to note that this is not true. Most OSes scale tcp congestion control windows (snd_cwnd) based on the target IP, that is any increase of the window in any tcp connection to the same host increases the window for all open connections. Even if you close all connections, the kernel caches the last known window size for this host and re-uses this on subsequent connections for a given timeframe.

tannhaeuser 6y ago

Indeed, and I won't be surprised to hear that the other advertised benefit of http/2 and beyond (avoid head of line blocking) turns out to be marginal at best, and just as difficult to take advantage of. I've got to ask, what was all the buzz about then, and why was it necessary to "improve" a simple text protocol into a giant ball of binary protocols replacing and circumventing regular TCP/IP. In combination with subverting web standardization, I can only assume it was a deliberate effort to deprive the web of its open nature, by making it so fscking complicated that only Google is capable to implement it. Same with Kubernetes, which turns out just as the one ring to bind them all, and enslave humanity into "the cloud". I have a suspicion a coming generation will not look favorably to Google tech in a post-SV era, who managed to subvert RFC and just about every other org we had to keep the net sane.

CyberRabbi 6y ago
It saves google a very large amount of money if every chrome browser opens http3 connections to google.com on startup instead of the layered stack. You don’t see much of an effort to deprecate the layered stack and in my opinion that’s because standardizing http3 is enough to lower their costs, what other server hosts do is irrelevant to their bottom line.
chrismorgan 6y ago
You’re getting the narrative quite wrong here and assuming a great deal of bad faith that simply doesn’t exist. People often seem to think Google presented SPDY and QUIC to IETF as faits accomplis, and they were just adopted as HTTP/2 and HTTP/3 because Google said so. This is not how the standardisation processes work. Rather: (1) many involved parties recognised that something like this would be of value; (2) Google developed something, because they happened to be one of the parties that cared the most about it; (3) Google gave it to IETF; (4) all the relevant stakeholders joined in on improving it until there was consensus and practical experience that it really was worth it; and (5) it was finalised and published. What ends up being standardised normally has some quite major differences from what was initially proposed. IETF QUIC is definitely quite different from gQUIC, as HTTP/2 is from SPDY. Standardisation within IETF brings diverse parties together to improve things based upon their experience and expertise. Certainly there will be dissenters, because there are trade-offs everywhere (e.g. the Varnish author was lukewarm about HTTP/2, reckoning it wasn’t worth it and that more radical changes should be made to HTTP semantics), but the end result will be better than what was initially presented, and there must be broad consensus that what is to be published is better than what preceded it (in this case HTTP/1.1). At Fastmail I observed a fair bit of the process of the standardisation of JMAP at IETF, and it benefited enormously from the process, changing shape quite significantly in some areas from what Fastmail initially presented. The end result of HTTP/2 is definitely harder to implement than HTTP/1 (though it’s still not too bad—I implemented it in the draft days and didn’t have any real trouble, there was mostly just more to implement than with HTTP/1), but of its operational parameters, it’s better in every way than HTTP/1. Turns out that a protocol being plain-text really just isn’t useful, so long as the semantics are conveyed—literally the only people that need to care about the wire protocol are the people making tools that speak it (that is, HTTP libraries). There’s really just one issue with HTTP/2: it makes it possible to use a single TCP connection where HTTP/1 probably used up to six, but this leads to TCP head-of-line blocking issues becoming more serious. And make no mistake, TCP head-of-line blocking is a real issue on high-loss networks like the outskirts of wi-fi and cellular networks. HTTP/2’s multiplexing solved real problems, and HTTP/3’s HOLB-fixing solves the last real problem with HTTP/2. Google didn’t browbeat people into doing their will; rather, they presented a draft, and then everyone worked together to improve upon that draft, and they all (or at the least, almost all) agreed that the end result was a good improvement. > a deliberate effort to deprive the web of its own nature by making it so complicated that only Google is capable to implement it. … are you aware of how many HTTP/3 implementations there are already? In Rust alone, there are at least three fairly mature implementations: Quinn by various people, Quiche by Cloudflare, Neqo by Mozilla; as well as a handful more not-so-mature implementations. Look, I’m not fond of Google, and I do think they abuse their position in many and various places, but this is not one of them.

LoSboccacc 6y ago

> google is just arbitrarily deciding what they want the web stack to look like well yes they do try but not arbitrarily, with them running as much infra they do and margin being as they are, a bit here and there on the stack can dramatically influence their bottom line.