9 ms·

The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. https://http3-explained.haxx.se/en/why-quic

The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. https://http3-explained.haxx.se/en/why-quic/why-ossification https://http3-explained.haxx.se/en/why-quic/why-ossification Many have optimisations around TCP and long-standing TCP characteristics, which they are able to do because the TCP headers are unencrypted. This led to TCP being difficult to improve, because of all the implementations out there making assumptions and doing these optimisations that weren’t compatible with new developments, or worse, dropping traffic with new, unrecognised TCP options. In comparison QUIC is almost 100% encrypted, and exposes comparatively little data for routers to ossify on. So even if network cards start optimising, they can only do so much damage because they are given very little information. For example, there is no way for them to do their own special per-stream flow control, because the stream identifier is encrypted. The only things visible are the UDP src/dest port, the QUIC connection ID, and a minimal set of flags. Most important is that the flow control information is encrypted, which is one of the main things TCP couldn’t improve on due to ossification. That’s not to say the number of implementations stops being a problem — just means that you can actually use QUIC version numbers and negotiate features at the endpoints and ignore anybody along the routes. So you can get adoption of new ideas much more quickly. Quick illustration of the things no longer visible to network operators: https://blog.apnic.net/wp-content/uploads/2019/03/quic-fig2.png https://blog.apnic.net/wp-content/uploads/2019/03/quic-fig2...., from https://blog.apnic.net/2019/03/04/a-quick-look-at-quic/ https://blog.apnic.net/2019/03/04/a-quick-look-at-quic/

zozbot234 5y ago
> The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. Being 'opaque' to the network is not a foolproof choice, since it also makes stuff like NAT, congestion control and QoS a lot less robust. By exposing connection identifiers to the network, SCTP gains a lot of flexibility wrt. such scenarios compared to QUIC.
- capableweb 5y ago
  Correct me if I'm wrong, but bypassing NAT, congestion control and QoS is also a benefit when you want to build things like P2P services where you want a direct connection between peers and full control over the connection itself. QUIC will help a lot with those use cases.
- 10000truths 5y ago
  NAT is a hack that only ever existed due to IP address exhaustion, and I’ve yet to find a good reason why congestion control and QoS handling in a router needs anything beyond what is already in the IP headers. SCTP’s flexibility is irrelevant since nobody uses it, arguably because of how much the Internet has ossified around TCP and UDP.
  - Orphis 5y ago
    Hey, that's not true. People use it... in the telecom industry and everyone using WebRTC's DataChannel (SCTP over DTLS actually). The protocol is probably mostly fine, but there are no quality library for using it in a general purpose way. Because of that, no one can really use the protocol to its full designed capabilities anyway (and the ossified network nodes don't help either).
- salawat 5y ago
  I'm happy whenever anyone brings up SCTP. Such a maligned protocol. Congestion control and QoS, however, are a loser's game. If you're running into realistic issues meeting demand due to statistical multiplexing breaking down to the point you must rely on caring about the internal state of traffic streams, you should have thrown more link hardware at the problem long ago. Many dumb pipes > fewer smarter pipes. >NAT Oh, didn't you know, IPv6 was going to solve everything. No more NAT. We're just going to assign everyone an IP at birth so people can continue to ignore the User-Agent/User distinction.
  - Hikikomori 5y ago
    QoS has its uses, and you don't have to look any deeper than TCP/UDP headers for it to be good enough. It allows VOIP/IPTV services to continue working under DDoS conditions for example (ISP level).
    - richardwhiuk 5y ago
      IP TOS flag and TCI field in Ethernet header (802.1Q header) should be sufficient, without looking at TCP/UDP/SCTP headers.
- Orphis 5y ago
  It doesn't matter if SCTP has good features if no one is willing to implement quality libraries for it. The protocol exists for a while and only one general purpose library exists for it that's widely used. And it's not without a lot of flaws and limitations. There are good reasons no one wants to invest in it, and I'm sure they did consider it.
  - Sean-Der 5y ago
    I am sure there are even more implementations that I am not aware of. * https://github.com/pion/sctp https://github.com/pion/sctp * https://github.com/aiortc/aiortc/blob/main/src/aiortc/rtcsctptransport.py https://github.com/aiortc/aiortc/blob/main/src/aiortc/rtcsct... * https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/net/dcsctp/ https://source.chromium.org/chromium/chromium/src/+/main:thi... * https://github.com/sctplab/usrsctp https://github.com/sctplab/usrsctp People don't make these decisions for technical reasons only. Career wise it is a bad choice to spend your time working on pre-existing technologies. You don't become a distinguished engineer by iterating on existing technologies. You become one by being the creator of something new. I think QUIC is great and does a good job solving the problems it was designed to solve. It is disingenuous to pretend these decisions were made only for technical reasons.
    - Orphis 5y ago
      3 of those are related to DataChannels in WebRTC, which at this point is baked into the standard and can't be replaced by another protocol. The SCTP in DataChannels is quite limited, that subset is reasonable to implement somehow. Only the last one is really general purpose and has had a lot of security issues as well. Fun fact: I do work on one of those implementations and with another one, and it's out of necessity rather than a career choice.
      - Sean-Der 5y ago
        I know you understand libusrsctp Florent. I created Pion, I feel prety comfortable with SCTP and WebRTC in general as well. RtcQuicTransport tried to replace SCTP and it didn't work. It would be possible to replace SCTP if something better was available. SCTP has lots of great stuff like FORWARD-TSN. QUIC didn't offer anything compelling over it. Also would come with a huge cost of making WebRTC larger and losing interop with all the existing clients. libusrsctp has had security issues, but I don't think the protocol is the problem. The issue is C/C++. QUIC implementations are going to have the same class of bugs. Chrome/libwebrtc has plenty of security issues in other areas besides SCTP. Rust/Go doesn't fix everything, but one less thing to worry about at least.
- jjoonathan 5y ago
  > it also makes stuff like NAT, congestion control and QoS a lot less robust That's a feature, not a bug. NAT is cancer. It killed the open internet by enabling a particular lazy form of "security" that forced everyone to hand control of their most important data to cloud services. QoS is worse but has less fallout due to less deployment.
  - ksec 5y ago
    >NAT is cancer. It killed the open internet by enabling a particular lazy form of "security" that forced everyone to hand control of their most important data to cloud services. Sorry I dont follow. What Cloud Services?
    - jjoonathan 5y ago
      All cloud services. Personal computers and devices are not allowed to talk to each other on the internet. They are only allowed to talk to specially designated servers -- specially designated by virtue of having a public IPv4 and open port. Enforcement happens through liability: if you put data in a cloud service and the cloud service gets hacked, it's the cloud service's fault, but if you run a program on an open port and the program gets hacked, it's your fault, not the program's fault. This is an arbitrary social choice that follows directly from enshrining NAT as a "best practice." Even the term "open port" presupposes that there is something unnatural about plebian computers talking to each other on the open internet! This effectively forces services to be centralized, or at least pushes very strongly in that direction. Of course, we can quibble: what about video conferencing? What about gaming? They use direct connections! Yes, but they pay an extraordinarily steep complexity and reliability price to obtain them and the solutions still wind up using centralized servers. They are exceptions that prove the rule because they allow us to observe how "stepping out of line" is discouraged. It's an accidental line drawn by monkies rather than an intentional line drawn by lizard people, but it's strongly enforced and society-shaping all the same. It's totally wild to consider the social and market consequences of NAT, the unassuming lazy and kludgy security hack. It got out of hand and reshaped the internet, the technology market, and society from the ground up! That's hardly even an exaggeration!
- fpoling 5y ago
  NAT does not need anything more than port numbers or UDP cannot be used with it. As for QoS going beyond port number, source IP and rate of packets asks for troubles.
  - BlueTemplar 5y ago
    I wonder : IPs are considered PI (heh). So does this mean that badly implemented QoS can end up violating RGPD ?
deleted 5y ago
[deleted]
throwawaaarrgh 5y ago
Ah, the old middleboxes excuse. Changing TCP would break the middleboxes, so instead we'll invent a brand new protocol which is (theoretically) ignored by the middleboxes. The idea being if you slip something in via UDP, nobody will notice, and then you encrypt it so the middleboxes give up on it. This assumes that lack of transparency is the only way forward. I'd argue that IP and TCP ossified not because of transparency, but because the design didn't make it easy enough to simultaneously remain compatible while adding new functionality. Yet we know a transparent, grokable, mungeable, backwards-and-forwards-compatible protocol that plays well with middle boxes is doable: HTTP. We could design lower-level protocols to have the same design properties, with both long-lived backwards compatibility and the ability to adopt new functionality. If the problem is with middleboxes, then the solution shouldn't be "how can we avoid them", but "how can we make a protocol that works well with them". (As a side note, a lack of transparency has chilling effects. Since the adoption of TLS v1.3, more and more TLS security has been broken in order to provide the necessary IT functions of traffic inspection, caching, routing, classifying, shaping, and blocking. Nation states will need to continue creating their own fragmented walled gardens, and continue invading privacy more, precisely because we're giving them less control over how they manage these functions which are a matter of national policy. You can't software you way out of a process problem.)
- overgard 5y ago
  > Yet we know a transparent, grokable, mungeable, backwards-and-forwards-compatible protocol that plays well with middle boxes is doable: HTTP Not all traffic is web traffic, maybe not even most of it. There are so many things that would have absolutely wretched performance if you tried to implement them with HTTP instead of TCP. HTTP is way too generic to be truly performant.
- bcoates 5y ago
  Nah, middleboxes are garbage. The security features they claim to offer are all snakeoil checkbox-ware, businesses and government agencies/state-controlled telcos use them because they have a nominal compliance obligation to do a thing so they buy a middlebox that they can wave their arms at when the auditor/commissar comes around. Generalized "optimizer" middleboxes don't actually work in real world performance testing, with the possible exception of machines that apply dumb global queue discipline to fix a defective buffering device somewhere else in the network, but these don't need to do packet inspection at all, just byte-counting/early-drop. Traffic shaping mostly only works in the endpoint, but aside from that, all you need is src/dest/byte count to do as good as traffic shaping as it is possible to do without being circumvented trivially (by the far end, without the users knowledge, thanks to the web letting the server script the client) If you want to run a totalitarian state, do what actually competent totalitarian states do and take over the endpoints.
KirillPanov 5y ago
Are deep packet inspection and modification boxes really that prevalent outside of the Great Firewall of China and its imitators (Russia has something similar now)? If QUIC gets in the way of these countries they will simply set up a decrypt-and-recrypt TLS proxy and block connections that don't honor its certificate. Lots of banks and trading firms already do this to their employees for regulatory purposes; there is no technical hurdle.
- throwawaaarrgh 5y ago
  > there is no technical hurdle As someone who has been forced to use ZScaler, there is a slew of technical challenges. 80% of the traffic can be handled "easily" with a custom CA cert, and the rest have varying issues ranging from client problems to server problems to problems in the proxy itself. Just trying to clone a GitHub repo over HTTPS breaks with ZScaler.
  - KirillPanov 5y ago
    I can't comment on this specific product, but if the transparent proxy is causing you so much grief why not just explicitly configure your machine to use the HTTPS proxy? HTTPS proxies are not voodoo. There are plenty that work quite well.
    - throwawaaarrgh 5y ago
      Actually there is "voodoo" in both transparent and non-transparent proxies, but the bigger problem is that ZScaler's entire raison d'etre is to be a transparent proxy using Carrier-grade NAT. But the point is, ZScaler is a widely-deployed commercial solution, and it does indeed have technical hurdles. Whether it should theoretically work or not, I'm telling you in practice there are problems, and this shouldn't be dismissed out of hand as an outlier.
      - KirillPanov 5y ago
        You really seem to have a beef with this one particular commercial product that I don't have access to. I've used plenty of other HTTPS proxies and the well-written ones all worked fine. Many are open source. I think you need to direct these comments to whoever is charging you money for this "Zscaler" thing.
DougN7 5y ago
But isn’t the “damage” these middleware layers do actually good optimizations/features that people pay for? The ossification is the problem, not the ability to optimize IMHO.

zozbot234 5y ago

> The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. Being 'opaque' to the network is not a foolproof choice, since it also makes stuff like NAT, congestion control and QoS a lot less robust. By exposing connection identifiers to the network, SCTP gains a lot of flexibility wrt. such scenarios compared to QUIC.

capableweb 5y ago
Correct me if I'm wrong, but bypassing NAT, congestion control and QoS is also a benefit when you want to build things like P2P services where you want a direct connection between peers and full control over the connection itself. QUIC will help a lot with those use cases.
10000truths 5y ago
NAT is a hack that only ever existed due to IP address exhaustion, and I’ve yet to find a good reason why congestion control and QoS handling in a router needs anything beyond what is already in the IP headers. SCTP’s flexibility is irrelevant since nobody uses it, arguably because of how much the Internet has ossified around TCP and UDP.
- Orphis 5y ago
  Hey, that's not true. People use it... in the telecom industry and everyone using WebRTC's DataChannel (SCTP over DTLS actually). The protocol is probably mostly fine, but there are no quality library for using it in a general purpose way. Because of that, no one can really use the protocol to its full designed capabilities anyway (and the ossified network nodes don't help either).
salawat 5y ago
I'm happy whenever anyone brings up SCTP. Such a maligned protocol. Congestion control and QoS, however, are a loser's game. If you're running into realistic issues meeting demand due to statistical multiplexing breaking down to the point you must rely on caring about the internal state of traffic streams, you should have thrown more link hardware at the problem long ago. Many dumb pipes > fewer smarter pipes. >NAT Oh, didn't you know, IPv6 was going to solve everything. No more NAT. We're just going to assign everyone an IP at birth so people can continue to ignore the User-Agent/User distinction.
- Hikikomori 5y ago
  QoS has its uses, and you don't have to look any deeper than TCP/UDP headers for it to be good enough. It allows VOIP/IPTV services to continue working under DDoS conditions for example (ISP level).
  - richardwhiuk 5y ago
    IP TOS flag and TCI field in Ethernet header (802.1Q header) should be sufficient, without looking at TCP/UDP/SCTP headers.
Orphis 5y ago
It doesn't matter if SCTP has good features if no one is willing to implement quality libraries for it. The protocol exists for a while and only one general purpose library exists for it that's widely used. And it's not without a lot of flaws and limitations. There are good reasons no one wants to invest in it, and I'm sure they did consider it.
- Sean-Der 5y ago
  I am sure there are even more implementations that I am not aware of. * https://github.com/pion/sctp https://github.com/pion/sctp * https://github.com/aiortc/aiortc/blob/main/src/aiortc/rtcsctptransport.py https://github.com/aiortc/aiortc/blob/main/src/aiortc/rtcsct... * https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/net/dcsctp/ https://source.chromium.org/chromium/chromium/src/+/main:thi... * https://github.com/sctplab/usrsctp https://github.com/sctplab/usrsctp People don't make these decisions for technical reasons only. Career wise it is a bad choice to spend your time working on pre-existing technologies. You don't become a distinguished engineer by iterating on existing technologies. You become one by being the creator of something new. I think QUIC is great and does a good job solving the problems it was designed to solve. It is disingenuous to pretend these decisions were made only for technical reasons.
  - Orphis 5y ago
    3 of those are related to DataChannels in WebRTC, which at this point is baked into the standard and can't be replaced by another protocol. The SCTP in DataChannels is quite limited, that subset is reasonable to implement somehow. Only the last one is really general purpose and has had a lot of security issues as well. Fun fact: I do work on one of those implementations and with another one, and it's out of necessity rather than a career choice.
    - Sean-Der 5y ago
      I know you understand libusrsctp Florent. I created Pion, I feel prety comfortable with SCTP and WebRTC in general as well. RtcQuicTransport tried to replace SCTP and it didn't work. It would be possible to replace SCTP if something better was available. SCTP has lots of great stuff like FORWARD-TSN. QUIC didn't offer anything compelling over it. Also would come with a huge cost of making WebRTC larger and losing interop with all the existing clients. libusrsctp has had security issues, but I don't think the protocol is the problem. The issue is C/C++. QUIC implementations are going to have the same class of bugs. Chrome/libwebrtc has plenty of security issues in other areas besides SCTP. Rust/Go doesn't fix everything, but one less thing to worry about at least.
jjoonathan 5y ago
> it also makes stuff like NAT, congestion control and QoS a lot less robust That's a feature, not a bug. NAT is cancer. It killed the open internet by enabling a particular lazy form of "security" that forced everyone to hand control of their most important data to cloud services. QoS is worse but has less fallout due to less deployment.
- ksec 5y ago
  >NAT is cancer. It killed the open internet by enabling a particular lazy form of "security" that forced everyone to hand control of their most important data to cloud services. Sorry I dont follow. What Cloud Services?
  - jjoonathan 5y ago
    All cloud services. Personal computers and devices are not allowed to talk to each other on the internet. They are only allowed to talk to specially designated servers -- specially designated by virtue of having a public IPv4 and open port. Enforcement happens through liability: if you put data in a cloud service and the cloud service gets hacked, it's the cloud service's fault, but if you run a program on an open port and the program gets hacked, it's your fault, not the program's fault. This is an arbitrary social choice that follows directly from enshrining NAT as a "best practice." Even the term "open port" presupposes that there is something unnatural about plebian computers talking to each other on the open internet! This effectively forces services to be centralized, or at least pushes very strongly in that direction. Of course, we can quibble: what about video conferencing? What about gaming? They use direct connections! Yes, but they pay an extraordinarily steep complexity and reliability price to obtain them and the solutions still wind up using centralized servers. They are exceptions that prove the rule because they allow us to observe how "stepping out of line" is discouraged. It's an accidental line drawn by monkies rather than an intentional line drawn by lizard people, but it's strongly enforced and society-shaping all the same. It's totally wild to consider the social and market consequences of NAT, the unassuming lazy and kludgy security hack. It got out of hand and reshaped the internet, the technology market, and society from the ground up! That's hardly even an exaggeration!
fpoling 5y ago
NAT does not need anything more than port numbers or UDP cannot be used with it. As for QoS going beyond port number, source IP and rate of packets asks for troubles.
- BlueTemplar 5y ago
  I wonder : IPs are considered PI (heh). So does this mean that badly implemented QoS can end up violating RGPD ?

deleted 5y ago

[deleted]

throwawaaarrgh 5y ago

Ah, the old middleboxes excuse. Changing TCP would break the middleboxes, so instead we'll invent a brand new protocol which is (theoretically) ignored by the middleboxes. The idea being if you slip something in via UDP, nobody will notice, and then you encrypt it so the middleboxes give up on it. This assumes that lack of transparency is the only way forward. I'd argue that IP and TCP ossified not because of transparency, but because the design didn't make it easy enough to simultaneously remain compatible while adding new functionality. Yet we know a transparent, grokable, mungeable, backwards-and-forwards-compatible protocol that plays well with middle boxes is doable: HTTP. We could design lower-level protocols to have the same design properties, with both long-lived backwards compatibility and the ability to adopt new functionality. If the problem is with middleboxes, then the solution shouldn't be "how can we avoid them", but "how can we make a protocol that works well with them". (As a side note, a lack of transparency has chilling effects. Since the adoption of TLS v1.3, more and more TLS security has been broken in order to provide the necessary IT functions of traffic inspection, caching, routing, classifying, shaping, and blocking. Nation states will need to continue creating their own fragmented walled gardens, and continue invading privacy more, precisely because we're giving them less control over how they manage these functions which are a matter of national policy. You can't software you way out of a process problem.)

overgard 5y ago
> Yet we know a transparent, grokable, mungeable, backwards-and-forwards-compatible protocol that plays well with middle boxes is doable: HTTP Not all traffic is web traffic, maybe not even most of it. There are so many things that would have absolutely wretched performance if you tried to implement them with HTTP instead of TCP. HTTP is way too generic to be truly performant.
bcoates 5y ago
Nah, middleboxes are garbage. The security features they claim to offer are all snakeoil checkbox-ware, businesses and government agencies/state-controlled telcos use them because they have a nominal compliance obligation to do a thing so they buy a middlebox that they can wave their arms at when the auditor/commissar comes around. Generalized "optimizer" middleboxes don't actually work in real world performance testing, with the possible exception of machines that apply dumb global queue discipline to fix a defective buffering device somewhere else in the network, but these don't need to do packet inspection at all, just byte-counting/early-drop. Traffic shaping mostly only works in the endpoint, but aside from that, all you need is src/dest/byte count to do as good as traffic shaping as it is possible to do without being circumvented trivially (by the far end, without the users knowledge, thanks to the web letting the server script the client) If you want to run a totalitarian state, do what actually competent totalitarian states do and take over the endpoints.

KirillPanov 5y ago

Are deep packet inspection and modification boxes really that prevalent outside of the Great Firewall of China and its imitators (Russia has something similar now)? If QUIC gets in the way of these countries they will simply set up a decrypt-and-recrypt TLS proxy and block connections that don't honor its certificate. Lots of banks and trading firms already do this to their employees for regulatory purposes; there is no technical hurdle.

throwawaaarrgh 5y ago
> there is no technical hurdle As someone who has been forced to use ZScaler, there is a slew of technical challenges. 80% of the traffic can be handled "easily" with a custom CA cert, and the rest have varying issues ranging from client problems to server problems to problems in the proxy itself. Just trying to clone a GitHub repo over HTTPS breaks with ZScaler.
- KirillPanov 5y ago
  I can't comment on this specific product, but if the transparent proxy is causing you so much grief why not just explicitly configure your machine to use the HTTPS proxy? HTTPS proxies are not voodoo. There are plenty that work quite well.
  - throwawaaarrgh 5y ago
    Actually there is "voodoo" in both transparent and non-transparent proxies, but the bigger problem is that ZScaler's entire raison d'etre is to be a transparent proxy using Carrier-grade NAT. But the point is, ZScaler is a widely-deployed commercial solution, and it does indeed have technical hurdles. Whether it should theoretically work or not, I'm telling you in practice there are problems, and this shouldn't be dismissed out of hand as an outlier.
    - KirillPanov 5y ago
      You really seem to have a beef with this one particular commercial product that I don't have access to. I've used plenty of other HTTPS proxies and the well-written ones all worked fine. Many are open source. I think you need to direct these comments to whoever is charging you money for this "Zscaler" thing.

DougN7 5y ago

But isn’t the “damage” these middleware layers do actually good optimizations/features that people pay for? The ossification is the problem, not the ability to optimize IMHO.