5 ms·

The obnoxious thing is that overly aggressive firewalls have killed any IP protocols that are not TCP or UDP. Even ICMP is often blocked or partially blocked.

The obnoxious thing is that overly aggressive firewalls have killed any IP protocols that are not TCP or UDP. Even ICMP is often blocked or partially blocked. In the mean time we could have had nice things: https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr... SCTP would be a fantastic protocol for HTTP/HTTPS. Pipelining, multi homing, multi streaming, oh my.

Hydraulix989 1y ago
Not sure I follow. QUIC is just UDP with an extra header in the opaque payload. To a firewall, it looks just like UDP.
- jabiko 1y ago
  That's the point he is making. QUIC has to be based on UDP because the networking stack is ossified enough to not allow the addition of any new Layer 4 protocol. It's not a huge drawback though.
  - Hydraulix989 1y ago
    It's an eight byte penalty for things needed for QUIC anyways.
- p_l 1y ago
  It exists partially because of ossification of protocols that killed SCTP. As in, we wouldn't have to implement it on top of UDP
- sedawkgrep 1y ago
  I think they’re saying that due to how firewalls are deployed, everything end up either being built on tcp or udp, instead of using existing (or building new) layer four protocols more suited to solving the problem like sctp, et al. I’m not sure I agree though, because many firewalls already pass other protocols today, like GRE, IPSEC, etc.
  - IgorPartola 1y ago
    That is exactly my point, thank you for clarifying. And yes IPSEC had started forcing people to open up their firewalls. If I had my way though it would be the other way around: all IP protocol numbers except those specifically deemed obsolete or insecure should be allowed, including a range for user defined custom protocols. We really painted ourselves into a corner of 6s and 17s.
- jlokier 1y ago
  Their point is that's a sad design choice, caused by firewalls forcing QUIC to take a first-class TCP-like internet protocol and wrap it with encrypted headers inside UDP for the sole purpose of preventing firewalls from blocking it or making it break in myriad subtle ways. Even QUICs unencrypted header parts are designed to be difficult for intermediate network equipment to track and modify, because of their history of making other protocols over TCP unreliable by doing this. SCTP is another high-performance protocol that provides many of the same network and performance features as QUIC, but it has been around for ~25 years. It was widely implemented. It was even in Linux 2.4 and available for Windows XP and MacOS. WebRTC in browsers can use SCTP, which makes sense (it predates QUIC) but is actually a bit of a problem if you want wide interoperability with WebRTC. But despite SCTP being around and widely implemented, and doing many of the good things QUIC does, SCTP was never widely adopted, nor seriously considered for HTTP, mainly because of the problem of firewalls. (And to be fair, some operating system API limitations). Basically, if TCP were invented today, it could not be deployed succesfully over much of the internet unless it was wrapped in UDP and encrypted. UDP-with-encryption is now playing the role that the IP protocol is supposed to play, just because too many firewalls block nearly every IP packet that doesn't have type TCP or UDP. If UDP is used but without encryption for a popular protocol, eventually too many firewalls deep-packet-inspect that UDP protocol too, and do the same kinds of blocking, tampering or making protocols break. This sad problem is called protocol ossification. As Google's AI puts it: "QUIC is the first IETF transport protocol to deliberately minimise its wire image to avoid ossification.". See also https://en.wikipedia.org/wiki/Protocol_ossification https://en.wikipedia.org/wiki/Protocol_ossification and https://http3-explained.haxx.se/en/why-quic/why-ossification https://http3-explained.haxx.se/en/why-quic/why-ossification There are some downsides to QUIC's ossification resistance. I did some work for a mobile equipment manufacter who adaptively manage the queues for multiple TCP streams from different users and different applications from the same user, to help ensure a better, fairer and lower-latency experience. QUICs properties made it difficult to measure what's going on at the protocol level, forcing the queue management to use crude statistical estimators instead, which don't work well on bursty protocols.
  - IgorPartola 1y ago
    Couldn’t have said it better myself.
tptacek 1y ago
Why would you need another IP protocol besides UDP? Anything you can do directly under an IP header, you can do under a UDP header as well, and the UDP header itself is tiny. Going back to David Reed, this is specifically why UDP exists: as the extension interface to build more non-TCP transport protocols.
- mtoner23 1y ago
  Because it's an extra header. Making data transfer that much less efficient and working to make sure that clients can decide it properly
  - lxgr 1y ago
    An extra header of 8 bytes. To put that into perspective, IPv6 added 20 bytes to the IP header compared to v4.
  - thadt 1y ago
    Yeah? It's an eight byte header. The OS needs something to tag IP packets to get them delivered to the correct application. So you're thinking maybe a four byte header for 50% savings here?
    - lxgr 1y ago
      Good point on there needing to be some application-level addressing anyway. On top of that, I believe the UDP checksum can be omitted as well at least on some OSes (and is arguably not necessary for fully encrypted/authenticated payloads) – leaving really just the two bytes for the "length field".
      - tptacek 1y ago
        You can't omit the UDP checksum in IPv6.
    - IgorPartola 1y ago
      So we have a checksum of the IP header, a checksum of the UDP header and a port number, an application level stream ID or message ID or whatever the application transport protocol is using, and finally almost certainly an even higher level message ID such as a URI. And that’s before you introduce encryption into it with all that overhead. A level 4 protocol providing full integrity verification, encryption, multi homing, multiplexing, out of band control, and control over transmission reliability would be amazing. But the only way you can experiment with these things is if you use UDP and ports. We take the concept of ports for granted but if you think of ICMP or some other L4 protocols that isn’t the only way to identify the sending and receiving application. If we just allowed all L4 protocol numbers through and ditched NAT we could have nice things. Or we could kick it up two layers to use QUIC for what SCTP could have been.
fulafel 1y ago
SCTP had some other nails to its coffin too. For example it came from the telco world and so had no end user app buyin and poor dev experience.
lxgr 1y ago
If you like SCPT, nobody's stopping you from using it today! Just put it in a minimal UDP wrapper. WebRTC has been doing just that for peer to peer data connections (which need TCP-like semantics while traversing NATs via UDP hole punching).
- IgorPartola 1y ago
  I get today’s realities of it but I would have preferred a world where IPv6 killed NAT and middleboxes properly supported more protocols than TCP and UDP. The original intent of IP was to have more than two protocols built on top of it. Many were built and deployed but then killed by IPv4 address exhaustion and NAT, as well as poorly configured firewalls and middleboxes that specifically wanted to mess with level 4 traffic. UDP is a good solution but all it does is provide an 8 byte overhead and nothing that IP itself doesn’t provide for something like SCTP.
  - lxgr 1y ago
    IPv6 doesn't provide a length header, so that's already 2 bytes arguably necessary for all protocols layered on top of that. Source and destination port just seem like a reasonable baseline for alternate protocols, that's 4 more – leaving just the checksum. (If you're really desperate for space and have integrity provided by your protocol, you can even cram two more bytes in there!) Sure, it would be conceptually nice to be able to skip UDP, but I think in terms of performance it absolutely does not matter.
    - IgorPartola 1y ago
      QUIC doesn’t use the UDP length header to designate message length, does it? But my point isn’t even about performance. It is about the fact that NAT and IPv4 address exhaustion and bad firewall practices have killed any innovation of level 4 protocols. Imagine if instead of TCP, SCTP had won the protocol wars in 1980-1990s. Or even better if we had realized that we were going to run out of IPv4 addresses much earlier when the cost of switching was smaller. It would have been so much better to have firewalls that don’t filter anything but protocols 6 and 17. We could have had the opportunity to experiment with different types of transports, baked encryption in at a lower level, etc. Basically where we are is that we have 6 and 8 dot LEGO bricks to play with and are told that we can build anything with those but aren’t allowed to play with any other shapes.
CuriouslyC 1y ago
SRT is hot stuff over UDP: https://www.haivision.com/products/srt-secure-reliable-transport/ https://www.haivision.com/products/srt-secure-reliable-trans... But even UDP is heavily restricted in most cases.
- lxgr 1y ago
  Is it really? It's been a while since I've been on a network that actually filters UDP. Too many things that people actually need to work run over it these days, including VPNs, videoconferencing etc.
yobid20 1y ago
^^^^ this. I work for a big company (15k engineers). Trying to use anything that is not TCP or UDP simply doesnt work here. For years, even UDP was blocked and the answer we got was always "why are you using UDP, use TCP instead". Yep you read that right. Most of these folks are very short sighted or narrow minded. We tried to use SCTP for one project, major blunder. Zero support from network teams. Sctp is blocked everywhere. All their custom software and scripts for network deployments only work with tcp and udp. And they will not change that. And that comes from higher ups the ppl in charge. They are set in there ways and will not budge. As for as QUIC support? Never gonna happen here.
- lxgr 1y ago
  > Trying to use anything that is not TCP or UDP simply doesnt work here. Good news, then: QUIC uses UDP. > As for as QUIC support? Never gonna happen here. So generic UDP is allowed, but QUIC is specifically detected and filtered? That would be bizarre.
  - IgorPartola 1y ago
    I have also had to deal with UDP getting blocked by a middle box before. It is rare now across the internet but you never know what intranet grey beards can do. And UDP hole punching is a crutch for the fact that we still use IPv4.
- UltraSane 1y ago
  I'm one of a few people who can say they used SCTP for its original purpose, SIGTRAN transmission of Signaling System 7 call setup data over IPSec tunnels. It was really cool seeing SS7 packets in the wireshark dumps.

Hydraulix989 1y ago

Not sure I follow. QUIC is just UDP with an extra header in the opaque payload. To a firewall, it looks just like UDP.

jabiko 1y ago
That's the point he is making. QUIC has to be based on UDP because the networking stack is ossified enough to not allow the addition of any new Layer 4 protocol. It's not a huge drawback though.
- Hydraulix989 1y ago
  It's an eight byte penalty for things needed for QUIC anyways.
p_l 1y ago
It exists partially because of ossification of protocols that killed SCTP. As in, we wouldn't have to implement it on top of UDP
sedawkgrep 1y ago
I think they’re saying that due to how firewalls are deployed, everything end up either being built on tcp or udp, instead of using existing (or building new) layer four protocols more suited to solving the problem like sctp, et al. I’m not sure I agree though, because many firewalls already pass other protocols today, like GRE, IPSEC, etc.
- IgorPartola 1y ago
  That is exactly my point, thank you for clarifying. And yes IPSEC had started forcing people to open up their firewalls. If I had my way though it would be the other way around: all IP protocol numbers except those specifically deemed obsolete or insecure should be allowed, including a range for user defined custom protocols. We really painted ourselves into a corner of 6s and 17s.
jlokier 1y ago
Their point is that's a sad design choice, caused by firewalls forcing QUIC to take a first-class TCP-like internet protocol and wrap it with encrypted headers inside UDP for the sole purpose of preventing firewalls from blocking it or making it break in myriad subtle ways. Even QUICs unencrypted header parts are designed to be difficult for intermediate network equipment to track and modify, because of their history of making other protocols over TCP unreliable by doing this. SCTP is another high-performance protocol that provides many of the same network and performance features as QUIC, but it has been around for ~25 years. It was widely implemented. It was even in Linux 2.4 and available for Windows XP and MacOS. WebRTC in browsers can use SCTP, which makes sense (it predates QUIC) but is actually a bit of a problem if you want wide interoperability with WebRTC. But despite SCTP being around and widely implemented, and doing many of the good things QUIC does, SCTP was never widely adopted, nor seriously considered for HTTP, mainly because of the problem of firewalls. (And to be fair, some operating system API limitations). Basically, if TCP were invented today, it could not be deployed succesfully over much of the internet unless it was wrapped in UDP and encrypted. UDP-with-encryption is now playing the role that the IP protocol is supposed to play, just because too many firewalls block nearly every IP packet that doesn't have type TCP or UDP. If UDP is used but without encryption for a popular protocol, eventually too many firewalls deep-packet-inspect that UDP protocol too, and do the same kinds of blocking, tampering or making protocols break. This sad problem is called protocol ossification. As Google's AI puts it: "QUIC is the first IETF transport protocol to deliberately minimise its wire image to avoid ossification.". See also https://en.wikipedia.org/wiki/Protocol_ossification https://en.wikipedia.org/wiki/Protocol_ossification and https://http3-explained.haxx.se/en/why-quic/why-ossification https://http3-explained.haxx.se/en/why-quic/why-ossification There are some downsides to QUIC's ossification resistance. I did some work for a mobile equipment manufacter who adaptively manage the queues for multiple TCP streams from different users and different applications from the same user, to help ensure a better, fairer and lower-latency experience. QUICs properties made it difficult to measure what's going on at the protocol level, forcing the queue management to use crude statistical estimators instead, which don't work well on bursty protocols.
- IgorPartola 1y ago
  Couldn’t have said it better myself.

tptacek 1y ago

Why would you need another IP protocol besides UDP? Anything you can do directly under an IP header, you can do under a UDP header as well, and the UDP header itself is tiny. Going back to David Reed, this is specifically why UDP exists: as the extension interface to build more non-TCP transport protocols.

mtoner23 1y ago
Because it's an extra header. Making data transfer that much less efficient and working to make sure that clients can decide it properly
- lxgr 1y ago
  An extra header of 8 bytes. To put that into perspective, IPv6 added 20 bytes to the IP header compared to v4.
- thadt 1y ago
  Yeah? It's an eight byte header. The OS needs something to tag IP packets to get them delivered to the correct application. So you're thinking maybe a four byte header for 50% savings here?
  - lxgr 1y ago
    Good point on there needing to be some application-level addressing anyway. On top of that, I believe the UDP checksum can be omitted as well at least on some OSes (and is arguably not necessary for fully encrypted/authenticated payloads) – leaving really just the two bytes for the "length field".
    - tptacek 1y ago
      You can't omit the UDP checksum in IPv6.
  - IgorPartola 1y ago
    So we have a checksum of the IP header, a checksum of the UDP header and a port number, an application level stream ID or message ID or whatever the application transport protocol is using, and finally almost certainly an even higher level message ID such as a URI. And that’s before you introduce encryption into it with all that overhead. A level 4 protocol providing full integrity verification, encryption, multi homing, multiplexing, out of band control, and control over transmission reliability would be amazing. But the only way you can experiment with these things is if you use UDP and ports. We take the concept of ports for granted but if you think of ICMP or some other L4 protocols that isn’t the only way to identify the sending and receiving application. If we just allowed all L4 protocol numbers through and ditched NAT we could have nice things. Or we could kick it up two layers to use QUIC for what SCTP could have been.

fulafel 1y ago

SCTP had some other nails to its coffin too. For example it came from the telco world and so had no end user app buyin and poor dev experience.

lxgr 1y ago

If you like SCPT, nobody's stopping you from using it today! Just put it in a minimal UDP wrapper. WebRTC has been doing just that for peer to peer data connections (which need TCP-like semantics while traversing NATs via UDP hole punching).

IgorPartola 1y ago
I get today’s realities of it but I would have preferred a world where IPv6 killed NAT and middleboxes properly supported more protocols than TCP and UDP. The original intent of IP was to have more than two protocols built on top of it. Many were built and deployed but then killed by IPv4 address exhaustion and NAT, as well as poorly configured firewalls and middleboxes that specifically wanted to mess with level 4 traffic. UDP is a good solution but all it does is provide an 8 byte overhead and nothing that IP itself doesn’t provide for something like SCTP.
- lxgr 1y ago
  IPv6 doesn't provide a length header, so that's already 2 bytes arguably necessary for all protocols layered on top of that. Source and destination port just seem like a reasonable baseline for alternate protocols, that's 4 more – leaving just the checksum. (If you're really desperate for space and have integrity provided by your protocol, you can even cram two more bytes in there!) Sure, it would be conceptually nice to be able to skip UDP, but I think in terms of performance it absolutely does not matter.
  - IgorPartola 1y ago
    QUIC doesn’t use the UDP length header to designate message length, does it? But my point isn’t even about performance. It is about the fact that NAT and IPv4 address exhaustion and bad firewall practices have killed any innovation of level 4 protocols. Imagine if instead of TCP, SCTP had won the protocol wars in 1980-1990s. Or even better if we had realized that we were going to run out of IPv4 addresses much earlier when the cost of switching was smaller. It would have been so much better to have firewalls that don’t filter anything but protocols 6 and 17. We could have had the opportunity to experiment with different types of transports, baked encryption in at a lower level, etc. Basically where we are is that we have 6 and 8 dot LEGO bricks to play with and are told that we can build anything with those but aren’t allowed to play with any other shapes.

CuriouslyC 1y ago

SRT is hot stuff over UDP: https://www.haivision.com/products/srt-secure-reliable-transport/ https://www.haivision.com/products/srt-secure-reliable-trans... But even UDP is heavily restricted in most cases.

lxgr 1y ago
Is it really? It's been a while since I've been on a network that actually filters UDP. Too many things that people actually need to work run over it these days, including VPNs, videoconferencing etc.

yobid20 1y ago

^^^^ this. I work for a big company (15k engineers). Trying to use anything that is not TCP or UDP simply doesnt work here. For years, even UDP was blocked and the answer we got was always "why are you using UDP, use TCP instead". Yep you read that right. Most of these folks are very short sighted or narrow minded. We tried to use SCTP for one project, major blunder. Zero support from network teams. Sctp is blocked everywhere. All their custom software and scripts for network deployments only work with tcp and udp. And they will not change that. And that comes from higher ups the ppl in charge. They are set in there ways and will not budge. As for as QUIC support? Never gonna happen here.

lxgr 1y ago
> Trying to use anything that is not TCP or UDP simply doesnt work here. Good news, then: QUIC uses UDP. > As for as QUIC support? Never gonna happen here. So generic UDP is allowed, but QUIC is specifically detected and filtered? That would be bizarre.
- IgorPartola 1y ago
  I have also had to deal with UDP getting blocked by a middle box before. It is rare now across the internet but you never know what intranet grey beards can do. And UDP hole punching is a crutch for the fact that we still use IPv4.
UltraSane 1y ago
I'm one of a few people who can say they used SCTP for its original purpose, SIGTRAN transmission of Signaling System 7 call setup data over IPSec tunnels. It was really cool seeing SS7 packets in the wireshark dumps.