21 ms·

> The internet transport ecosystem has been ossified for decades now, and QUIC breaks out of this ossification But it's still just a layer on top of UDP, and s

> The internet transport ecosystem has been ossified for decades now, and QUIC breaks out of this ossification But it's still just a layer on top of UDP, and still implemented at the application, like in the past. So how is the ossification broken? Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities (same as for TLS, making fewer apps implement it without a lot of extra work). Which also increases context switching. In the future more protocols will be built on top of QUIC, expanding the user-space stack, increasing fragmentation of application-space IP stacks. And are network cards now going to start implementing it? It's painful to watch us stride headlong into the future depending on band-aids because surgery is too complicated.

wmf 5y ago
I think from their perspective the (Windows and Android) kernel is what causes ossification so moving transport into the app, along with encryption, solves it.
- DaiPlusPlus 5y ago
  You're forgetting the rest of userspace that exists between the application and the kernel. > moving transport into the app, along with encryption, solves it This is only a good idea if the application's development moves faster than both the kernel and the rest of the userspace platform - which is a fine strategy if you're targeting old or obsolete platforms, but probably a bad idea if you're targeting currently-supported operating systems with a long support lifetime (like Windows). It's a bad idea when a major security issue or compatibility problem with a transport or encryption library pops-up (which happens all the time, btw) - as it's part of your application then you don't benefit when the OS vendor ships a security update that your application gets for free. If you used dynamic-load+linking at runtime instead of a statically-linked encryption library then you might get lucky with a drop-in .so/.dll replacement without needing a rebuild, but YMMV. Doing everything in the app instead of letting the parent platform handle it by default is kinda the software-engineering equivalent of libertarian ideological thinking: it only benefits you if you (and your team) really are far better than the state - otherwise you'll quickly run into problems - and it isn't suitable for the vast majority of the ecosystem.
  - wmf 5y ago
    Let's be honest; we're talking about Chrome here. Chrome auto-updates and Windows (pre-10) did not so bugs will be fixed faster in Chrome. Everybody else is along for the ride.
    - DaiPlusPlus 5y ago
      Right - so in Chrome's case then Google using QUIC as an in-app service is probably fine, as indeed, I trust Google to ship fixes and updates much faster than Microsoft. Also it's far more likely that enterprises have Windows Updates blocked or delayed than Chrome updates - and with Microsoft updates you have to wait for Patch Tuesday every month unless it's something really bad. My point still stands though, that if you aren't prepared to support an "evergreen" software product for the life of your users, then you should still support OS-provided services, even if you do have your own QUIC support baked-in to your product.
    - touisteur 5y ago
      I agree that for updatability it should be something that can be shutdown, reloaded, restarted as simply and with least risk possible. Now I'm thinking that it'll be easier to checkpoint/restore a quic connection than any other thing based on tcp or hidden in the kernel, so you could even do zero-downtime update.
  - touisteur 5y ago
    I'm thinking let's just use something like netty or zmq over a simple ip+icmp interface, socket or not. Instead of having everyone learn some ossified quic kernel api... And since everything will soon move to io_uring let's wrap that thing in high-level APIs.
- yangl1996 5y ago
  There are also the "middle boxes" that networking researchers talk a lot about. Such devices sit in the middle of a link and easily become unhappy if the packets transmitted do not fit some (possibly outdated or buggy) predefined scheme. Think of cooperate firewalls with "deep pack inspection" that intelligently shut down connections they do not like. Once all middle box vendors start to assume a certain way that a protocol (say, TCP) should behave, it's impossible to change the protocol because it will break the middle boxes. Encrypting QUIC datagrams prevents middle box vendors from assuming anything about QUIC (at least the encrypted part), so that QUIC can change if there's a need in the future without worrying about supporting legacy middle boxes. Although I do agree using UDP does not allow QUIC to break out from any ossification in UDP itself.
  - randomifcpfan 5y ago
    Yes, exactly. See this lwn article for more discussion of ossification: https://lwn.net/Articles/745590/ https://lwn.net/Articles/745590/
    - BlueTemplar 5y ago
      > SCTP has been around for years, but middleboxes still do not recognize it and tend to block it. As a result, SCTP cannot be reliably used on the net. Actually deploying a new IP-based protocol, he said, is simply impossible on today's Internet. Well, IPv6 shows that it's hard, but possible. Also see how new broadcast "protocols" were forced on TV manufacturers by various governments worldwide.
  - unethical_ban 5y ago
    Most enterprise security vendors at the moment are advising their customers to block QUIC at the perimeter to force fallback to HTTPS so their TLS decryption can function. There are valid, ethical reasons for an organization to want to see unencrypted network traffic at their perimeter, and until that problem is solved, you better not go QUIC-only if you are in the business to make money.
gumby 5y ago
The IETF doesn't care about OSes. What difference is it for the caller if the interface is inside the kernel or in a userspace library? What if they can't tell?
skissane 5y ago
> Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities (same as for TLS, making fewer apps implement it without a lot of extra work). Which also increases context switching Nowhere does RFC9000 say that it has to be implemented in user-space. A kernel-space implementation of QUIC would be conforming. Any OS kernel project (whether open source or proprietary) which would like to develop one is free to do so.
- amelius 5y ago
  And why does it have to be handled in the OS, if a library would do just fine and provide more flexibility?
  - deleted 5y ago
    [deleted]
  - touisteur 5y ago
    Well you know that someone is gonna implement it in ebpf sooner than later.
    - Matthias247 5y ago
      As someone with experience in eBPF and QUIC I can tell you it will be a tough one. QUIC is very complicated and requires some rather sophisticated (and long running) algorithms, and eBPF is [intentionally] rather limited on what it can do. However there are some interesting use-cases for eBPF in the context of QUIC. E.g. it can definitely be used to route packets by connection IDs.
      - touisteur 5y ago
        You're right. I think on the overall we'll gonna have to learn to restrain/harness ebpf somehow, since it will keep growing. We're already starting to see some complex applications appear over tc+ebpf and I believe I've seen some tcp congestion algorithms prototyped there? Those can be somehow complex already. Crypto could be offloaded to dedicated 'nodes' (as you find in dpdk for example). But, since you can now chain epbf programs I can imagine some kind of task-graph programming model emerging there allowing for very complex applications. Maybe even specific complex kernel modules that can only be called through ebpf... In fact I already tried putting most of a high-throughput packet processor there and it's really fun, this feeling of being in 1998 with my first turbo pascal + asm programs. Hard to debug, though. I hope we'll be able soon to converge on sw archs similar to tbb::flow or cudagraphs or vpp/dpdk pipelines. The next years are going to be very, very interesting.
jeltz 5y ago
Why can't QUIC be implemented in the Kernel? Just because something can be implemented in user space does not mean that it has to.
- throwawaaarrgh 5y ago
  Anything can be implemented in a kernel. But unless it's natively supported in other kernels, networks, and systems, that just means that one kernel is easier to work with. The big idea behind TCP/IP is that it's a stack which you can use anywhere and everywhere. Not just any network, but any host. To do that, you need it to be portable and ubiquitous. And there is no protocol above OSI layer 4 that is ubiquitous. By nature of being above UDP, QUIC is layer 5 or higher. And it's got more features than other protocols do, meaning it needs more functions. So not only is it unlikely that it'll be supported like the layer 4 protocols, it's unlikely to be supported by the Berkeley/POSIX sockets API, the one portable network interface specification for every operating system. One solution to all this would be to make QUIC a layer 4 protocol and update the sockets API. But nobody wants to do this because it's hard. So, we could just have everyone add a layer 5 protocol to their stack, implement it in kernels, NICs, routers, etc, update the sockets API, and wait a few years for adoption. But will that happen? Or will the industry just go "let userspace handle it" like with TLS?
  - jeltz 5y ago
    I really do not see how the layers matter. They are just an artificial categorization. E.g. SCTP is both a layer 4 protocol and a layer 5 protocol (SCTP-over-UDP) and as far as I can tell from some quick googling at least both Linux and FreeBSD implement both in their kernels. What matters is if various libcs add QUIC or not. And the Linux kernel implements other layer 5 protcols like WireGuard.
    - Orphis 5y ago
      SCTP can also be run over DTLS over ICE over UDP. How many layers is that?!
  - anticensor 5y ago
d110af5ccf 5y ago
> Every app has to implement it itself No? I'll just link against one of the cross platform libraries that implements it, the same as for any other networking protocol. (Ex https://github.com/microsoft/msquic https://github.com/microsoft/msquic) Also see: https://github.com/quicwg/base-drafts/wiki/Implementations https://github.com/quicwg/base-drafts/wiki/Implementations > same as for TLS, making fewer apps implement it without a lot of extra work ... do you not just link against a relevant networking library at this point? If not, why?!
- fierro 5y ago
  my guy out here rolling his own TLS implementation
  - westmeal 5y ago
    I dont know about you but I implement TLS in x86 assembler for breakfast.
inetknght 5y ago
> Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities (same as for TLS, making fewer apps implement it without a lot of extra work) TLS is available in the Linux kernel. https://www.kernel.org/doc/html/latest/networking/tls.html https://www.kernel.org/doc/html/latest/networking/tls.html
- detaro 5y ago
  That is merely offloading for the transfer after userland has handled the entire connection setup, key exchange etc.
petre 5y ago
There is also SCTP at layer 4 but it doesn't get much use outside certain domains, so certain OSes have deprecated it. Also as others pointed out applications will just use a library as they do now with OpenSSL.
- detaro 5y ago
  because of the ossification that makes it hard/impossible to use.
  - azernik 5y ago
    I assume you're referring to the disuse of SCTP? That's not because of ossification, it's because of middleware boxes like NATs and firewalls that will refuse to pass anything that isn't TCP, UDP, or ICMP.
    - bonzini 5y ago
      Which is an example of ossification, in this case at the IP level.
      - azernik 5y ago
        From context, I assumed detaro was referring to an ossification of SCTP.
        detaro 5y ago
        Ah, yes, that wasn't quite clear, sorry. No, I was referring to the ossification of middleboxes/OSes/knowledge/... that made it unviable to support a new IP protocol alongside the existing ones for the general internet case.
    - touisteur 5y ago
      Sctp over UDP is a thing and still hasn't gained much general public use over the net... Sadly!
cormacrelf 5y ago
The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. https://http3-explained.haxx.se/en/why-quic/why-ossification https://http3-explained.haxx.se/en/why-quic/why-ossification Many have optimisations around TCP and long-standing TCP characteristics, which they are able to do because the TCP headers are unencrypted. This led to TCP being difficult to improve, because of all the implementations out there making assumptions and doing these optimisations that weren’t compatible with new developments, or worse, dropping traffic with new, unrecognised TCP options. In comparison QUIC is almost 100% encrypted, and exposes comparatively little data for routers to ossify on. So even if network cards start optimising, they can only do so much damage because they are given very little information. For example, there is no way for them to do their own special per-stream flow control, because the stream identifier is encrypted. The only things visible are the UDP src/dest port, the QUIC connection ID, and a minimal set of flags. Most important is that the flow control information is encrypted, which is one of the main things TCP couldn’t improve on due to ossification. That’s not to say the number of implementations stops being a problem — just means that you can actually use QUIC version numbers and negotiate features at the endpoints and ignore anybody along the routes. So you can get adoption of new ideas much more quickly. Quick illustration of the things no longer visible to network operators: https://blog.apnic.net/wp-content/uploads/2019/03/quic-fig2.png https://blog.apnic.net/wp-content/uploads/2019/03/quic-fig2...., from https://blog.apnic.net/2019/03/04/a-quick-look-at-quic/ https://blog.apnic.net/2019/03/04/a-quick-look-at-quic/
- zozbot234 5y ago
  > The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. Being 'opaque' to the network is not a foolproof choice, since it also makes stuff like NAT, congestion control and QoS a lot less robust. By exposing connection identifiers to the network, SCTP gains a lot of flexibility wrt. such scenarios compared to QUIC.
  - capableweb 5y ago
    Correct me if I'm wrong, but bypassing NAT, congestion control and QoS is also a benefit when you want to build things like P2P services where you want a direct connection between peers and full control over the connection itself. QUIC will help a lot with those use cases.
MichaelMoser123 5y ago
i think the intent is that normally you would do a lot of syscalls in order to do networking; with QUICK you would currently be forced to do do zero-copy networking and pass the frames to the kernal via pf-ring; things don't become simpler, as the network layers will have to be handled by a user mode library, however they would potentially be much faster.
- gotbeans 5y ago
  > forced ?? why shouldn't quick be able to use a udp socket?
  - MichaelMoser123 5y ago
    you can do that, if you can to accept the overhead of a system call per outgoing/incoming frame.
    - slrz 5y ago
      You can always use sendmmsg(2), writev(2) or other iovec-based APIs.
    - fanf2 5y ago
      or use sendmmsg(2) and recvmmsg(2) https://www.man7.org/linux/man-pages/man2/sendmmsg.2.html https://www.man7.org/linux/man-pages/man2/sendmmsg.2.html
      - Matthias247 5y ago
        Those work and will reduce the system call overhead. But testing showed that it isn't actually the main culprit (e.g. you might gain 5% efficiency by going for it). A far bigger bottleneck with the kernel stack is that e.g. the route lookup, iptable rules and similar things will be evaluated per packet - which is taking up most of the time. That will happen independent of you deliver one packet per system call or multiple of those. UDP Generic segmentation offload (GSO - https://lwn.net/Articles/752184/ https://lwn.net/Articles/752184/) reduces that overhead by amortizing all in-kernel and driver operations over batches of datagrams. It makes a far bigger difference in efficiency than purely reducing syscalls (e.g. to +100% efficiency - but it will all depend on the other work the application and QUIC stack does and what drivers support).
azernik 5y ago
The ossification has nothing to do with the distinction between userspace vs kernel. The kernel can implement layer 5 (there are in-kernel HTTP servers!), and userspace programs can implement layer 4 (using raw sockets). It has to do with the distinction that middleware boxes make between layer 4 and layer 5. NATs and firewalls and assorted traffic optimizers tend to vomit if you pass anything over IP that isn't TCP, UDP, or ICMP. UDP is, by luck and design, a very thin shim on top of which a "layer 4.5" protocol is very easy to implement. Whether this layering is done in the kernel or in a userspace library is an implementation detail, and one that will probably be obscured even from many userspace programs.
- aaravchen 5y ago
  Ironically, one of the proponent documents of QUIC and HTTP/3 (https://http3-explained.haxx.se/en/the-protocol/feature-udp https://http3-explained.haxx.se/en/the-protocol/feature-udp) actually calls out how QUIC doesn't solve this problem either. Middleware boxes are blocking or deprioritizing UDP packets already, so UDP based 4.5 protocols can't solve the very problem they claim to. Circumventing badly behaving middleware is obviously not actually possible. You can't account for every protocol-breaking/limiting decision every middleware vendor has ever made, and the "they'll come around" argument is a better argument for NOT using QUIC.
overgard 5y ago
> Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities I think for almost all developers, library support for features is much more relevant than OS support anyway. How many of you are directly writing winsock code? Implementing it on UDP just means that you don't need to update a lot of ancient routers and firmware. I don't see "ossification" being a problem. Things haven't changed much at the transport layer because largely it just works.
- josefx 5y ago
  > Things haven't changed much at the transport layer because largely it just works. There is a reason this isn't built on top of TCP, because TCP has so many problems the moment you try to do anything non trivial with it. UDP on the other hand is kind of redundant since you already have IP packages, so you have a tiny package based protocol on top of a different package based protocol. Also you don't get any ordering information in UDP, which is kind of weird since the IP layer has to handle ordering to deal with package fragmentation, which is a fun feature that some hardware sees as a request to reorder your packages. I may have some PTSD from dealing with "smart" switches and a bit of software that did not expect package reordering issues on a tiny in house network.
skyde 5y ago
« Every app has to implement it itself rather than calling a syscall » It actually will be part of the Windows kernel similar to Http.sys

wmf 5y ago

I think from their perspective the (Windows and Android) kernel is what causes ossification so moving transport into the app, along with encryption, solves it.

DaiPlusPlus 5y ago
You're forgetting the rest of userspace that exists between the application and the kernel. > moving transport into the app, along with encryption, solves it This is only a good idea if the application's development moves faster than both the kernel and the rest of the userspace platform - which is a fine strategy if you're targeting old or obsolete platforms, but probably a bad idea if you're targeting currently-supported operating systems with a long support lifetime (like Windows). It's a bad idea when a major security issue or compatibility problem with a transport or encryption library pops-up (which happens all the time, btw) - as it's part of your application then you don't benefit when the OS vendor ships a security update that your application gets for free. If you used dynamic-load+linking at runtime instead of a statically-linked encryption library then you might get lucky with a drop-in .so/.dll replacement without needing a rebuild, but YMMV. Doing everything in the app instead of letting the parent platform handle it by default is kinda the software-engineering equivalent of libertarian ideological thinking: it only benefits you if you (and your team) really are far better than the state - otherwise you'll quickly run into problems - and it isn't suitable for the vast majority of the ecosystem.
- wmf 5y ago
  Let's be honest; we're talking about Chrome here. Chrome auto-updates and Windows (pre-10) did not so bugs will be fixed faster in Chrome. Everybody else is along for the ride.
  - DaiPlusPlus 5y ago
    Right - so in Chrome's case then Google using QUIC as an in-app service is probably fine, as indeed, I trust Google to ship fixes and updates much faster than Microsoft. Also it's far more likely that enterprises have Windows Updates blocked or delayed than Chrome updates - and with Microsoft updates you have to wait for Patch Tuesday every month unless it's something really bad. My point still stands though, that if you aren't prepared to support an "evergreen" software product for the life of your users, then you should still support OS-provided services, even if you do have your own QUIC support baked-in to your product.
  - touisteur 5y ago
    I agree that for updatability it should be something that can be shutdown, reloaded, restarted as simply and with least risk possible. Now I'm thinking that it'll be easier to checkpoint/restore a quic connection than any other thing based on tcp or hidden in the kernel, so you could even do zero-downtime update.
- touisteur 5y ago
  I'm thinking let's just use something like netty or zmq over a simple ip+icmp interface, socket or not. Instead of having everyone learn some ossified quic kernel api... And since everything will soon move to io_uring let's wrap that thing in high-level APIs.
yangl1996 5y ago
There are also the "middle boxes" that networking researchers talk a lot about. Such devices sit in the middle of a link and easily become unhappy if the packets transmitted do not fit some (possibly outdated or buggy) predefined scheme. Think of cooperate firewalls with "deep pack inspection" that intelligently shut down connections they do not like. Once all middle box vendors start to assume a certain way that a protocol (say, TCP) should behave, it's impossible to change the protocol because it will break the middle boxes. Encrypting QUIC datagrams prevents middle box vendors from assuming anything about QUIC (at least the encrypted part), so that QUIC can change if there's a need in the future without worrying about supporting legacy middle boxes. Although I do agree using UDP does not allow QUIC to break out from any ossification in UDP itself.
- randomifcpfan 5y ago
  Yes, exactly. See this lwn article for more discussion of ossification: https://lwn.net/Articles/745590/ https://lwn.net/Articles/745590/
  - BlueTemplar 5y ago
    > SCTP has been around for years, but middleboxes still do not recognize it and tend to block it. As a result, SCTP cannot be reliably used on the net. Actually deploying a new IP-based protocol, he said, is simply impossible on today's Internet. Well, IPv6 shows that it's hard, but possible. Also see how new broadcast "protocols" were forced on TV manufacturers by various governments worldwide.
- unethical_ban 5y ago
  Most enterprise security vendors at the moment are advising their customers to block QUIC at the perimeter to force fallback to HTTPS so their TLS decryption can function. There are valid, ethical reasons for an organization to want to see unencrypted network traffic at their perimeter, and until that problem is solved, you better not go QUIC-only if you are in the business to make money.

gumby 5y ago

The IETF doesn't care about OSes. What difference is it for the caller if the interface is inside the kernel or in a userspace library? What if they can't tell?

skissane 5y ago

> Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities (same as for TLS, making fewer apps implement it without a lot of extra work). Which also increases context switching Nowhere does RFC9000 say that it has to be implemented in user-space. A kernel-space implementation of QUIC would be conforming. Any OS kernel project (whether open source or proprietary) which would like to develop one is free to do so.

amelius 5y ago
And why does it have to be handled in the OS, if a library would do just fine and provide more flexibility?
- deleted 5y ago
  [deleted]
- touisteur 5y ago
  Well you know that someone is gonna implement it in ebpf sooner than later.
  - Matthias247 5y ago
    As someone with experience in eBPF and QUIC I can tell you it will be a tough one. QUIC is very complicated and requires some rather sophisticated (and long running) algorithms, and eBPF is [intentionally] rather limited on what it can do. However there are some interesting use-cases for eBPF in the context of QUIC. E.g. it can definitely be used to route packets by connection IDs.
    - touisteur 5y ago
      You're right. I think on the overall we'll gonna have to learn to restrain/harness ebpf somehow, since it will keep growing. We're already starting to see some complex applications appear over tc+ebpf and I believe I've seen some tcp congestion algorithms prototyped there? Those can be somehow complex already. Crypto could be offloaded to dedicated 'nodes' (as you find in dpdk for example). But, since you can now chain epbf programs I can imagine some kind of task-graph programming model emerging there allowing for very complex applications. Maybe even specific complex kernel modules that can only be called through ebpf... In fact I already tried putting most of a high-throughput packet processor there and it's really fun, this feeling of being in 1998 with my first turbo pascal + asm programs. Hard to debug, though. I hope we'll be able soon to converge on sw archs similar to tbb::flow or cudagraphs or vpp/dpdk pipelines. The next years are going to be very, very interesting.

jeltz 5y ago

Why can't QUIC be implemented in the Kernel? Just because something can be implemented in user space does not mean that it has to.

throwawaaarrgh 5y ago
Anything can be implemented in a kernel. But unless it's natively supported in other kernels, networks, and systems, that just means that one kernel is easier to work with. The big idea behind TCP/IP is that it's a stack which you can use anywhere and everywhere. Not just any network, but any host. To do that, you need it to be portable and ubiquitous. And there is no protocol above OSI layer 4 that is ubiquitous. By nature of being above UDP, QUIC is layer 5 or higher. And it's got more features than other protocols do, meaning it needs more functions. So not only is it unlikely that it'll be supported like the layer 4 protocols, it's unlikely to be supported by the Berkeley/POSIX sockets API, the one portable network interface specification for every operating system. One solution to all this would be to make QUIC a layer 4 protocol and update the sockets API. But nobody wants to do this because it's hard. So, we could just have everyone add a layer 5 protocol to their stack, implement it in kernels, NICs, routers, etc, update the sockets API, and wait a few years for adoption. But will that happen? Or will the industry just go "let userspace handle it" like with TLS?
- jeltz 5y ago
  I really do not see how the layers matter. They are just an artificial categorization. E.g. SCTP is both a layer 4 protocol and a layer 5 protocol (SCTP-over-UDP) and as far as I can tell from some quick googling at least both Linux and FreeBSD implement both in their kernels. What matters is if various libcs add QUIC or not. And the Linux kernel implements other layer 5 protcols like WireGuard.
  - Orphis 5y ago
    SCTP can also be run over DTLS over ICE over UDP. How many layers is that?!
- anticensor 5y ago

d110af5ccf 5y ago

> Every app has to implement it itself No? I'll just link against one of the cross platform libraries that implements it, the same as for any other networking protocol. (Ex https://github.com/microsoft/msquic https://github.com/microsoft/msquic) Also see: https://github.com/quicwg/base-drafts/wiki/Implementations https://github.com/quicwg/base-drafts/wiki/Implementations > same as for TLS, making fewer apps implement it without a lot of extra work ... do you not just link against a relevant networking library at this point? If not, why?!

fierro 5y ago
my guy out here rolling his own TLS implementation
- westmeal 5y ago
  I dont know about you but I implement TLS in x86 assembler for breakfast.

inetknght 5y ago

> Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities (same as for TLS, making fewer apps implement it without a lot of extra work) TLS is available in the Linux kernel. https://www.kernel.org/doc/html/latest/networking/tls.html https://www.kernel.org/doc/html/latest/networking/tls.html

detaro 5y ago
That is merely offloading for the transfer after userland has handled the entire connection setup, key exchange etc.

petre 5y ago

There is also SCTP at layer 4 but it doesn't get much use outside certain domains, so certain OSes have deprecated it. Also as others pointed out applications will just use a library as they do now with OpenSSL.

detaro 5y ago
because of the ossification that makes it hard/impossible to use.
- azernik 5y ago
  I assume you're referring to the disuse of SCTP? That's not because of ossification, it's because of middleware boxes like NATs and firewalls that will refuse to pass anything that isn't TCP, UDP, or ICMP.
  - bonzini 5y ago
    Which is an example of ossification, in this case at the IP level.
    - azernik 5y ago
      From context, I assumed detaro was referring to an ossification of SCTP.
      - detaro 5y ago
        Ah, yes, that wasn't quite clear, sorry. No, I was referring to the ossification of middleboxes/OSes/knowledge/... that made it unviable to support a new IP protocol alongside the existing ones for the general internet case.
  - touisteur 5y ago
    Sctp over UDP is a thing and still hasn't gained much general public use over the net... Sadly!

cormacrelf 5y ago

The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. https://http3-explained.haxx.se/en/why-quic/why-ossification https://http3-explained.haxx.se/en/why-quic/why-ossification Many have optimisations around TCP and long-standing TCP characteristics, which they are able to do because the TCP headers are unencrypted. This led to TCP being difficult to improve, because of all the implementations out there making assumptions and doing these optimisations that weren’t compatible with new developments, or worse, dropping traffic with new, unrecognised TCP options. In comparison QUIC is almost 100% encrypted, and exposes comparatively little data for routers to ossify on. So even if network cards start optimising, they can only do so much damage because they are given very little information. For example, there is no way for them to do their own special per-stream flow control, because the stream identifier is encrypted. The only things visible are the UDP src/dest port, the QUIC connection ID, and a minimal set of flags. Most important is that the flow control information is encrypted, which is one of the main things TCP couldn’t improve on due to ossification. That’s not to say the number of implementations stops being a problem — just means that you can actually use QUIC version numbers and negotiate features at the endpoints and ignore anybody along the routes. So you can get adoption of new ideas much more quickly. Quick illustration of the things no longer visible to network operators: https://blog.apnic.net/wp-content/uploads/2019/03/quic-fig2.png https://blog.apnic.net/wp-content/uploads/2019/03/quic-fig2...., from https://blog.apnic.net/2019/03/04/a-quick-look-at-quic/ https://blog.apnic.net/2019/03/04/a-quick-look-at-quic/

zozbot234 5y ago
> The “ossification” that QUIC deals with is primarily about internet routers who decide to dig into their IP packets. Being 'opaque' to the network is not a foolproof choice, since it also makes stuff like NAT, congestion control and QoS a lot less robust. By exposing connection identifiers to the network, SCTP gains a lot of flexibility wrt. such scenarios compared to QUIC.
- capableweb 5y ago
  Correct me if I'm wrong, but bypassing NAT, congestion control and QoS is also a benefit when you want to build things like P2P services where you want a direct connection between peers and full control over the connection itself. QUIC will help a lot with those use cases.

MichaelMoser123 5y ago

i think the intent is that normally you would do a lot of syscalls in order to do networking; with QUICK you would currently be forced to do do zero-copy networking and pass the frames to the kernal via pf-ring; things don't become simpler, as the network layers will have to be handled by a user mode library, however they would potentially be much faster.

gotbeans 5y ago
> forced ?? why shouldn't quick be able to use a udp socket?
- MichaelMoser123 5y ago
  you can do that, if you can to accept the overhead of a system call per outgoing/incoming frame.
  - slrz 5y ago
    You can always use sendmmsg(2), writev(2) or other iovec-based APIs.
  - fanf2 5y ago
    or use sendmmsg(2) and recvmmsg(2) https://www.man7.org/linux/man-pages/man2/sendmmsg.2.html https://www.man7.org/linux/man-pages/man2/sendmmsg.2.html
    - Matthias247 5y ago
      Those work and will reduce the system call overhead. But testing showed that it isn't actually the main culprit (e.g. you might gain 5% efficiency by going for it). A far bigger bottleneck with the kernel stack is that e.g. the route lookup, iptable rules and similar things will be evaluated per packet - which is taking up most of the time. That will happen independent of you deliver one packet per system call or multiple of those. UDP Generic segmentation offload (GSO - https://lwn.net/Articles/752184/ https://lwn.net/Articles/752184/) reduces that overhead by amortizing all in-kernel and driver operations over batches of datagrams. It makes a far bigger difference in efficiency than purely reducing syscalls (e.g. to +100% efficiency - but it will all depend on the other work the application and QUIC stack does and what drivers support).

azernik 5y ago

The ossification has nothing to do with the distinction between userspace vs kernel. The kernel can implement layer 5 (there are in-kernel HTTP servers!), and userspace programs can implement layer 4 (using raw sockets). It has to do with the distinction that middleware boxes make between layer 4 and layer 5. NATs and firewalls and assorted traffic optimizers tend to vomit if you pass anything over IP that isn't TCP, UDP, or ICMP. UDP is, by luck and design, a very thin shim on top of which a "layer 4.5" protocol is very easy to implement. Whether this layering is done in the kernel or in a userspace library is an implementation detail, and one that will probably be obscured even from many userspace programs.

aaravchen 5y ago
Ironically, one of the proponent documents of QUIC and HTTP/3 (https://http3-explained.haxx.se/en/the-protocol/feature-udp https://http3-explained.haxx.se/en/the-protocol/feature-udp) actually calls out how QUIC doesn't solve this problem either. Middleware boxes are blocking or deprioritizing UDP packets already, so UDP based 4.5 protocols can't solve the very problem they claim to. Circumventing badly behaving middleware is obviously not actually possible. You can't account for every protocol-breaking/limiting decision every middleware vendor has ever made, and the "they'll come around" argument is a better argument for NOT using QUIC.

overgard 5y ago

> Every app has to implement it itself rather than calling a syscall and letting the OS deal with its complexities I think for almost all developers, library support for features is much more relevant than OS support anyway. How many of you are directly writing winsock code? Implementing it on UDP just means that you don't need to update a lot of ancient routers and firmware. I don't see "ossification" being a problem. Things haven't changed much at the transport layer because largely it just works.

josefx 5y ago
> Things haven't changed much at the transport layer because largely it just works. There is a reason this isn't built on top of TCP, because TCP has so many problems the moment you try to do anything non trivial with it. UDP on the other hand is kind of redundant since you already have IP packages, so you have a tiny package based protocol on top of a different package based protocol. Also you don't get any ordering information in UDP, which is kind of weird since the IP layer has to handle ordering to deal with package fragmentation, which is a fun feature that some hardware sees as a request to reorder your packages. I may have some PTSD from dealing with "smart" switches and a bit of software that did not expect package reordering issues on a tiny in house network.

skyde 5y ago

« Every app has to implement it itself rather than calling a syscall » It actually will be part of the Windows kernel similar to Http.sys