8 ms·

LiteSpeed QUIC (LSQUIC) is an open-source implementation of QUIC and HTTP/3

69 points by ZnZirconium 6y ago

garblegarble 6y ago
I know it's an oft-repeated comment here but it does bear repeating: is it really a good idea to keep writing these libraries dealing with complex untrusted user supplied data in C?
- antoinealb 6y ago
  A good reason is that if you want interop with other languages, C is still the lingua franca of ABIs. All scripting languages have a way to interact with C through some form of FFI mechanism. I guess you could write it in Rust and maintain C bindings to it, but that would be painful.
  - rapsey 6y ago
    The rust based cloudflare implementation has C APIs. Also C bindings to rust libs are not that painful at all to maintain.
  - tlamponi 6y ago
    Not really. We wrote a content-addressable-storage backup solution in rust[0], and one consumer is QEMU for doing backups of VMs, so we expose C bindings of our rust code[1] and use them in QEMU[2], effectively having rust async stuff handled by QEMU co-routine library. It wasn't exactly complicate or the like, required a bit of "plumbing code" but that's OK. Allowing to get (relatively) easy C bindings is a goal and feature of rust, at least the project itself provides and maintains the rust-bindgen crate[3][4]. Having a bit bigger and slightly complex project like a backup server done in rust is such a relief compared to C or similar languages. Good speed, fast start up times like C but refactoring is really a breeze lots of safety is guaranteed and more time can be spent on fixing the semantic bugs. [0]: https://pbs.proxmox.com/docs/introduction.html#introduction https://pbs.proxmox.com/docs/introduction.html#introduction [1]: https://git.proxmox.com/?p=proxmox-backup-qemu.git;a=summary https://git.proxmox.com/?p=proxmox-backup-qemu.git;a=summary [2]: https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=debian/patches/pve/0028-PVE-Backup-proxmox-backup-patches-for-qemu.patch;h=cb8334ee34006851fdf5b7b5c8e5f2586719e785;hb=HEAD#l312 https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=debian/patc... [3]: https://github.com/rust-lang/rust-bindgen https://github.com/rust-lang/rust-bindgen [4]: https://rust-lang.github.io/rust-bindgen/ https://rust-lang.github.io/rust-bindgen/
    - antoinealb 6y ago
      Interesting, I stand corrected!
- jarym 6y ago
  Well if you're an experienced C developer then yes it does.
  - garblegarble 6y ago
    But lots of the vulnerable C code we have in the world today was written by very experienced C developers, so I don't know that's a particularly strong argument
    - jarym 6y ago
      You must remember, a LOT of C code was written back in a time when the Internet was a more trusting environment. Lots of developers didn't even comprehend that there'd be malicious actors. The point stands, if you're an experienced developer in one particular language then of course you will write in that language.
      - tlamponi 6y ago
        > You must remember, a LOT of C code was written back in a time when the Internet was a more trusting environment. Lots of developers didn't even comprehend that there'd be malicious actors. Hackers were a well established thing in the 80s, with its origin "phreaking" being even older than that, so no it wasn't a naïve time when everybody though all just had the best interest possible in mind. Note also that not all crashes and breakages are intended and result of a malicious actor, or do you now argue that human entry errors just did not happen back then? Further, most code today does not come from that era but rather originated from 90s to 2000s, with not even much of that being still around in that form. > The point stands, if you're an experienced developer in one particular language then of course you will write in that language. 1. That's the first time you make that point, the original reply of yours has no argument whatsoever, so nothing "still stands". 2. If you're really an experienced developer in one particular language then you are aware of its shortcomings and will also look closely for languages addressing them without adding other disadvantages. Everything else would be just short-sighted, which isn't really a good virtue to have as experienced developer. 3. How comes that new code written by highly experienced developers, e.g., working on OS kernels, with a massive testing effort still let slip through buffer overflows, use after free, integer underflows, etc. etc.?
- vlmutolo 6y ago
  There are some good rust implementations of QUIC, too. Cloudflare: https://lib.rs/crates/quiche https://lib.rs/crates/quiche Mozilla: https://lib.rs/gh/mozilla/neqo/neqo-interop https://lib.rs/gh/mozilla/neqo/neqo-interop
  - rapsey 6y ago
    Also quinn: https://crates.io/crates/quinn https://crates.io/crates/quinn
- jeffbee 6y ago
  I guess there is an audience for this stuff, among embedded systems developers or whatever, but after a brief look at the project it doesn't strike me as a project written with the care needed for C libraries. It is severely under-documented, for starters. For example, the word "thread" does not appear anywhere. Not sure why one would choose this over QUICHE.
  - mondainx 6y ago
    I had an issue using quiche as a dependency via JNI and it was a blocker (I've moved on). The event would lose its reference and segfault; I was unable to track it to an actual cause in the debugger; lots of time wasted.
    - Matthias247 6y ago
      That seems more like an implementation issue than a general one. Netty integrates with quiche via JNI for QUIC support: https://github.com/netty/netty-incubator-codec-quic https://github.com/netty/netty-incubator-codec-quic
  - Matthias247 6y ago
    > the word "thread" does not appear anywhere. because it doesn't use threads? The library is intended to be used inside an eventloop. I think the same also applies for other typical transport libraries - e.g. HTTP/2 or TLS ones. > Not sure why one would choose this over QUICHE. I think there are certainly reasons. lsquic seems a lot more optimized than quiche and most other libraries out there. It makes use of some pretty clever datastructures (e.g. https://github.com/litespeedtech/lsquic/blob/master/src/liblsquic/lsquic_di_hash.c https://github.com/litespeedtech/lsquic/blob/master/src/libl...), and likely has a drastically lower rate of heap allocations than other implementations. Some of those things - like the use of intrusive linked lists - are unfortunately not that easy to apply in Rust. I wouldn't be suprised if lsquic outperforms various other implementations - and if that's important to users it might be a reason to choose it (but as always: measure for your use-case). I personally also think Rust is the way to go for system level code. But I wouldn't dismiss a project for not using Rust. And this one at least has a fair set of unit-tests, so it looks to me a lot more sane than a lot of other C based projects.
- rapsey 6y ago
  There is zero reason for this to exist.
  - deleted 6y ago
    [deleted]
- Nokinside 6y ago
  I don't know about open source developer case. If you develop commercial embedded software and need to be able to develop safety critical software there is no contest: C and Ada are the two best tools. Obviously you limit the coding to subset of C. MISRA C for example. You can buy commercial tools for C and Ada that do static analysis only your source-code way past anything that you get from other languages. You can prove the absence of run time errors. No divisions by zero, no sudden application termination etc. If you write safety critical code, you don't do dynamic memory allocation or unrestricted recursion anyway. The strengths that Rust has in memory allocation are irrelevant. Running out of memory or unlimited recursion are errors.
- gostsamo 6y ago
  Rust so far does not cover all architectures that C can reach. When the GCC implementation is ready, it might change, but not yet.
  - cogman10 6y ago
    What sort of architecture are you envisioning where you'd need HTTP3 and Rust can't build for it? Once you get a microcontroller fast enough to handle Http3, you are talking about well known platforms (such as ARM, MIPS, x86). All of which are supported by Rust and LLVM. https://doc.rust-lang.org/nightly/rustc/platform-support.html https://doc.rust-lang.org/nightly/rustc/platform-support.htm...
    - gostsamo 6y ago
      Keeping in mind that I needed to write code against node 0.13 because it was the last version of node supporting floating point emulation when there is no hardware fpu, I was surprised to learn what kind of trash is out there running the internet. There is list of supported architectures of the software in question and though I don't remember it in details, I'm sure that there were items in it that I couldn't find in the rust tables.
      - cogman10 6y ago
        :) sounds like bcm4709. Funnily, rust wouldn't have a problem there, that's specifically a node problem because their JIT doesn't handle software FPUs. LLVM does. I'm fairly confident in saying any place you can run any node version, you can run rust. https://github.com/nxhack/openwrt-node-packages/issues/15 https://github.com/nxhack/openwrt-node-packages/issues/15
        gostsamo 6y ago
        In this case the node had to run on Luxul routers, I don't remember the architecture.
- foobiekr 6y ago
  No. It's not. And I love C.
justin66 6y ago
I bet almost all of the implementations of QUIC right now are open source.
- emteycz 6y ago
  For sure, but why does it matter here?
- tyingq 6y ago
  I believe they are calling it out as open source because their LiteSpeed web server has non open source editions.
- moderation 6y ago
  Most of them, yes - https://github.com/quicwg/base-drafts/wiki/Implementations https://github.com/quicwg/base-drafts/wiki/Implementations
turminal 6y ago
What problem do QUIC and http/3 solve?
- jeffbee 6y ago
  By moving everything into the application, QUIC circumvents the extremely slow pace of innovation in operating systems' TCP stacks. Unlike userspace TCP, which requires elevated privileges such as CAP_NET_RAW on Linux, QUIC does not require any special privileges.
- ocdtrekkie 6y ago
  It is a technology for adtech companies to circumvent operating system and network security features.
  - tialaramex 6y ago
    If you've got enough cynicism to decide the only beneficiaries of QUIC are just "adtech companies" then I suggest saving a little of that cycnicism to put scare quotes around that word "security" in the second half of your sentence. If you have effective security neither TLS 1.3 nor QUIC change anything in that regard. But they probably do change things for you. This means you did not have effective security. Those making appliances or software to deliver "security" would very much like you to blame the designers of QUIC or TLS 1.3 for the fact that their products are ineffective. It is as if the Carbolic Smoke Ball Company had tried to blame doctors for diagnosing Carlill with Influenza rather than accepting that, in fact, Carbolic Smoke Balls don't work and so they owe her £100 per their advertisement.
- jayd16 6y ago
  Lots of resources are out there but essentially it expands http/2's multiplexed requests by building on top of UDP instead of TCP so it can ensure those requests don't block each other.
- nicoburns 6y ago
  Head of line blocking in http/2, and performance/reliability on dodgy mobile connections. It's definitely getting to the point of diminishing returns, but given how widely http is used it does seem to justify the effort.

69 points by ZnZirconium 6y ago

I know it's an oft-repeated comment here but it does bear repeating: is it really a good idea to keep writing these libraries dealing with complex untrusted user supplied data in C?

antoinealb 6y ago
A good reason is that if you want interop with other languages, C is still the lingua franca of ABIs. All scripting languages have a way to interact with C through some form of FFI mechanism. I guess you could write it in Rust and maintain C bindings to it, but that would be painful.
- rapsey 6y ago
  The rust based cloudflare implementation has C APIs. Also C bindings to rust libs are not that painful at all to maintain.
- tlamponi 6y ago
  Not really. We wrote a content-addressable-storage backup solution in rust[0], and one consumer is QEMU for doing backups of VMs, so we expose C bindings of our rust code[1] and use them in QEMU[2], effectively having rust async stuff handled by QEMU co-routine library. It wasn't exactly complicate or the like, required a bit of "plumbing code" but that's OK. Allowing to get (relatively) easy C bindings is a goal and feature of rust, at least the project itself provides and maintains the rust-bindgen crate[3][4]. Having a bit bigger and slightly complex project like a backup server done in rust is such a relief compared to C or similar languages. Good speed, fast start up times like C but refactoring is really a breeze lots of safety is guaranteed and more time can be spent on fixing the semantic bugs. [0]: https://pbs.proxmox.com/docs/introduction.html#introduction https://pbs.proxmox.com/docs/introduction.html#introduction [1]: https://git.proxmox.com/?p=proxmox-backup-qemu.git;a=summary https://git.proxmox.com/?p=proxmox-backup-qemu.git;a=summary [2]: https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=debian/patches/pve/0028-PVE-Backup-proxmox-backup-patches-for-qemu.patch;h=cb8334ee34006851fdf5b7b5c8e5f2586719e785;hb=HEAD#l312 https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=debian/patc... [3]: https://github.com/rust-lang/rust-bindgen https://github.com/rust-lang/rust-bindgen [4]: https://rust-lang.github.io/rust-bindgen/ https://rust-lang.github.io/rust-bindgen/
  - antoinealb 6y ago
    Interesting, I stand corrected!
jarym 6y ago
Well if you're an experienced C developer then yes it does.
- garblegarble 6y ago
  But lots of the vulnerable C code we have in the world today was written by very experienced C developers, so I don't know that's a particularly strong argument
  - jarym 6y ago
    You must remember, a LOT of C code was written back in a time when the Internet was a more trusting environment. Lots of developers didn't even comprehend that there'd be malicious actors. The point stands, if you're an experienced developer in one particular language then of course you will write in that language.
    - tlamponi 6y ago
      > You must remember, a LOT of C code was written back in a time when the Internet was a more trusting environment. Lots of developers didn't even comprehend that there'd be malicious actors. Hackers were a well established thing in the 80s, with its origin "phreaking" being even older than that, so no it wasn't a naïve time when everybody though all just had the best interest possible in mind. Note also that not all crashes and breakages are intended and result of a malicious actor, or do you now argue that human entry errors just did not happen back then? Further, most code today does not come from that era but rather originated from 90s to 2000s, with not even much of that being still around in that form. > The point stands, if you're an experienced developer in one particular language then of course you will write in that language. 1. That's the first time you make that point, the original reply of yours has no argument whatsoever, so nothing "still stands". 2. If you're really an experienced developer in one particular language then you are aware of its shortcomings and will also look closely for languages addressing them without adding other disadvantages. Everything else would be just short-sighted, which isn't really a good virtue to have as experienced developer. 3. How comes that new code written by highly experienced developers, e.g., working on OS kernels, with a massive testing effort still let slip through buffer overflows, use after free, integer underflows, etc. etc.?
vlmutolo 6y ago
There are some good rust implementations of QUIC, too. Cloudflare: https://lib.rs/crates/quiche https://lib.rs/crates/quiche Mozilla: https://lib.rs/gh/mozilla/neqo/neqo-interop https://lib.rs/gh/mozilla/neqo/neqo-interop
- rapsey 6y ago
  Also quinn: https://crates.io/crates/quinn https://crates.io/crates/quinn
jeffbee 6y ago
I guess there is an audience for this stuff, among embedded systems developers or whatever, but after a brief look at the project it doesn't strike me as a project written with the care needed for C libraries. It is severely under-documented, for starters. For example, the word "thread" does not appear anywhere. Not sure why one would choose this over QUICHE.
- mondainx 6y ago
  I had an issue using quiche as a dependency via JNI and it was a blocker (I've moved on). The event would lose its reference and segfault; I was unable to track it to an actual cause in the debugger; lots of time wasted.
  - Matthias247 6y ago
    That seems more like an implementation issue than a general one. Netty integrates with quiche via JNI for QUIC support: https://github.com/netty/netty-incubator-codec-quic https://github.com/netty/netty-incubator-codec-quic
- Matthias247 6y ago
  > the word "thread" does not appear anywhere. because it doesn't use threads? The library is intended to be used inside an eventloop. I think the same also applies for other typical transport libraries - e.g. HTTP/2 or TLS ones. > Not sure why one would choose this over QUICHE. I think there are certainly reasons. lsquic seems a lot more optimized than quiche and most other libraries out there. It makes use of some pretty clever datastructures (e.g. https://github.com/litespeedtech/lsquic/blob/master/src/liblsquic/lsquic_di_hash.c https://github.com/litespeedtech/lsquic/blob/master/src/libl...), and likely has a drastically lower rate of heap allocations than other implementations. Some of those things - like the use of intrusive linked lists - are unfortunately not that easy to apply in Rust. I wouldn't be suprised if lsquic outperforms various other implementations - and if that's important to users it might be a reason to choose it (but as always: measure for your use-case). I personally also think Rust is the way to go for system level code. But I wouldn't dismiss a project for not using Rust. And this one at least has a fair set of unit-tests, so it looks to me a lot more sane than a lot of other C based projects.
rapsey 6y ago
There is zero reason for this to exist.
- deleted 6y ago
  [deleted]
Nokinside 6y ago
I don't know about open source developer case. If you develop commercial embedded software and need to be able to develop safety critical software there is no contest: C and Ada are the two best tools. Obviously you limit the coding to subset of C. MISRA C for example. You can buy commercial tools for C and Ada that do static analysis only your source-code way past anything that you get from other languages. You can prove the absence of run time errors. No divisions by zero, no sudden application termination etc. If you write safety critical code, you don't do dynamic memory allocation or unrestricted recursion anyway. The strengths that Rust has in memory allocation are irrelevant. Running out of memory or unlimited recursion are errors.
gostsamo 6y ago
Rust so far does not cover all architectures that C can reach. When the GCC implementation is ready, it might change, but not yet.
- cogman10 6y ago
  What sort of architecture are you envisioning where you'd need HTTP3 and Rust can't build for it? Once you get a microcontroller fast enough to handle Http3, you are talking about well known platforms (such as ARM, MIPS, x86). All of which are supported by Rust and LLVM. https://doc.rust-lang.org/nightly/rustc/platform-support.html https://doc.rust-lang.org/nightly/rustc/platform-support.htm...
  - gostsamo 6y ago
    Keeping in mind that I needed to write code against node 0.13 because it was the last version of node supporting floating point emulation when there is no hardware fpu, I was surprised to learn what kind of trash is out there running the internet. There is list of supported architectures of the software in question and though I don't remember it in details, I'm sure that there were items in it that I couldn't find in the rust tables.
    - cogman10 6y ago
      :) sounds like bcm4709. Funnily, rust wouldn't have a problem there, that's specifically a node problem because their JIT doesn't handle software FPUs. LLVM does. I'm fairly confident in saying any place you can run any node version, you can run rust. https://github.com/nxhack/openwrt-node-packages/issues/15 https://github.com/nxhack/openwrt-node-packages/issues/15
      - gostsamo 6y ago
        In this case the node had to run on Luxul routers, I don't remember the architecture.
foobiekr 6y ago
No. It's not. And I love C.

justin66 6y ago

I bet almost all of the implementations of QUIC right now are open source.

emteycz 6y ago
For sure, but why does it matter here?
tyingq 6y ago
I believe they are calling it out as open source because their LiteSpeed web server has non open source editions.
moderation 6y ago
Most of them, yes - https://github.com/quicwg/base-drafts/wiki/Implementations https://github.com/quicwg/base-drafts/wiki/Implementations

turminal 6y ago

What problem do QUIC and http/3 solve?

jeffbee 6y ago
By moving everything into the application, QUIC circumvents the extremely slow pace of innovation in operating systems' TCP stacks. Unlike userspace TCP, which requires elevated privileges such as CAP_NET_RAW on Linux, QUIC does not require any special privileges.
ocdtrekkie 6y ago
It is a technology for adtech companies to circumvent operating system and network security features.
- tialaramex 6y ago
  If you've got enough cynicism to decide the only beneficiaries of QUIC are just "adtech companies" then I suggest saving a little of that cycnicism to put scare quotes around that word "security" in the second half of your sentence. If you have effective security neither TLS 1.3 nor QUIC change anything in that regard. But they probably do change things for you. This means you did not have effective security. Those making appliances or software to deliver "security" would very much like you to blame the designers of QUIC or TLS 1.3 for the fact that their products are ineffective. It is as if the Carbolic Smoke Ball Company had tried to blame doctors for diagnosing Carlill with Influenza rather than accepting that, in fact, Carbolic Smoke Balls don't work and so they owe her £100 per their advertisement.
jayd16 6y ago
Lots of resources are out there but essentially it expands http/2's multiplexed requests by building on top of UDP instead of TCP so it can ensure those requests don't block each other.
nicoburns 6y ago
Head of line blocking in http/2, and performance/reliability on dodgy mobile connections. It's definitely getting to the point of diminishing returns, but given how widely http is used it does seem to justify the effort.