6 ms·

Please fire away any questions you may have! I lead the team that built this library. This blog has details on current development status and adoption within M

Please fire away any questions you may have! I lead the team that built this library. This blog has details on current development status and adoption within Microsoft: https://techcommunity.microsoft.com/t5/networking-blog/msquic-is-open-source/ba-p/1345441 https://techcommunity.microsoft.com/t5/networking-blog/msqui...

gravypod 6y ago
Sorry, this might be a bit off topic but it's something I've been excited about for a while. From what I've heard Microsoft is sort of getting behind gRPC. Have you tested using QUIC as a transport layer for the gRPC client/server libraries that MS maintains? Are you seeing notable performance benefits with QUIC as the underlying channel?
- slowstart 6y ago
  RPC/gRPC are certainly possible use cases for the QUIC transport protocol. But no, we have not yet explored the use of QUIC in this context. For now we are focused on workloads that will benefit the most from the tail latency performance and security improvements that QUIC brings.
- davidfowl 6y ago
  I'm from the .NET Core team and we've looked at it a little from the HTTP/3 angle but not from a pure QUIC angle. This is because the gRPC RPC protocol is described in terms of HTTP/2 frames today.
  - Matthias247 6y ago
    gRPC encoded data is actually not tightly coupled to HTTP/2 frames. It's described as a stream of gRPC chunk encoded data on top of a HTTP stream. But the gRPC frames do not necessarily have to align with HTTP/2 data frames boundaries. See https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md#data-frames https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.... The specification makes gRPC sound rather tightly bound to HTTP/2 level details, but I don't think it really is. You should be able to speak gRPC just fine over any kind of HTTP. One main pain point for browser support however had been APIs to exchange trailers, which are necessary for gRPC.
    - JamesNK 6y ago
      > gRPC encoded data is actually not tightly coupled to HTTP/2 frames. The headers frame, and the capability to have headers before AND after content is a requirement. gRPC requires trailing headers for the status code of the call. Trailing headers frame is a new concept in HTTP/2. gRPC-Web supports HTTP/1.1 and browsers. It is able to do that by encoding the status into the end of the response body. However gRPC-Web is a different spec. If HTTP/3+QUIC supports the same features of HTTP/2 then gRPC should work on it. There might be a HTTP/3 specific spec for details around the management of a HTTP/3 connection, but gRPC headers, message content, and proto contracts shouldn't need to change. Take what I say with a grain of salt because I haven't looked closely at HTTP/3+QUIC yet.
      - Matthias247 6y ago
        A headers frame before content is just equivalent so „sending headers“, since Http/2 also only allows to send headers once per Stream unless those are informational headers. In the same fashion, sending a headers frame after content is equivalent to „sending trailing headers“ - which are allowed to be sent at most once after the body (which may be empty). Therefore the fact that there is a frame involved doesn’t really matter. HTTP/3 doesn’t change the HTTP semantics: Peers are sending 0-N informational headers, 1 set of request headers, a stream of body data, and 0-1 set of trailing headers. Therefore gRPC should run fine over it at long as the underlying HTTP library exposes all those necessary features.
tialaramex 6y ago
Any strong reasons you chose C? Maybe I skimmed too quickly but I didn't see this mentioned in that blog post. Was there a requirement from other teams?
- slowstart 6y ago
  Since this had to run in kernel mode on Windows to power our HTTP stack, C was the language of choice. There exist other open source implementations of QUIC in C++ and Rust etc.
  - saagarjha 6y ago
    Since this is going in the kernel and is exposed to the network, what kinds of things are you doing to prevent security or reliability bugs due to undefined behavior? Love the username, by the way :)
    - slowstart 6y ago
      We do extensive testing including stress testing and make use of tooling that can catch bugs early. We also partner with internal security teams to do fuzz testing and security reviews for all networking code. That said, none of the networking stacks deployed widely today are completely immune to security vulnerabilities. Responsible disclosure also plays an important role.
      - sansnomme 6y ago
        Any plans to integrate or collaborate with Project Everest?
    - __s 6y ago
      You can see some of the tooling they're using, .azure outlines CI & /tools has scripts like https://github.com/microsoft/msquic/blob/6fa51a42f69c59748dd6a54b313843b6b3f68dfb/src/tools/attack/attack.cpp https://github.com/microsoft/msquic/blob/6fa51a42f69c59748dd... There'll also be static analysis being thrown at it
zvrba 6y ago
I know it's not your team, but now that it's going in Windows kernel, maybe you could ask around... Any plans to support QUIC for data transfers from/to Azure Blob Storage?
fulafel 6y ago
Does QUIC or your implementation of it support application control of keying or is it always based on x.509 certs and CAs? edit: the spec (https://tools.ietf.org/html/draft-ietf-quic-tls-27 https://tools.ietf.org/html/draft-ietf-quic-tls-27) seems agnostic on this. Also simple APIs especially in security are important, so supporting certs only is no flaw in my book, just curious about the edges of how the OS QUIC could be used in the future.
- tialaramex 6y ago
  QUIC outsources this part of the solution to TLS (specifically TLS 1.3) so you just need a way to meet your needs in TLS 1.3 and it'll work in QUIC
  - dochtman 6y ago
    Though it has been discussed that future versions of QUIC might allow other authentication/encryption protocols. Noise would be an interesting candidate.
    - fulafel 6y ago
      Note that TLS doesn't necessarily imply certs either. TSL-PSK, TLS-SRP, anon DH, etc.
      - tialaramex 6y ago
        Sure, but, it's important to caveat that QUIC requires specifically TLS 1.3 (or potentially subsequent versions in the future) and so features which require older TLS versions aren't useful. Pre-shared keys are a thing in TLS 1.3 though there are subtle differences you ought to be aware of before implementing, but as I understand it SRP is not (at time of writing) and neither is anonymity. It isn't possible to "just" take an extension to TLS 1.2 that altered the handshake mechanism and have it work in TLS 1.3 because the handshake is very different even though it was camouflaged so that rusted-in-place TLS 1.2 middleboxes think it's just TLS 1.2 and don't freak out.
netheril96 6y ago
Will BBR congestion control be supported? Without it QUIC performance cannot match that of TCP, in my environment at least.
- nibanks 6y ago
  It's definitely on the TODO list. We're looking into it.
chaz6 6y ago
Will you add support for boringssl?
- nibanks 6y ago
  Not likely, unless we get a customer ask for it. But when we start accepting external contributions it shouldn't be too hard for someone else to add the support. We already (unofficially) support 3 different TLS libraries (schannel, openssl, mitls).
Lvl999Noob 6y ago
I recently had a project in my college course to implement MP-QUIC, but there was sever lack of resources on it. What do you guys think of MPQUIC vs QUIC?
akmittal 6y ago
Was't QUIC was standarized as HTTP/3 what makes it call QUIC not HTTP/3.
- detaro 6y ago
  No. HTTP/3 is HTTP using QUIC as the lower-layer transport. QUIC itself allows for different protocols to be build on top of it, and was standardized on its own.
  - elithrar 6y ago
    To expand on this: • QUIC’s original implementation bound the transport protocol & app protocol to HTTP/2 - this was led by Google and is commonly referred to as “gQUIC”. This is what Chromium implemented and could be used via the Cronet library in other contexts. • A desire to use QUIC as a stand-alone transport layer (above UDP) grew, and is now standards-tracked as “IETF QUIC”. • HTTP/3 is a evolution (rather than a revolution) of HTTP/2 that requires IETF QUIC as a transport.
deleted 6y ago
[deleted]
MrStonedOne 6y ago
Do you think you've adquently built something that can be used as a transport layer first, compared to googles attempt that seems more like a http snowflake first and a transport layer second. QUIC has the potential to be helpful in game development but suffers from an overly specialized approach.
- nibanks 6y ago
  Yes, msquic should be a good general purpose transport. We already have usage from SMB (file sharing) and HTTP in Windows. Both are very different and provided good test cases for msquic.
profquail 6y ago
Have you considered implementing any parts of this in F* (so they can be verified) and extracting back to C, as is being done for TLS? https://project-everest.github.io/ https://project-everest.github.io/
- nibanks 6y ago
  We do work with the Everest team. We have unofficial support on top of miTLS (which they produce). We haven't looked into actually using F* for any of the QUIC code though.
- catalin_hritcu 6y ago
  Some work on verifying QUIC packet encryption using F* is happening at Microsoft Research: https://github.com/project-everest/everquic-crypto https://github.com/project-everest/everquic-crypto
  - protz 6y ago
    Just to build on Catalin's answer. We are actively working on an implementation of QUIC's transport layer (i.e. packet encryption and decryption), along with a proof of cryptographic security. This is what Catalin linked to (https://github.com/project-everest/everquic-crypto https://github.com/project-everest/everquic-crypto). EverQuic-Crypto builds upon two previous projects: EverParse, a library of verified low-level parsers and serializers which we apply to the QUIC network formats, and EverCrypt, a cryptographic provider with agility and multiplexing, which we use for all the cryptography, e.g. packet number encryption, AEAD, etc. This is not yet a full QUIC implementation, but we have plans for extending this codebase to cover more of the QUIC protocol.
skyde 6y ago
will this be integrated in some rpc framework like gRPC we all can use ?
skyde 6y ago
is this using BBR congestion algorithm ?
- nibanks 6y ago
  Not yet. BBR is on the TODO list though.
skyde 6y ago
will it be possible to use the « sendfile » system call to do zero copy file transfers on a quick connection?

gravypod 6y ago

Sorry, this might be a bit off topic but it's something I've been excited about for a while. From what I've heard Microsoft is sort of getting behind gRPC. Have you tested using QUIC as a transport layer for the gRPC client/server libraries that MS maintains? Are you seeing notable performance benefits with QUIC as the underlying channel?

slowstart 6y ago
RPC/gRPC are certainly possible use cases for the QUIC transport protocol. But no, we have not yet explored the use of QUIC in this context. For now we are focused on workloads that will benefit the most from the tail latency performance and security improvements that QUIC brings.
davidfowl 6y ago
I'm from the .NET Core team and we've looked at it a little from the HTTP/3 angle but not from a pure QUIC angle. This is because the gRPC RPC protocol is described in terms of HTTP/2 frames today.
- Matthias247 6y ago
  gRPC encoded data is actually not tightly coupled to HTTP/2 frames. It's described as a stream of gRPC chunk encoded data on top of a HTTP stream. But the gRPC frames do not necessarily have to align with HTTP/2 data frames boundaries. See https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md#data-frames https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.... The specification makes gRPC sound rather tightly bound to HTTP/2 level details, but I don't think it really is. You should be able to speak gRPC just fine over any kind of HTTP. One main pain point for browser support however had been APIs to exchange trailers, which are necessary for gRPC.
  - JamesNK 6y ago
    > gRPC encoded data is actually not tightly coupled to HTTP/2 frames. The headers frame, and the capability to have headers before AND after content is a requirement. gRPC requires trailing headers for the status code of the call. Trailing headers frame is a new concept in HTTP/2. gRPC-Web supports HTTP/1.1 and browsers. It is able to do that by encoding the status into the end of the response body. However gRPC-Web is a different spec. If HTTP/3+QUIC supports the same features of HTTP/2 then gRPC should work on it. There might be a HTTP/3 specific spec for details around the management of a HTTP/3 connection, but gRPC headers, message content, and proto contracts shouldn't need to change. Take what I say with a grain of salt because I haven't looked closely at HTTP/3+QUIC yet.
    - Matthias247 6y ago
      A headers frame before content is just equivalent so „sending headers“, since Http/2 also only allows to send headers once per Stream unless those are informational headers. In the same fashion, sending a headers frame after content is equivalent to „sending trailing headers“ - which are allowed to be sent at most once after the body (which may be empty). Therefore the fact that there is a frame involved doesn’t really matter. HTTP/3 doesn’t change the HTTP semantics: Peers are sending 0-N informational headers, 1 set of request headers, a stream of body data, and 0-1 set of trailing headers. Therefore gRPC should run fine over it at long as the underlying HTTP library exposes all those necessary features.

tialaramex 6y ago

Any strong reasons you chose C? Maybe I skimmed too quickly but I didn't see this mentioned in that blog post. Was there a requirement from other teams?

slowstart 6y ago
Since this had to run in kernel mode on Windows to power our HTTP stack, C was the language of choice. There exist other open source implementations of QUIC in C++ and Rust etc.
- saagarjha 6y ago
  Since this is going in the kernel and is exposed to the network, what kinds of things are you doing to prevent security or reliability bugs due to undefined behavior? Love the username, by the way :)
  - slowstart 6y ago
    We do extensive testing including stress testing and make use of tooling that can catch bugs early. We also partner with internal security teams to do fuzz testing and security reviews for all networking code. That said, none of the networking stacks deployed widely today are completely immune to security vulnerabilities. Responsible disclosure also plays an important role.
    - sansnomme 6y ago
      Any plans to integrate or collaborate with Project Everest?
  - __s 6y ago
    You can see some of the tooling they're using, .azure outlines CI & /tools has scripts like https://github.com/microsoft/msquic/blob/6fa51a42f69c59748dd6a54b313843b6b3f68dfb/src/tools/attack/attack.cpp https://github.com/microsoft/msquic/blob/6fa51a42f69c59748dd... There'll also be static analysis being thrown at it

zvrba 6y ago

I know it's not your team, but now that it's going in Windows kernel, maybe you could ask around... Any plans to support QUIC for data transfers from/to Azure Blob Storage?

fulafel 6y ago

Does QUIC or your implementation of it support application control of keying or is it always based on x.509 certs and CAs? edit: the spec (https://tools.ietf.org/html/draft-ietf-quic-tls-27 https://tools.ietf.org/html/draft-ietf-quic-tls-27) seems agnostic on this. Also simple APIs especially in security are important, so supporting certs only is no flaw in my book, just curious about the edges of how the OS QUIC could be used in the future.

tialaramex 6y ago
QUIC outsources this part of the solution to TLS (specifically TLS 1.3) so you just need a way to meet your needs in TLS 1.3 and it'll work in QUIC
- dochtman 6y ago
  Though it has been discussed that future versions of QUIC might allow other authentication/encryption protocols. Noise would be an interesting candidate.
  - fulafel 6y ago
    Note that TLS doesn't necessarily imply certs either. TSL-PSK, TLS-SRP, anon DH, etc.
    - tialaramex 6y ago
      Sure, but, it's important to caveat that QUIC requires specifically TLS 1.3 (or potentially subsequent versions in the future) and so features which require older TLS versions aren't useful. Pre-shared keys are a thing in TLS 1.3 though there are subtle differences you ought to be aware of before implementing, but as I understand it SRP is not (at time of writing) and neither is anonymity. It isn't possible to "just" take an extension to TLS 1.2 that altered the handshake mechanism and have it work in TLS 1.3 because the handshake is very different even though it was camouflaged so that rusted-in-place TLS 1.2 middleboxes think it's just TLS 1.2 and don't freak out.

netheril96 6y ago

Will BBR congestion control be supported? Without it QUIC performance cannot match that of TCP, in my environment at least.

nibanks 6y ago
It's definitely on the TODO list. We're looking into it.

chaz6 6y ago

Will you add support for boringssl?

nibanks 6y ago
Not likely, unless we get a customer ask for it. But when we start accepting external contributions it shouldn't be too hard for someone else to add the support. We already (unofficially) support 3 different TLS libraries (schannel, openssl, mitls).

Lvl999Noob 6y ago

I recently had a project in my college course to implement MP-QUIC, but there was sever lack of resources on it. What do you guys think of MPQUIC vs QUIC?

akmittal 6y ago

Was't QUIC was standarized as HTTP/3 what makes it call QUIC not HTTP/3.

detaro 6y ago
No. HTTP/3 is HTTP using QUIC as the lower-layer transport. QUIC itself allows for different protocols to be build on top of it, and was standardized on its own.
- elithrar 6y ago
  To expand on this: • QUIC’s original implementation bound the transport protocol & app protocol to HTTP/2 - this was led by Google and is commonly referred to as “gQUIC”. This is what Chromium implemented and could be used via the Cronet library in other contexts. • A desire to use QUIC as a stand-alone transport layer (above UDP) grew, and is now standards-tracked as “IETF QUIC”. • HTTP/3 is a evolution (rather than a revolution) of HTTP/2 that requires IETF QUIC as a transport.

deleted 6y ago

[deleted]

MrStonedOne 6y ago

Do you think you've adquently built something that can be used as a transport layer first, compared to googles attempt that seems more like a http snowflake first and a transport layer second. QUIC has the potential to be helpful in game development but suffers from an overly specialized approach.

nibanks 6y ago
Yes, msquic should be a good general purpose transport. We already have usage from SMB (file sharing) and HTTP in Windows. Both are very different and provided good test cases for msquic.

profquail 6y ago

Have you considered implementing any parts of this in F* (so they can be verified) and extracting back to C, as is being done for TLS? https://project-everest.github.io/ https://project-everest.github.io/

nibanks 6y ago
We do work with the Everest team. We have unofficial support on top of miTLS (which they produce). We haven't looked into actually using F* for any of the QUIC code though.
catalin_hritcu 6y ago
Some work on verifying QUIC packet encryption using F* is happening at Microsoft Research: https://github.com/project-everest/everquic-crypto https://github.com/project-everest/everquic-crypto
- protz 6y ago
  Just to build on Catalin's answer. We are actively working on an implementation of QUIC's transport layer (i.e. packet encryption and decryption), along with a proof of cryptographic security. This is what Catalin linked to (https://github.com/project-everest/everquic-crypto https://github.com/project-everest/everquic-crypto). EverQuic-Crypto builds upon two previous projects: EverParse, a library of verified low-level parsers and serializers which we apply to the QUIC network formats, and EverCrypt, a cryptographic provider with agility and multiplexing, which we use for all the cryptography, e.g. packet number encryption, AEAD, etc. This is not yet a full QUIC implementation, but we have plans for extending this codebase to cover more of the QUIC protocol.

skyde 6y ago

will this be integrated in some rpc framework like gRPC we all can use ?

skyde 6y ago

is this using BBR congestion algorithm ?

nibanks 6y ago
Not yet. BBR is on the TODO list though.

skyde 6y ago

will it be possible to use the « sendfile » system call to do zero copy file transfers on a quick connection?