5 ms·

Security is the top priority. But being useful is also important; if nobody can use it, nobody benefits from that security. Acceptable performance is very impor

by zx2c4 5y ago

Security is the top priority. But being useful is also important; if nobody can use it, nobody benefits from that security. Acceptable performance is very important for being useful, especially when tunneling layer 3 packets, where latency has rippling effects. The prior non-kernel WireGuard for Windows simply was not sufficiently useful for real world workloads people wanted to run, because of the lower performance, both on servers and on laptop wifi alike. A big part of the WireGuard project since the beginning has been trying to figure out how to do high security tunneling that also performs acceptably. It's easy to do one without the other, but doing them both together has meant thinking about a lot of fundamentals, from the protocol state machines on up. It's hard to do tunneling in the kernel at high speed while still maintaining a strong security posture. That's a principal challenge the project endeavors to solve. More generally, it's worth noting that cryptographers also care about performance quite a bit in things like symmetric crypto. We know well how to make a good cipher now, but making one that also performs at increasingly high speeds remains an open area of research, with whole conferences, such as FSE, devoted to it.

pdenton 5y ago
Seems to me like if you need to do your task inside the kernel to get acceptable performance, there's something wrong with that kernel. It's also arguably more difficult to do a given task in the kernel than in userland, code for the kernel is much more security-sensitive and even subtle bugs can be detrimental to overall system performance (or even exploitable). Best thing would be to fix the OS instead of piling up kludges.
- sorenjan 5y ago
  What do you suggest that the Wireguard team should do about that?
- Godel_unicode 5y ago
  Basically all high performance VPN implementations are kernel-mode. Just because something is difficult does not make it automatically wrong. Note that the main userland VPN which gets cited (OpenVPN) has terrible battery and latency performance. https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-it-to-1-0-0-and-into-the-next-linux-kernel/ https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-... https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&format=html https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&fo...
- eptcyka 5y ago
  How would one circumvent the need to copy traffic between different userspace processes before it leaves the host in the case of a userspace tunnel implementation? With a kernel implementation, the userspace process sends some data to a socket, and then the kernel can directly pass the data to the tunnel implementation with as little copying as the design for the kernel allows, and as little context switching as the design allows.
  - Dylan16807 5y ago
    > How would do one circumvent the need to copy traffic between different userspace processes By mapping the same buffer into both processes, or moving it from one process to the other, depending on security needs. Mapping the buffer into multiple memory spaces is how a kernel VPN driver would accomplish the same thing. > as little context switching as the design allows A raw context switch only costs 20 nanoseconds for a round trip. And it's only a few more to save the normal registers. The rest is up to kernel design. So it sure sounds like it's the kernel's fault if putting tasks like that outside the kernel is too expensive. Especially since one context switch can process as many packets as you want.
    - DSingularity 5y ago
      These days a lot of the context switch delays come from security mitigation’s due to vulnerable processors. Flushing TLBs. Flushing caches. Etc etc.
    - zx2c4 5y ago
      Wintun -- https://www.wintun.net https://www.wintun.net -- actually uses this technique via shared memory ring buffers. And Winsock RIO has something similar for packets. We were using both of these already prior to WireGuardNT. They're fast but not conclusively so.
  - topspin 5y ago
    "How would one circumvent the need to copy traffic" Fixing that is the point of, for example, io_uring in Linux. I don't know if Windows has an equivalent. Perhaps not, thus a kernel driver. But there is nothing mysterious about solving that problem.
- agent327 5y ago
  Hear, hear! I've always said microkernels are the best. If you have thousands of drivers in your kernel, each one is just another attack surface, something that can take the machine down on something as simple as RS232 traffic. Instead drivers should be isolated in their own process. It has lots of advantages: * Dynamic loading of drivers is no longer an issue, the kernel already knows how to dynamically load processes anyway and a driver is just another proces. No need for special 'kernel maintainers' for drivers, or for drivers to be open source (in case of the Linux kernel). * Much better system stability, since drivers can do no harm. Kernel architecture can be simpler too. * Much simpler drivers. Instead of strict cooperation rules the kernel can just pre-empt them when needed. Unfortunately it appears there is no credible effort in developing a mainstream microkernel OS at this time. Nonetheless, the few I've worked with in the past were amazing and I'd love to see this idea come back.
  - corty 5y ago
    Context switching overhead is bad for microkernel performance. All the reasons why the userspace wireguard implementation was slower apply to each and every part of a microkernel system. And all of this got worse with spectre and meltdown, secure context switches are now even more expensive. Modern CPUs generally always increase the context switching cost, they optimize for single-process benchmarks. So microkernels are dead, performance buries them deeper and deeper.
tptacek 5y ago
The world is mired in IPSEC and OpenVPN, two of the Internet's great buzzing stagnant water pools. Part of the reason for that is that proponents (anyone who has gotten IPSEC working is inclined to be a proponent, since it's an investment of effort and brain space one might reasonably hope would pay off) can appeal to standards, known-quantity-ism, and multi-vendor ecosystems. Legacy VPN protocols fare terribly against WireGuard on what should be the most important concern, of design and implementation security. But the reasons why are, though obvious to people with domain expertise, subtle enough that you can't mic-drop them in a conversation and expect to overcome the standard "but it's standard" objection in favor of IPSEC. Performance has been a potent tool of persuasion for WireGuard. I've convinced people based on WireGuard speed that I don't think I could have persuaded based on security. If the goal is getting people off fetid 1990s-grade cryptography and C code, WireGuard's performance is an indirect force towards that goal. It was smart to focus on it, because people obviously care a lot about it. It's great to see a security project where simplicity and performance line up with improved security.
- toast0 5y ago
  > anyone who has gotten IPSEC working is inclined to be a proponent, since it's an investment of effort and brain space one might reasonably hope would pay off Oh dear me, getting IPSEC to work did not win it any admiration from me. Of course, what I was doing with it would have been better served by TLS with server and client certificates, but the other end were telcos so sure, I can setup IPSEC and cry myself to sleep.
- chousuke 5y ago
  If I could I would instantly replace every L2L IPsec tunnel configured everywhere I have access, but unfortunately until Cisco, Juniper, Fortigate etc. implement Wireguard, IPsec remains the only option. I am already using it for production purposes where both endpoints are Linux, though.
- znpy 5y ago
  > subtle enough that you can't mic-drop them in a conversation and expect to overcome the standard "but it's standard" objection in favor of IPSEC. I wonder if we shouldn't standardize something very similar to wireguard as a general successor of ipsec. Something like wireguard + iana/ietf considerations.
combyn8tor 5y ago
Hi Jason, are there any plans for process based routing within the WindowsNT driver? For example, if a Windows user wanted to route only Chrome through a Wireguard tunnel but leave all other applications network traffic untouched.
- zx2c4 5y ago
  Routing is actually a layer before packets make it to the WireGuardNT miniport driver. Usually the routing table controls this, but WFP callout drivers can also steer traffic to specific adapters based on things like process image path or security descriptor. So for this functionality, WireGuardNT can be used with various WFP drivers or routing table configurations. It's not something that'd belong in WireGuardNT itself. With regards to the WireGuard for Windows client, though, whether we'll put the time in to develop such a WFP driver and integrate it there I guess is still up in the air, like many feature wishlist items are.
  - combyn8tor 5y ago
    Okay that makes sense, thanks for the info. I'm new to Windows networking and thought WireguardNT was somehow using a WFP callout to encrypt plus reroute packets and then drop the original. Keep up the great work you're doing!

by zx2c4 5y ago

pdenton 5y ago

Seems to me like if you need to do your task inside the kernel to get acceptable performance, there's something wrong with that kernel. It's also arguably more difficult to do a given task in the kernel than in userland, code for the kernel is much more security-sensitive and even subtle bugs can be detrimental to overall system performance (or even exploitable). Best thing would be to fix the OS instead of piling up kludges.

sorenjan 5y ago
What do you suggest that the Wireguard team should do about that?
Godel_unicode 5y ago
Basically all high performance VPN implementations are kernel-mode. Just because something is difficult does not make it automatically wrong. Note that the main userland VPN which gets cited (OpenVPN) has terrible battery and latency performance. https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-it-to-1-0-0-and-into-the-next-linux-kernel/ https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-... https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&format=html https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&fo...
eptcyka 5y ago
How would one circumvent the need to copy traffic between different userspace processes before it leaves the host in the case of a userspace tunnel implementation? With a kernel implementation, the userspace process sends some data to a socket, and then the kernel can directly pass the data to the tunnel implementation with as little copying as the design for the kernel allows, and as little context switching as the design allows.
- Dylan16807 5y ago
  > How would do one circumvent the need to copy traffic between different userspace processes By mapping the same buffer into both processes, or moving it from one process to the other, depending on security needs. Mapping the buffer into multiple memory spaces is how a kernel VPN driver would accomplish the same thing. > as little context switching as the design allows A raw context switch only costs 20 nanoseconds for a round trip. And it's only a few more to save the normal registers. The rest is up to kernel design. So it sure sounds like it's the kernel's fault if putting tasks like that outside the kernel is too expensive. Especially since one context switch can process as many packets as you want.
  - DSingularity 5y ago
    These days a lot of the context switch delays come from security mitigation’s due to vulnerable processors. Flushing TLBs. Flushing caches. Etc etc.
  - zx2c4 5y ago
    Wintun -- https://www.wintun.net https://www.wintun.net -- actually uses this technique via shared memory ring buffers. And Winsock RIO has something similar for packets. We were using both of these already prior to WireGuardNT. They're fast but not conclusively so.
- topspin 5y ago
  "How would one circumvent the need to copy traffic" Fixing that is the point of, for example, io_uring in Linux. I don't know if Windows has an equivalent. Perhaps not, thus a kernel driver. But there is nothing mysterious about solving that problem.
agent327 5y ago
Hear, hear! I've always said microkernels are the best. If you have thousands of drivers in your kernel, each one is just another attack surface, something that can take the machine down on something as simple as RS232 traffic. Instead drivers should be isolated in their own process. It has lots of advantages: * Dynamic loading of drivers is no longer an issue, the kernel already knows how to dynamically load processes anyway and a driver is just another proces. No need for special 'kernel maintainers' for drivers, or for drivers to be open source (in case of the Linux kernel). * Much better system stability, since drivers can do no harm. Kernel architecture can be simpler too. * Much simpler drivers. Instead of strict cooperation rules the kernel can just pre-empt them when needed. Unfortunately it appears there is no credible effort in developing a mainstream microkernel OS at this time. Nonetheless, the few I've worked with in the past were amazing and I'd love to see this idea come back.
- corty 5y ago
  Context switching overhead is bad for microkernel performance. All the reasons why the userspace wireguard implementation was slower apply to each and every part of a microkernel system. And all of this got worse with spectre and meltdown, secure context switches are now even more expensive. Modern CPUs generally always increase the context switching cost, they optimize for single-process benchmarks. So microkernels are dead, performance buries them deeper and deeper.

tptacek 5y ago

The world is mired in IPSEC and OpenVPN, two of the Internet's great buzzing stagnant water pools. Part of the reason for that is that proponents (anyone who has gotten IPSEC working is inclined to be a proponent, since it's an investment of effort and brain space one might reasonably hope would pay off) can appeal to standards, known-quantity-ism, and multi-vendor ecosystems. Legacy VPN protocols fare terribly against WireGuard on what should be the most important concern, of design and implementation security. But the reasons why are, though obvious to people with domain expertise, subtle enough that you can't mic-drop them in a conversation and expect to overcome the standard "but it's standard" objection in favor of IPSEC. Performance has been a potent tool of persuasion for WireGuard. I've convinced people based on WireGuard speed that I don't think I could have persuaded based on security. If the goal is getting people off fetid 1990s-grade cryptography and C code, WireGuard's performance is an indirect force towards that goal. It was smart to focus on it, because people obviously care a lot about it. It's great to see a security project where simplicity and performance line up with improved security.

toast0 5y ago
> anyone who has gotten IPSEC working is inclined to be a proponent, since it's an investment of effort and brain space one might reasonably hope would pay off Oh dear me, getting IPSEC to work did not win it any admiration from me. Of course, what I was doing with it would have been better served by TLS with server and client certificates, but the other end were telcos so sure, I can setup IPSEC and cry myself to sleep.
chousuke 5y ago
If I could I would instantly replace every L2L IPsec tunnel configured everywhere I have access, but unfortunately until Cisco, Juniper, Fortigate etc. implement Wireguard, IPsec remains the only option. I am already using it for production purposes where both endpoints are Linux, though.
znpy 5y ago
> subtle enough that you can't mic-drop them in a conversation and expect to overcome the standard "but it's standard" objection in favor of IPSEC. I wonder if we shouldn't standardize something very similar to wireguard as a general successor of ipsec. Something like wireguard + iana/ietf considerations.

combyn8tor 5y ago

Hi Jason, are there any plans for process based routing within the WindowsNT driver? For example, if a Windows user wanted to route only Chrome through a Wireguard tunnel but leave all other applications network traffic untouched.

zx2c4 5y ago
Routing is actually a layer before packets make it to the WireGuardNT miniport driver. Usually the routing table controls this, but WFP callout drivers can also steer traffic to specific adapters based on things like process image path or security descriptor. So for this functionality, WireGuardNT can be used with various WFP drivers or routing table configurations. It's not something that'd belong in WireGuardNT itself. With regards to the WireGuard for Windows client, though, whether we'll put the time in to develop such a WFP driver and integrate it there I guess is still up in the air, like many feature wishlist items are.
- combyn8tor 5y ago
  Okay that makes sense, thanks for the info. I'm new to Windows networking and thought WireguardNT was somehow using a WFP callout to encrypt plus reroute packets and then drop the original. Keep up the great work you're doing!