5 ms·

Seems to me like if you need to do your task inside the kernel to get acceptable performance, there's something wrong with that kernel. It's also arguably more

Seems to me like if you need to do your task inside the kernel to get acceptable performance, there's something wrong with that kernel. It's also arguably more difficult to do a given task in the kernel than in userland, code for the kernel is much more security-sensitive and even subtle bugs can be detrimental to overall system performance (or even exploitable). Best thing would be to fix the OS instead of piling up kludges.

sorenjan 5y ago
What do you suggest that the Wireguard team should do about that?
Godel_unicode 5y ago
Basically all high performance VPN implementations are kernel-mode. Just because something is difficult does not make it automatically wrong. Note that the main userland VPN which gets cited (OpenVPN) has terrible battery and latency performance. https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-it-to-1-0-0-and-into-the-next-linux-kernel/ https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-... https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&format=html https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&fo...
eptcyka 5y ago
How would one circumvent the need to copy traffic between different userspace processes before it leaves the host in the case of a userspace tunnel implementation? With a kernel implementation, the userspace process sends some data to a socket, and then the kernel can directly pass the data to the tunnel implementation with as little copying as the design for the kernel allows, and as little context switching as the design allows.
- Dylan16807 5y ago
  > How would do one circumvent the need to copy traffic between different userspace processes By mapping the same buffer into both processes, or moving it from one process to the other, depending on security needs. Mapping the buffer into multiple memory spaces is how a kernel VPN driver would accomplish the same thing. > as little context switching as the design allows A raw context switch only costs 20 nanoseconds for a round trip. And it's only a few more to save the normal registers. The rest is up to kernel design. So it sure sounds like it's the kernel's fault if putting tasks like that outside the kernel is too expensive. Especially since one context switch can process as many packets as you want.
  - DSingularity 5y ago
    These days a lot of the context switch delays come from security mitigation’s due to vulnerable processors. Flushing TLBs. Flushing caches. Etc etc.
  - zx2c4 5y ago
    Wintun -- https://www.wintun.net https://www.wintun.net -- actually uses this technique via shared memory ring buffers. And Winsock RIO has something similar for packets. We were using both of these already prior to WireGuardNT. They're fast but not conclusively so.
- topspin 5y ago
  "How would one circumvent the need to copy traffic" Fixing that is the point of, for example, io_uring in Linux. I don't know if Windows has an equivalent. Perhaps not, thus a kernel driver. But there is nothing mysterious about solving that problem.
  - eptcyka 5y ago
    The original, canonical implementation of WireGuard on Linux predates io_during, and Windows has had IOCP for decades now.
agent327 5y ago
Hear, hear! I've always said microkernels are the best. If you have thousands of drivers in your kernel, each one is just another attack surface, something that can take the machine down on something as simple as RS232 traffic. Instead drivers should be isolated in their own process. It has lots of advantages: * Dynamic loading of drivers is no longer an issue, the kernel already knows how to dynamically load processes anyway and a driver is just another proces. No need for special 'kernel maintainers' for drivers, or for drivers to be open source (in case of the Linux kernel). * Much better system stability, since drivers can do no harm. Kernel architecture can be simpler too. * Much simpler drivers. Instead of strict cooperation rules the kernel can just pre-empt them when needed. Unfortunately it appears there is no credible effort in developing a mainstream microkernel OS at this time. Nonetheless, the few I've worked with in the past were amazing and I'd love to see this idea come back.
- corty 5y ago
  Context switching overhead is bad for microkernel performance. All the reasons why the userspace wireguard implementation was slower apply to each and every part of a microkernel system. And all of this got worse with spectre and meltdown, secure context switches are now even more expensive. Modern CPUs generally always increase the context switching cost, they optimize for single-process benchmarks. So microkernels are dead, performance buries them deeper and deeper.

sorenjan 5y ago

What do you suggest that the Wireguard team should do about that?

Godel_unicode 5y ago

Basically all high performance VPN implementations are kernel-mode. Just because something is difficult does not make it automatically wrong. Note that the main userland VPN which gets cited (OpenVPN) has terrible battery and latency performance. https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-it-to-1-0-0-and-into-the-next-linux-kernel/ https://arstechnica.com/gadgets/2020/03/wireguard-vpn-makes-... https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&format=html https://www.freebsd.org/cgi/man.cgi?query=ipsec&sektion=4&fo...

eptcyka 5y ago

How would one circumvent the need to copy traffic between different userspace processes before it leaves the host in the case of a userspace tunnel implementation? With a kernel implementation, the userspace process sends some data to a socket, and then the kernel can directly pass the data to the tunnel implementation with as little copying as the design for the kernel allows, and as little context switching as the design allows.

Dylan16807 5y ago
> How would do one circumvent the need to copy traffic between different userspace processes By mapping the same buffer into both processes, or moving it from one process to the other, depending on security needs. Mapping the buffer into multiple memory spaces is how a kernel VPN driver would accomplish the same thing. > as little context switching as the design allows A raw context switch only costs 20 nanoseconds for a round trip. And it's only a few more to save the normal registers. The rest is up to kernel design. So it sure sounds like it's the kernel's fault if putting tasks like that outside the kernel is too expensive. Especially since one context switch can process as many packets as you want.
- DSingularity 5y ago
  These days a lot of the context switch delays come from security mitigation’s due to vulnerable processors. Flushing TLBs. Flushing caches. Etc etc.
- zx2c4 5y ago
  Wintun -- https://www.wintun.net https://www.wintun.net -- actually uses this technique via shared memory ring buffers. And Winsock RIO has something similar for packets. We were using both of these already prior to WireGuardNT. They're fast but not conclusively so.
topspin 5y ago
"How would one circumvent the need to copy traffic" Fixing that is the point of, for example, io_uring in Linux. I don't know if Windows has an equivalent. Perhaps not, thus a kernel driver. But there is nothing mysterious about solving that problem.
- eptcyka 5y ago
  The original, canonical implementation of WireGuard on Linux predates io_during, and Windows has had IOCP for decades now.

agent327 5y ago

Hear, hear! I've always said microkernels are the best. If you have thousands of drivers in your kernel, each one is just another attack surface, something that can take the machine down on something as simple as RS232 traffic. Instead drivers should be isolated in their own process. It has lots of advantages: * Dynamic loading of drivers is no longer an issue, the kernel already knows how to dynamically load processes anyway and a driver is just another proces. No need for special 'kernel maintainers' for drivers, or for drivers to be open source (in case of the Linux kernel). * Much better system stability, since drivers can do no harm. Kernel architecture can be simpler too. * Much simpler drivers. Instead of strict cooperation rules the kernel can just pre-empt them when needed. Unfortunately it appears there is no credible effort in developing a mainstream microkernel OS at this time. Nonetheless, the few I've worked with in the past were amazing and I'd love to see this idea come back.

corty 5y ago
Context switching overhead is bad for microkernel performance. All the reasons why the userspace wireguard implementation was slower apply to each and every part of a microkernel system. And all of this got worse with spectre and meltdown, secure context switches are now even more expensive. Modern CPUs generally always increase the context switching cost, they optimize for single-process benchmarks. So microkernels are dead, performance buries them deeper and deeper.