5 ms·

Oh, and why would you want to have it as part of the kernel?

by comboy 5y ago

SigmundA 5y ago
Performance: https://www.netgate.com/blog/wireguard-in-pfsense-2-5-performance.html https://www.netgate.com/blog/wireguard-in-pfsense-2-5-perfor...
- carlhjerpe 5y ago
  DPDK is fast in userspace by taking ownership of devices (I believe) so that there's no context switching anyways.
  - SigmundA 5y ago
    My understanding is DPDK has much smaller set a supported nic's since it doesn't use the kernel drivers for the card along with all the other things that the kernel provides. Wireguard in kernel gives great performance while just working if your kernel is already working on the hardware. Wireguard is so small it is considered easy to audit so is a prime candidate for kernel implementation. DPDK is listed as an "Exotic" todo for Wireguard: https://www.wireguard.com/todo/ https://www.wireguard.com/todo/
    - gonzo 5y ago
      There is already an implementation of Wireguard in VPP, which runs over DPDK.
  - tlamponi 5y ago
    Throwing out a few points regarding DPDK/low overhead data/event exchange between kernel and userspace: 1. It is faster if you make it so, i.e., you need to invest quite some work to get to a point where it's faster than kernel - but yea, with that you can make it go brrrrrrrt in quite a few use cases, it's more like an SDK. 2. Some NICs support loading a smaller program to operate directly on the NIC, e.g., through eBPF - nothing gets faster than that as it avoids that (some) traffic hits the CPU at all. But again, it's work to create and maintain and only feasible for specialized stuff. 3. Note that with Linux[0] a lot of the context switch overhead can be avoided by leveraging the state of the art async framework IO-uring, BSD may already have or get something similarily[0]. I recommend reading the paper of the original author[0], it's fascinating IMO. [0]: I know this is about BSD, so somewhat off-topic, but still shows that most of the context switch complexity can be avoided while not re-inventing the wheels in user-space. [1]: https://kernel.dk/io_uring.pdf https://kernel.dk/io_uring.pdf
    - 0x000000E2 5y ago
      I'll add that with DPDK you end up needing your own TCP stack if you're doing anything above layer 3. TCP is a beast with opportunity for countless subtle bugs. The Linux kernel is perhaps the best implementation there is. Many userspace implementations are simplified and missing features. Even giants like Cloudflare use Linux kernel for routing when they need to operate above layer 3. They only use DPDK for very low level features. DPDK has its place but for most use cases you need to man handle TCP and you're better off using kernel packet handling for that
      - gonzo 5y ago
        > use Linux kernel for routing when they need to operate above layer 3. What could you possibly mean by this?
        0x000000E2 5y ago
        When they need to handle UPD and TCP. Cloudflare's web application firewalls run at TCP and HTTP layer and use tuned Linux kernels for processing, not DPDK and userspace TCP as some think Their line rate stuff for flood mitigation uses DPDK or similar
  - gonzo 5y ago
    Nothing there was DPDK, just FreeBSD kernel networking. The Wireguard implementation in VPP runs at about 5Gbps in a simple single-stream iperf test.
    - SigmundA 5y ago
      5Gbps on what hardware and how does that compare to the kernel implementation on the same hardware? Curious to know how big a difference DPDK makes. Also what about cpu time, I know typically DPDK takes more CPU due to polling, normally want to dedicate cpus to DPDK right?
      - gonzo 5y ago
        Very new Xeon at 3.5GHz. 5.75gbps. Kernel Wireguard is almost 3gbps. But Wireguard is mostly held back by the slow crypto implementation, which is CPU-bound. AES-GCM IPsec on the same hardware is 19.5gbps, limited by the 25gbps NIC, using again, single-stream iperf with ipsec in VPP. DPDK only needs dedicated cores for the poll mode drivers, and these days the Intel and mellanox NICs can be run in interrupt mode.
        SigmundA 5y ago
        I thought interrupt mode reduced performance, that was the tradeoff, is the throughput your quoting in interrupt or polling mode? Good numbers either way, but you again much more limited hardware support since the drivers need to be DPDK specific.
        gonzo 5y ago
        You can also run VPP over AF_PACKET (slower, but faster than the kernel) of AF_XDP (almost as fast as DPDK interrupt mode.) But Wireguard is still crypto-limited, the AES-GCM result proves it.
        SigmundA 5y ago
        Wireguard isn't doing to bad considering its not hardware accelerated like AES-GCM. Also looks like AF_PACKET and AF_XDP are basically an alternative to DPDK with similar functionality but built into the kernel using kernel drivers, new to me.
messe 5y ago
As the other poster said, performance. The idea behind wireguard was that it would be a VPN protocol small enough to implement securely in kernel space.
- deleted 5y ago
  [deleted]
gonzo 5y ago
Even Donenfeld has said the Userspace implementation in Go isn’t very good.
- atatatat 5y ago
  Is it even 1.0 yet...?
0x000000E2 5y ago
I've given up on Xsense distros and use OpenWrt. It's Linux based so Wiregaurd is in kernel and easy to setup. I keep trying to get people to try it out :) . I left pfSense community years ago when drama started and it became clear somebody was commercializing it. Linux based distros have better features and hardware compatibility these days anyways. BSD is slowly dying (at risk of downvotes here)

Oh, and why would you want to have it as part of the kernel?

by comboy 5y ago

Oh, and why would you want to have it as part of the kernel?

Performance: https://www.netgate.com/blog/wireguard-in-pfsense-2-5-performance.html https://www.netgate.com/blog/wireguard-in-pfsense-2-5-perfor...

carlhjerpe 5y ago
DPDK is fast in userspace by taking ownership of devices (I believe) so that there's no context switching anyways.
- SigmundA 5y ago
  My understanding is DPDK has much smaller set a supported nic's since it doesn't use the kernel drivers for the card along with all the other things that the kernel provides. Wireguard in kernel gives great performance while just working if your kernel is already working on the hardware. Wireguard is so small it is considered easy to audit so is a prime candidate for kernel implementation. DPDK is listed as an "Exotic" todo for Wireguard: https://www.wireguard.com/todo/ https://www.wireguard.com/todo/
  - gonzo 5y ago
    There is already an implementation of Wireguard in VPP, which runs over DPDK.
- tlamponi 5y ago
  Throwing out a few points regarding DPDK/low overhead data/event exchange between kernel and userspace: 1. It is faster if you make it so, i.e., you need to invest quite some work to get to a point where it's faster than kernel - but yea, with that you can make it go brrrrrrrt in quite a few use cases, it's more like an SDK. 2. Some NICs support loading a smaller program to operate directly on the NIC, e.g., through eBPF - nothing gets faster than that as it avoids that (some) traffic hits the CPU at all. But again, it's work to create and maintain and only feasible for specialized stuff. 3. Note that with Linux[0] a lot of the context switch overhead can be avoided by leveraging the state of the art async framework IO-uring, BSD may already have or get something similarily[0]. I recommend reading the paper of the original author[0], it's fascinating IMO. [0]: I know this is about BSD, so somewhat off-topic, but still shows that most of the context switch complexity can be avoided while not re-inventing the wheels in user-space. [1]: https://kernel.dk/io_uring.pdf https://kernel.dk/io_uring.pdf
  - 0x000000E2 5y ago
    I'll add that with DPDK you end up needing your own TCP stack if you're doing anything above layer 3. TCP is a beast with opportunity for countless subtle bugs. The Linux kernel is perhaps the best implementation there is. Many userspace implementations are simplified and missing features. Even giants like Cloudflare use Linux kernel for routing when they need to operate above layer 3. They only use DPDK for very low level features. DPDK has its place but for most use cases you need to man handle TCP and you're better off using kernel packet handling for that
    - gonzo 5y ago
      > use Linux kernel for routing when they need to operate above layer 3. What could you possibly mean by this?
      - 0x000000E2 5y ago
        When they need to handle UPD and TCP. Cloudflare's web application firewalls run at TCP and HTTP layer and use tuned Linux kernels for processing, not DPDK and userspace TCP as some think Their line rate stuff for flood mitigation uses DPDK or similar
- gonzo 5y ago
  Nothing there was DPDK, just FreeBSD kernel networking. The Wireguard implementation in VPP runs at about 5Gbps in a simple single-stream iperf test.
  - SigmundA 5y ago
    5Gbps on what hardware and how does that compare to the kernel implementation on the same hardware? Curious to know how big a difference DPDK makes. Also what about cpu time, I know typically DPDK takes more CPU due to polling, normally want to dedicate cpus to DPDK right?
    - gonzo 5y ago
      Very new Xeon at 3.5GHz. 5.75gbps. Kernel Wireguard is almost 3gbps. But Wireguard is mostly held back by the slow crypto implementation, which is CPU-bound. AES-GCM IPsec on the same hardware is 19.5gbps, limited by the 25gbps NIC, using again, single-stream iperf with ipsec in VPP. DPDK only needs dedicated cores for the poll mode drivers, and these days the Intel and mellanox NICs can be run in interrupt mode.
      - SigmundA 5y ago
        I thought interrupt mode reduced performance, that was the tradeoff, is the throughput your quoting in interrupt or polling mode? Good numbers either way, but you again much more limited hardware support since the drivers need to be DPDK specific.
        gonzo 5y ago
        You can also run VPP over AF_PACKET (slower, but faster than the kernel) of AF_XDP (almost as fast as DPDK interrupt mode.) But Wireguard is still crypto-limited, the AES-GCM result proves it.
        SigmundA 5y ago
        Wireguard isn't doing to bad considering its not hardware accelerated like AES-GCM. Also looks like AF_PACKET and AF_XDP are basically an alternative to DPDK with similar functionality but built into the kernel using kernel drivers, new to me.

messe 5y ago

As the other poster said, performance. The idea behind wireguard was that it would be a VPN protocol small enough to implement securely in kernel space.

deleted 5y ago
[deleted]

gonzo 5y ago

Even Donenfeld has said the Userspace implementation in Go isn’t very good.

atatatat 5y ago
Is it even 1.0 yet...?

0x000000E2 5y ago

I've given up on Xsense distros and use OpenWrt. It's Linux based so Wiregaurd is in kernel and easy to setup. I keep trying to get people to try it out :) . I left pfSense community years ago when drama started and it became clear somebody was commercializing it. Linux based distros have better features and hardware compatibility these days anyways. BSD is slowly dying (at risk of downvotes here)