3 ms·

Throwing out a few points regarding DPDK/low overhead data/event exchange between kernel and userspace: 1. It is faster if you make it so, i.e., you need to in

Throwing out a few points regarding DPDK/low overhead data/event exchange between kernel and userspace: 1. It is faster if you make it so, i.e., you need to invest quite some work to get to a point where it's faster than kernel - but yea, with that you can make it go brrrrrrrt in quite a few use cases, it's more like an SDK. 2. Some NICs support loading a smaller program to operate directly on the NIC, e.g., through eBPF - nothing gets faster than that as it avoids that (some) traffic hits the CPU at all. But again, it's work to create and maintain and only feasible for specialized stuff. 3. Note that with Linux[0] a lot of the context switch overhead can be avoided by leveraging the state of the art async framework IO-uring, BSD may already have or get something similarily[0]. I recommend reading the paper of the original author[0], it's fascinating IMO. [0]: I know this is about BSD, so somewhat off-topic, but still shows that most of the context switch complexity can be avoided while not re-inventing the wheels in user-space. [1]: https://kernel.dk/io_uring.pdf https://kernel.dk/io_uring.pdf

0x000000E2 5y ago
I'll add that with DPDK you end up needing your own TCP stack if you're doing anything above layer 3. TCP is a beast with opportunity for countless subtle bugs. The Linux kernel is perhaps the best implementation there is. Many userspace implementations are simplified and missing features. Even giants like Cloudflare use Linux kernel for routing when they need to operate above layer 3. They only use DPDK for very low level features. DPDK has its place but for most use cases you need to man handle TCP and you're better off using kernel packet handling for that
- gonzo 5y ago
  > use Linux kernel for routing when they need to operate above layer 3. What could you possibly mean by this?
  - 0x000000E2 5y ago
    When they need to handle UPD and TCP. Cloudflare's web application firewalls run at TCP and HTTP layer and use tuned Linux kernels for processing, not DPDK and userspace TCP as some think Their line rate stuff for flood mitigation uses DPDK or similar

0x000000E2 5y ago

I'll add that with DPDK you end up needing your own TCP stack if you're doing anything above layer 3. TCP is a beast with opportunity for countless subtle bugs. The Linux kernel is perhaps the best implementation there is. Many userspace implementations are simplified and missing features. Even giants like Cloudflare use Linux kernel for routing when they need to operate above layer 3. They only use DPDK for very low level features. DPDK has its place but for most use cases you need to man handle TCP and you're better off using kernel packet handling for that

gonzo 5y ago
> use Linux kernel for routing when they need to operate above layer 3. What could you possibly mean by this?
- 0x000000E2 5y ago
  When they need to handle UPD and TCP. Cloudflare's web application firewalls run at TCP and HTTP layer and use tuned Linux kernels for processing, not DPDK and userspace TCP as some think Their line rate stuff for flood mitigation uses DPDK or similar