4 ms·

For those like me going "......what is dpdk" The Data Plane Development Kit (DPDK) is an open source software project managed by the Linux Foundation. It p

For those like me going "......what is dpdk" The Data Plane Development Kit (DPDK) is an open source software project managed by the Linux Foundation. It provides a set of data plane libraries and network interface controller polling-mode drivers for offloading TCP packet processing from the operating system kernel to processes running in user space. This offloading achieves higher computing efficiency and higher packet throughput than is possible using the interrupt-driven processing provided in the kernel. https://en.wikipedia.org/wiki/Data_Plane_Development_Kit https://en.wikipedia.org/wiki/Data_Plane_Development_Kit https://www.packetcoders.io/what-is-dpdk/ https://www.packetcoders.io/what-is-dpdk/

jaimex2 4y ago
Basically its what people who want to bypass the kernel network stack because they think its slow. They then spend the next few years writing their own stack till they realise they've just re-written what the kernel does and its slower and full of exploits. Yeah, receiving packets is fast when you aren't doing anything with them.
- AlphaSite 4y ago
  Apps probably don’t but networking related code might benefit a lot from DPDK.
- dclusin 4y ago
  It’s typically done when end to end latency is more important than full protocol compliance and hardening against myriad types of attackers. High frequency trading is typically where the applications where you see these sorts of implementations being really compelling. For these types of applications there are proprietary implementations that you can buy from vendors that are more suited to latency sensitive applications. The next level of optimization after kernel bypass is to build or buy FPGAs which implement the wire protocol + transport as an integrated circuit.
  - grive 4y ago
    FPGAs are falling out of favor, ASICs are sufficient to implement most network processing. Funnily, one of the biggest DPDK feature is an API to program smartNICs exactly in that way.
    - dclusin 4y ago
      Fair enough. My Knowledge of what’s on the bleeding edge and mass adoption in algo trading is fairly dated. I haven’t been adjacent to that industry in over 5 years. Even back then there was turnkey ASICS you could buy that would implement the network plus whatever protocols you used to talk to the exchange. I think the neatest thing that sticks out to me is how logging in implementations of low latency trading applications was essentially pushed to the network. Basically they just have copper or fiber taps between each node in their system and it gets sent to another box that aggregates the traffic and processes all the packets to provide trace level logging through their entire system. Even these have solutions you can buy from a number of vendors.
- ardel95 4y ago
  DPDK is common within network appliances, where you don't need most of the features within kernel networking stack.
- shaklee3 4y ago
  this is completely wrong. as the article points out, there are plenty of tcp stacks and other user space stacks available. dpdk is used in places where performance matters, and just because you haven't worked in that area doesn't mean it's useless. all telcos use it for the most part.
- wumpus 4y ago
  > They then spend the next few years writing their own stack Supercomputing people have done this repeatedly and successfully. A more general purpose stack is UDP-based applications. I wouldn't do this with TCP, which I agree is complicated and difficult to get right.
- touisteur 4y ago
  Sometimes you really don't do much more on packets than receive and store in (e.g) ring buffers until you can trigger some computation on aggregated data. Sometimes your application has critical latency and you need very light parsing. Not everyone uses tcp and all the options, some people are just down to jumbo udp packets, no fragmentation, and 800Gb/s rx...
- gjulianm 4y ago
  That's not the reality. The kernel network stack does a lot of things that for some applications are not needed, so DPDK is used for those cases when they need very high performance. For example, packet capture, routing, packet generation... I don't think I've seen anyone rewriting what the kernel does precisely because when you use DPDK you don't want to do what the kernel does.
- grive 4y ago
  Networking stacks are used in two ways: endpoints and forwarders. Forwarders are usually not doing much with packets, just reading a few fields and choosing an output port. These are very common function in network cores (telcos, datacenters, ...). DPDK is not well-suited for endpoint application programming, even though here you can still squeeze some additional performance. But don't dismiss a framework that is currently so widely deployed just because you are not familiar with its most common use-case.
- baruch 4y ago
  Normally the endpoint stack you need is more limited initially and you don't have to have all the baggage. The biggest advantage you get at the end is zero copy network and if what you do involves tons of reading and writing to the network you can definitely gain performance in both bandwidth and latency by better using your CPU. You can do XDP as well and gain zero copy but for a small subset of NICs and you still need to implement you own network stack I don't see ac way to avoid it much. There are existing TCP stacks for DPDK that you can use as well.
harpratap 4y ago
How does this differ from eBPF?
- monocasa 4y ago
  eBPF pushes computation into the kernel to be colocated with kernel data/events. DPDK pushes all of the relevant data (the network packets) into user space to be colocated with user data/events. You can mix both a bit to make the selection of where packets go with a eBPF program, and running higher level stacks in user space where user space is still dealing with raw packets rather than sockets.
  - touisteur 4y ago
    And now you're in AF_XDP territory, which is an alternative to dpdk, a tentative to normalise kernel 'bypass' for network operations.
    - baruch 4y ago
      DPDK has an XDP driver too so you can use that and also work with network cards that do not support XDP or the zero copy variant for more flexibility. Though DPDK can be a pain in and of itself, it is fairly opinionated and I don't like all of its opinions. It has its roots in switching and routing applications and using it for an endpoint can have some mismatches in opinion. It is open source so you can modify it which works out for me in the end.
    - gjulianm 4y ago
      I wouldn't say XDP is an alternative to DPDK. XDP severely limits what you can do as you insert code inside of the kernel. DPDK exposes the NIC to regular user code so you can do what you want. XDP is useful when you want to modify what the kernel does with packets before they go to their destination application, DPDK when you want to do whatever with the packets without sending them to any other application.
      - Matthias247 4y ago

jaimex2 4y ago

Basically its what people who want to bypass the kernel network stack because they think its slow. They then spend the next few years writing their own stack till they realise they've just re-written what the kernel does and its slower and full of exploits. Yeah, receiving packets is fast when you aren't doing anything with them.

AlphaSite 4y ago
Apps probably don’t but networking related code might benefit a lot from DPDK.
dclusin 4y ago
It’s typically done when end to end latency is more important than full protocol compliance and hardening against myriad types of attackers. High frequency trading is typically where the applications where you see these sorts of implementations being really compelling. For these types of applications there are proprietary implementations that you can buy from vendors that are more suited to latency sensitive applications. The next level of optimization after kernel bypass is to build or buy FPGAs which implement the wire protocol + transport as an integrated circuit.
- grive 4y ago
  FPGAs are falling out of favor, ASICs are sufficient to implement most network processing. Funnily, one of the biggest DPDK feature is an API to program smartNICs exactly in that way.
  - dclusin 4y ago
    Fair enough. My Knowledge of what’s on the bleeding edge and mass adoption in algo trading is fairly dated. I haven’t been adjacent to that industry in over 5 years. Even back then there was turnkey ASICS you could buy that would implement the network plus whatever protocols you used to talk to the exchange. I think the neatest thing that sticks out to me is how logging in implementations of low latency trading applications was essentially pushed to the network. Basically they just have copper or fiber taps between each node in their system and it gets sent to another box that aggregates the traffic and processes all the packets to provide trace level logging through their entire system. Even these have solutions you can buy from a number of vendors.
ardel95 4y ago
DPDK is common within network appliances, where you don't need most of the features within kernel networking stack.
shaklee3 4y ago
this is completely wrong. as the article points out, there are plenty of tcp stacks and other user space stacks available. dpdk is used in places where performance matters, and just because you haven't worked in that area doesn't mean it's useless. all telcos use it for the most part.
wumpus 4y ago
> They then spend the next few years writing their own stack Supercomputing people have done this repeatedly and successfully. A more general purpose stack is UDP-based applications. I wouldn't do this with TCP, which I agree is complicated and difficult to get right.
touisteur 4y ago
Sometimes you really don't do much more on packets than receive and store in (e.g) ring buffers until you can trigger some computation on aggregated data. Sometimes your application has critical latency and you need very light parsing. Not everyone uses tcp and all the options, some people are just down to jumbo udp packets, no fragmentation, and 800Gb/s rx...
gjulianm 4y ago
That's not the reality. The kernel network stack does a lot of things that for some applications are not needed, so DPDK is used for those cases when they need very high performance. For example, packet capture, routing, packet generation... I don't think I've seen anyone rewriting what the kernel does precisely because when you use DPDK you don't want to do what the kernel does.
grive 4y ago
Networking stacks are used in two ways: endpoints and forwarders. Forwarders are usually not doing much with packets, just reading a few fields and choosing an output port. These are very common function in network cores (telcos, datacenters, ...). DPDK is not well-suited for endpoint application programming, even though here you can still squeeze some additional performance. But don't dismiss a framework that is currently so widely deployed just because you are not familiar with its most common use-case.
baruch 4y ago
Normally the endpoint stack you need is more limited initially and you don't have to have all the baggage. The biggest advantage you get at the end is zero copy network and if what you do involves tons of reading and writing to the network you can definitely gain performance in both bandwidth and latency by better using your CPU. You can do XDP as well and gain zero copy but for a small subset of NICs and you still need to implement you own network stack I don't see ac way to avoid it much. There are existing TCP stacks for DPDK that you can use as well.

harpratap 4y ago

How does this differ from eBPF?

monocasa 4y ago
eBPF pushes computation into the kernel to be colocated with kernel data/events. DPDK pushes all of the relevant data (the network packets) into user space to be colocated with user data/events. You can mix both a bit to make the selection of where packets go with a eBPF program, and running higher level stacks in user space where user space is still dealing with raw packets rather than sockets.
- touisteur 4y ago
  And now you're in AF_XDP territory, which is an alternative to dpdk, a tentative to normalise kernel 'bypass' for network operations.
  - baruch 4y ago
    DPDK has an XDP driver too so you can use that and also work with network cards that do not support XDP or the zero copy variant for more flexibility. Though DPDK can be a pain in and of itself, it is fairly opinionated and I don't like all of its opinions. It has its roots in switching and routing applications and using it for an endpoint can have some mismatches in opinion. It is open source so you can modify it which works out for me in the end.
  - gjulianm 4y ago
    I wouldn't say XDP is an alternative to DPDK. XDP severely limits what you can do as you insert code inside of the kernel. DPDK exposes the NIC to regular user code so you can do what you want. XDP is useful when you want to modify what the kernel does with packets before they go to their destination application, DPDK when you want to do whatever with the packets without sending them to any other application.
    - Matthias247 4y ago