4 ms·

Personal experience spanning across my time at Twitter and Facebook, DPDK is definitely faster but it is (/was) a pain in the butt to program and share the NIC

by bdd 7y ago

Personal experience spanning across my time at Twitter and Facebook, DPDK is definitely faster but it is (/was) a pain in the butt to program and share the NIC with the host. XDP allows you to reuse very many packet parsing / handling capabilities in the kernel while in DPDK world you’re shipping a tiny stack with your app. Re: Virtual Switch implemented in BPF: see Cilium’s work to connect containers with their BPF based connectors.

shaklee3 7y ago
That depends on the nic. Mellanox's bifurcated driver is very easy to share with the kernel. Intel's, not so much. Do you mind mentioning which NICs are used at FB? Also for GP, is anyone using vpp in production?
- aloknnikhil 7y ago
  Predominantly, some internal teams at Cisco. VPP and its CNI (Ligato Contiv) seem to be picking up steam off late. This is going by activity in the mailing lists. I know Yahoo Japan uses it. http://events19.linuxfoundation.org/wp-content/uploads/2018/07/ONS.NA_.2019.VPP_LB_public.pdf http://events19.linuxfoundation.org/wp-content/uploads/2018/...
- fdee 7y ago
  There is not even need for a bifurcated driver. The upstream kernel has AF_XDP which Intel and Mellanox NICs support in their drivers, and DPDK has official integration for it as well: https://doc.dpdk.org/guides/nics/af_xdp.html https://doc.dpdk.org/guides/nics/af_xdp.html This will make deploying DPDK significantly easier for those that need/want to use it and allows to share the same driver for pushing packets up to DPDK and into the normal kernel stack with very close to "native" (as in user space driver) DPDK performance (target is ~90-95%).
  - shaklee3 7y ago
    I'm aware of the xdp support, but it's brand new, and certainly nowhere near as mature as the normal mode. It doesn't even work on most NICs yet. But I agree that once it's more mature, it will be better overall.
    - shaklee3 7y ago
      From the latest DPDK docs in your link: "Current implementation only supports single queue, multi-queues feature will be added later. Note that MTU of AF_XDP PMD is limited due to XDP lacks support for fragmentation." That's a non-starter for many use cases, since multiple queues is a fundamental feature of DPDK.
jmeyerxz 7y ago
Sort of a 'virtual switch' but much more beyond that and all with a BPF data plane. Cilium provides connectivity but also security policy, load-balancing and introspection for container pods. Preferred choice in Kubernetes-based workloads.

by bdd 7y ago

shaklee3 7y ago

That depends on the nic. Mellanox's bifurcated driver is very easy to share with the kernel. Intel's, not so much. Do you mind mentioning which NICs are used at FB? Also for GP, is anyone using vpp in production?

aloknnikhil 7y ago
Predominantly, some internal teams at Cisco. VPP and its CNI (Ligato Contiv) seem to be picking up steam off late. This is going by activity in the mailing lists. I know Yahoo Japan uses it. http://events19.linuxfoundation.org/wp-content/uploads/2018/07/ONS.NA_.2019.VPP_LB_public.pdf http://events19.linuxfoundation.org/wp-content/uploads/2018/...
fdee 7y ago
There is not even need for a bifurcated driver. The upstream kernel has AF_XDP which Intel and Mellanox NICs support in their drivers, and DPDK has official integration for it as well: https://doc.dpdk.org/guides/nics/af_xdp.html https://doc.dpdk.org/guides/nics/af_xdp.html This will make deploying DPDK significantly easier for those that need/want to use it and allows to share the same driver for pushing packets up to DPDK and into the normal kernel stack with very close to "native" (as in user space driver) DPDK performance (target is ~90-95%).
- shaklee3 7y ago
  I'm aware of the xdp support, but it's brand new, and certainly nowhere near as mature as the normal mode. It doesn't even work on most NICs yet. But I agree that once it's more mature, it will be better overall.
  - shaklee3 7y ago
    From the latest DPDK docs in your link: "Current implementation only supports single queue, multi-queues feature will be added later. Note that MTU of AF_XDP PMD is limited due to XDP lacks support for fragmentation." That's a non-starter for many use cases, since multiple queues is a fundamental feature of DPDK.

jmeyerxz 7y ago

Sort of a 'virtual switch' but much more beyond that and all with a BPF data plane. Cilium provides connectivity but also security policy, load-balancing and introspection for container pods. Preferred choice in Kubernetes-based workloads.