4 ms·

I recently heard about VPP for the first time; VPP is one of those user-space networking frameworks. Apparently it's the fastest thing currently around by a lar

I recently heard about VPP for the first time; VPP is one of those user-space networking frameworks. Apparently it's the fastest thing currently around by a large margin ("10 mpps per core? Too slow for me!"); it's alluded that Cisco's ASR9000 routers use the same core tech to do all routing in software.

signa11 8y ago
> VPP is one of those user-space networking frameworks. oh yes, vpp is builds on what is provided by dpdk.
rayiner 8y ago
VPP is pretty neat. I just set it up as a router/bridge on my home network. (Upgraded to 10G and Linux can’t route or bridge 10G in software with small packets.) OpenSUSE packages everything and the instructions are great. Very easy to set up. There is really no other option for line rate 10G packet handling in software. Linux can handle maybe 1.5 Mpps per core. That’s barely line rate for 1G. Multiple cores speed that up, but for sequencing purposes packets are usually distributed to different cores by hashing the IP 5-tuple, which means that more cores don’t help for a single connection (or a DDOS like the one in the article).
- ra1n85 8y ago
  >Linux can handle maybe 1.5 Mpps per core. That’s barely line rate for 1G. At what packet size? At 1500B, 10Gbps is only 800Kpps. In my experience working a large network, 300-600B seems to be the average. Worst case, that's ~420Kpps.
  - deleted 8y ago
    [deleted]
  - 0xcde4c3db 8y ago
    As far as I've heard the term used in a packet-switched networking context, "line rate" implies the smallest valid packet size.
- lstamour 8y ago
  Neat. Based on a brief Google search, it appears to use DPDK if available. I’m going to switch my home router to use it, it seems to support PPPoE: https://docs.fd.io/vpp/18.07/clicmd_src_plugins_pppoe.html https://docs.fd.io/vpp/18.07/clicmd_src_plugins_pppoe.html And works with FRR: https://github.com/FRRouting/frr/wiki/Alternate-forwarding-planes:-VPP https://github.com/FRRouting/frr/wiki/Alternate-forwarding-p... More docs on VPP can be found here, including installation, getting started, configuration and FAQ: https://wiki.fd.io/view/VPP https://wiki.fd.io/view/VPP There’s even a Kubernetes networking plugin for VPP: https://github.com/contiv/vpp https://github.com/contiv/vpp
- dgemm 8y ago
  > There is really no other option for line rate 10G packet handling in software. DPDK is an example of another option.
  - emmericp 8y ago
    VPP uses DPDK
baybal2 8y ago
Important note, DPDK and VPP are not faster by the virtue of being based in userspace. It is best to say that they just provide fastest way in between interrupt handler, and the piece of userspace code. The application will then have to reimplement most of the network stack functions that run in kernel in itself. The speedup is there because the application author can freely slim down the network stack to minimal. Secondly to that DPDK took some deliberation on better driver buffer management, ensuring zero copy path, interrupt and syscall efficiency.
- monocasa 8y ago
  AFAIK, you don't even run interrupts off the NIC if you're doing DPDK; the overhead is too high. You poll instead, and just dedicate a core.
  - baybal2 8y ago
    I'm unsure. What I'm imagining is that NICs for which DPDK is made have option to do interrupts like "hey, we have just run out of 50% of buffer! Driver, go offload packets to RAM in bulk now"
    - jsolson 8y ago
      DPDK's drivers are (as far as I know?) all poll-mode: https://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html https://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html
      - shaklee3 8y ago
        Correct
- blattimwind 8y ago
  > It is best to say that they just provide fastest way in between interrupt handler I thought DPDK and friends don't use interrupts and instead map the buffer areas (for ring buffers etc.) used by the NIC into their process address space, directly polling those structures.

signa11 8y ago

> VPP is one of those user-space networking frameworks. oh yes, vpp is builds on what is provided by dpdk.

rayiner 8y ago

VPP is pretty neat. I just set it up as a router/bridge on my home network. (Upgraded to 10G and Linux can’t route or bridge 10G in software with small packets.) OpenSUSE packages everything and the instructions are great. Very easy to set up. There is really no other option for line rate 10G packet handling in software. Linux can handle maybe 1.5 Mpps per core. That’s barely line rate for 1G. Multiple cores speed that up, but for sequencing purposes packets are usually distributed to different cores by hashing the IP 5-tuple, which means that more cores don’t help for a single connection (or a DDOS like the one in the article).

ra1n85 8y ago
>Linux can handle maybe 1.5 Mpps per core. That’s barely line rate for 1G. At what packet size? At 1500B, 10Gbps is only 800Kpps. In my experience working a large network, 300-600B seems to be the average. Worst case, that's ~420Kpps.
- deleted 8y ago
  [deleted]
- 0xcde4c3db 8y ago
  As far as I've heard the term used in a packet-switched networking context, "line rate" implies the smallest valid packet size.
lstamour 8y ago
Neat. Based on a brief Google search, it appears to use DPDK if available. I’m going to switch my home router to use it, it seems to support PPPoE: https://docs.fd.io/vpp/18.07/clicmd_src_plugins_pppoe.html https://docs.fd.io/vpp/18.07/clicmd_src_plugins_pppoe.html And works with FRR: https://github.com/FRRouting/frr/wiki/Alternate-forwarding-planes:-VPP https://github.com/FRRouting/frr/wiki/Alternate-forwarding-p... More docs on VPP can be found here, including installation, getting started, configuration and FAQ: https://wiki.fd.io/view/VPP https://wiki.fd.io/view/VPP There’s even a Kubernetes networking plugin for VPP: https://github.com/contiv/vpp https://github.com/contiv/vpp
dgemm 8y ago
> There is really no other option for line rate 10G packet handling in software. DPDK is an example of another option.
- emmericp 8y ago
  VPP uses DPDK

baybal2 8y ago

Important note, DPDK and VPP are not faster by the virtue of being based in userspace. It is best to say that they just provide fastest way in between interrupt handler, and the piece of userspace code. The application will then have to reimplement most of the network stack functions that run in kernel in itself. The speedup is there because the application author can freely slim down the network stack to minimal. Secondly to that DPDK took some deliberation on better driver buffer management, ensuring zero copy path, interrupt and syscall efficiency.

monocasa 8y ago
AFAIK, you don't even run interrupts off the NIC if you're doing DPDK; the overhead is too high. You poll instead, and just dedicate a core.
- baybal2 8y ago
  I'm unsure. What I'm imagining is that NICs for which DPDK is made have option to do interrupts like "hey, we have just run out of 50% of buffer! Driver, go offload packets to RAM in bulk now"
  - jsolson 8y ago
    DPDK's drivers are (as far as I know?) all poll-mode: https://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html https://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html
    - shaklee3 8y ago
      Correct
blattimwind 8y ago
> It is best to say that they just provide fastest way in between interrupt handler I thought DPDK and friends don't use interrupts and instead map the buffer areas (for ring buffers etc.) used by the NIC into their process address space, directly polling those structures.