6 ms·
How to drop 10M packets per second
- blattimwind 8y agoI recently heard about VPP for the first time; VPP is one of those user-space networking frameworks. Apparently it's the fastest thing currently around by a large margin ("10 mpps per core? Too slow for me!"); it's alluded that Cisco's ASR9000 routers use the same core tech to do all routing in software.
- signa11 8y ago> VPP is one of those user-space networking frameworks. oh yes, vpp is builds on what is provided by dpdk.
- rayiner 8y agoVPP is pretty neat. I just set it up as a router/bridge on my home network. (Upgraded to 10G and Linux can’t route or bridge 10G in software with small packets.) OpenSUSE packages everything and the instructions are great. Very easy to set up. There is really no other option for line rate 10G packet handling in software. Linux can handle maybe 1.5 Mpps per core. That’s barely line rate for 1G. Multiple cores speed that up, but for sequencing purposes packets are usually distributed to different cores by hashing the IP 5-tuple, which means that more cores don’t help for a single connection (or a DDOS like the one in the article).
- ra1n85 8y ago>Linux can handle maybe 1.5 Mpps per core. That’s barely line rate for 1G. At what packet size? At 1500B, 10Gbps is only 800Kpps. In my experience working a large network, 300-600B seems to be the average. Worst case, that's ~420Kpps.
- deleted 8y ago[deleted]
- 0xcde4c3db 8y agoAs far as I've heard the term used in a packet-switched networking context, "line rate" implies the smallest valid packet size.
- lstamour 8y agoNeat. Based on a brief Google search, it appears to use DPDK if available. I’m going to switch my home router to use it, it seems to support PPPoE: https://docs.fd.io/vpp/18.07/clicmd_src_plugins_pppoe.html https://docs.fd.io/vpp/18.07/clicmd_src_plugins_pppoe.html And works with FRR: https://github.com/FRRouting/frr/wiki/Alternate-forwarding-planes:-VPP https://github.com/FRRouting/frr/wiki/Alternate-forwarding-p... More docs on VPP can be found here, including installation, getting started, configuration and FAQ: https://wiki.fd.io/view/VPP https://wiki.fd.io/view/VPP There’s even a Kubernetes networking plugin for VPP: https://github.com/contiv/vpp https://github.com/contiv/vpp
- dgemm 8y ago> There is really no other option for line rate 10G packet handling in software. DPDK is an example of another option.
- emmericp 8y agoVPP uses DPDK
- baybal2 8y agoImportant note, DPDK and VPP are not faster by the virtue of being based in userspace. It is best to say that they just provide fastest way in between interrupt handler, and the piece of userspace code. The application will then have to reimplement most of the network stack functions that run in kernel in itself. The speedup is there because the application author can freely slim down the network stack to minimal. Secondly to that DPDK took some deliberation on better driver buffer management, ensuring zero copy path, interrupt and syscall efficiency.
- monocasa 8y agoAFAIK, you don't even run interrupts off the NIC if you're doing DPDK; the overhead is too high. You poll instead, and just dedicate a core.
- baybal2 8y agoI'm unsure. What I'm imagining is that NICs for which DPDK is made have option to do interrupts like "hey, we have just run out of 50% of buffer! Driver, go offload packets to RAM in bulk now"
- blattimwind 8y ago> It is best to say that they just provide fastest way in between interrupt handler I thought DPDK and friends don't use interrupts and instead map the buffer areas (for ring buffers etc.) used by the NIC into their process address space, directly polling those structures.
- barbegal 8y ago"With XDP we can drop 10 million packets per second on a single CPU." is not really correct, the packets are being dropped on the network card which has its own processor. The packets never get transferred to the main CPU.
- bhouston 8y agoExactly. And thus this approach does not scale with more cores at all. Let me show how to render 100B polygons per second on a single cpu by using a gpu to do all the work... Argh.
- deleted 8y ago[deleted]
- rayiner 8y agoThat’s not true. While there are NICS that can drop packets in hardware (e.g. Chelsio) that is not what XDP does. XDP runs eBPF programs in a driver-implemented recieve callback. The recieve callback happens at the lowest level of the Linux network stack, when an interrupt is delivered to a processor and a packet descriptor is queued by the NIC, but before the generic packet handling metadata (SKB) is allocated. So XDP operates on the packet in place, but the code runs on the host CPU.
- iajrz 8y agoWhen you say "in place" you mean the data is still on the NIC? Or is the data 'elsewhere in RAM'? I'm curious because this would be the difference between "scale with more cores" or "scale with more NICs".
- penagwin 8y agoIf I read that correctly the data is still on the NIC. The CPU receives an interrupt - notifying it of a new packet, which it then decides to drop? Thus the CPU is the one dropping packets, but the data (most of it at least) is still on the NIC. Please correct me if I'm wrong.
- ck2 8y agonow find a way for me to enter an unknown/untrusted website that is using your "ddos protection" without having to allow inline javascript to execute because as you clearly know/demonstrate, the web can't be trusted by default
- JensRex 8y agoI'm an electrician. I can help you drop packets really quick. Available for hire.