4 ms·
Vector Packet Processing
- Traubenfuchs 3y agoWhat kept this from being available til now? Seems like Cisco had it for ages. > With experimental technologies, Linux has been shown to make some gains in artificial benchmarks, such as dropping all received packets Is this a joke? A jab?
- pyvpx 3y agoVPP has been publicly available since 2017 or before. it’s incredibly fast and feature-full dropping a packet after processing overhead and/or explicit classification work is a useful benchmark yes
- readams 3y agoThis is the same project that was developed at Cisco originally
- nyx 3y agoOh man, on first reading I definitely thought it was saying "with experimental technologies, Linux can drop all traffic", which would have been a hilarious dig... but I think "drop all incoming packets" is a useful benchmark for evaluating overhead, like doing conntrack for every packet that hits then immediately discarding it.
- Traubenfuchs 3y agoYeah that was my interpretation as well. Like the fastest Java Garbage Collector, the one that actually doesn't collect any garbage and lets it pile up til the JVM crashes.
- tw04 3y ago>What kept this from being available til now? Seems like Cisco had it for ages. Cisco is the one who wrote it and open sourced it. Netgate is just putting a wrapper around other people's (Cisco's) code. >Is this a joke? A jab? No, dropping packets is step 1 to proving you can get past some of the current CPU bottlenecks. Actually doing something useful is obviously significantly more work, but no point bothering with that work if the CPU is still the bottleneck.
- gonzo 3y ago> Netgate is just putting a wrapper around other people's (Cisco's) code. Cisco open sourced VPP in 2016 and we've been busy working on it ever since. https://www.stackalytics.io/unaffiliated?module=github.com/fdio/vpp https://www.stackalytics.io/unaffiliated?module=github.com/f...
- wmf 3y agoDropping packets efficiently is really important for handling DDoS. AFAIK that was the original motivation for Cloudflare to adopt XDP.
- nayuki 3y agoReminds me of: "PacketShader - GPU-accelerated Software Router". https://shader.kaist.edu/packetshader/ https://shader.kaist.edu/packetshader/ , http://keonjang.github.io/papers/sigcomm10ps.pdf http://keonjang.github.io/papers/sigcomm10ps.pdf
- pyvpx 3y agoyou trade a lot of latency to make GPU parallelism work for packet processing/classification. some massively clever work around hiding it — but simply no way to avoid it. thus its a niche solution
- touisteur 3y agoWith GPUDirect and active-wait kernels, you can get a tight controlled latency and saturate PCIe bandwidth without touching main memory. StorageDirect if you need to write to (or read from) disk.
- pyvpx 3y agopacket sojurn time is bounded by the latency of the GPU memory architecture. which as I understand has the design dial cranked to ten for parallelism and not so much for expediency
- touisteur 3y agoPeople have been using GPU + DMA for low latency / real-time / high compute intensity applications for some time (using them for adaptive optics of all things). My PhD student's been cranking it to 100/200/400G, with 'just' DPDK, gpudev, and persistent cuda kernels.. Depends on the application, batching policy, compute intensity, etc. But you can put 8 NICs and 8 GPUs in one node (and have them communicate through nvlink, so huge intergpu bandwidth!) which I can't for CPUs. You can maybe also get some unobtainium A100X or CX7+H100 to skimp on PCIe if you're well funded...
- nayuki 3y agoThey disclose their latency in the paper in section 6.4, pages 10 and 11: http://keonjang.github.io/papers/sigcomm10ps.pdf#page=10 http://keonjang.github.io/papers/sigcomm10ps.pdf#page=10 . It's something like 0.4 ms.
- nyx 3y agoVPP is really neat tech. I recently worked on a product that employed it, and it was impressive to see a commodity low-power CPU pushing tens of gigabits of traffic.
- jauntywundrkind 3y agoCalico CNI notably has beta support for VPP, including the userland memif interface if you really really really need speed. https://www.tigera.io/blog/high-throughput-kubernetes-cluster-networking-with-the-calico-vpp-dataplane-and-accelerated-memif/ https://www.tigera.io/blog/high-throughput-kubernetes-cluste... https://docs.tigera.io/calico/latest/getting-started/kubernetes/vpp/getting-started https://docs.tigera.io/calico/latest/getting-started/kuberne... With memif especially, it's fast as heck. But you need to rebuild your apps to target memif. There's some pretty good drop in stdlib replacements for languages like Go, but it's still some work to use the DMA accelerated shared memory packet processing high speed userland mode that VPP is capable of. Ex: https://github.com/KusakabeShi/wireguard-go-vpp https://github.com/KusakabeShi/wireguard-go-vpp
- dragontamer 3y ago100x faster than Linux is certainly fast. IIRC, Linux packet processing is considered rather slow (though full featured, well behaved and configurable). VPP here seems to be a "user-mode network stack", as far as I can tell. I was kind of attracted to the title because I was hoping for SIMD / Vector compute maybe even GPUs, but that doesn't seem to be the case. Still, a usermode network stack is apparently a must-have for any very-high performance network application. I've never needed it, but a lot of optimizers talk about how "slow" Linux networking is when you actually benchmark it.
- crotchfire 3y ago> "user-mode network stack" Kernels typically cannot use vector instructions because if they did they would need to save and restore the vector register state when servicing interrupts. There is a very large performance cost to doing that. Moving packet processing into userspace means adding latency, including TLB pressure, in order to do the context switch. I imagine that we might get some innovation by allowing to configure the system such that the kernel owns the vector registers and userspace is not allowed to use them. If your primary interest in vector registers/instructions is packet processing, and you're doing that in kernelspace, you might not mind it if userspace can't use those registers.
- wmf 3y agoMoving packet processing into userspace means adding latency, including TLB pressure, in order to do the context switch. This isn't the case because VPP polls the NIC from userspace and never enters the kernel. There are no context switches.
- vacuity 3y agoIs there something like an IOMMU to provide secure access to the NIC?
- ADSSDA 3y agoYes, DPDK (which VPP is built on) heavily utilizes IOMMU to provide host protection. https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html discusses it a bit.
- reflexe 3y agoI have been developing a product that uses vpp in production for a few years now. It is very cool to see how much you can squeeze out of cheap low power CPUs. You can easily handle tens of gbits in iMIX with a a few ARM cortex A72s. Vpp has very good documentation: https://s3-docs.fd.io/vpp/24.02/ https://s3-docs.fd.io/vpp/24.02/ A very cool unique feature is the graph representation for packet processing, and the ability to insert processing nodes to the graph dynamically per interface at some point in the processing using features (https://s3-docs.fd.io/vpp/24.02/developer/corearchitecture/featurearcs.html https://s3-docs.fd.io/vpp/24.02/developer/corearchitecture/f...)
- deleted 3y ago[deleted]
- gonzo 3y agoVPP has been shown to run at 22.1 Mpps on a single core of Gracemont (the efficient / Atom core in Alder Lake), and 42.3 Mpps on 2 cores. (Intel E810 4x25 NIC, DPDK 22.0, VPP 22.06, GCC 9.4.0, RFC 2544 test with packet loss <= 0.1%. The same core will do 14.99Gbps of IPsec (aes-128-gcm, 1480 byte packets) using VPP, largely because it supports (VEX-encoded) VAES. While these aren't ARM cortex A72s, they're quite close (cheap low power) for Intel.
- nik736 3y agoWe were running our core routers with BGP and VPP for several years pushing around 40-50 Gbps on a software stack without the need of those expensive ASICs. Worked great and stable. VPP is a great piece of technology.
- AceJohnny2 3y agoIn short: batch processing at the multi-packet level. Increases throughput, at the cost of latency.
- JoachimS 3y agoNoob question, but don't this potentially add per-packet latency and processing variance? My thinking is that one needs to have a set of packets before being able to start processing. The first packets arriving must wait until enough packets has arrived to fill min size of the vector. And if the last packet comes "late", the arrival time of the last packet adds to the time for the other packets, thus adding something that looks like variance. I assume there are parameters setting min number of packets in a vector, and timeouts for when to accept packets into a given vector.
- JoachimS 3y ago(Answering myself now, oh my.) ... And doesn't this also adds, creates a relationship between otherwise independent packets, potentially creating a way to tag packets through a network. Basically if I can control the arrival time of my packets to a router (I send them at a baseline fixed rate, but delay the transmit time with a pattern), packets that are then bunched together to be vector processed in the router will also be affected by this delay. I could possibly then observe this pattern at other places in the network. Thus tracing packets. Possibly.
- gpderetta 3y agoIt is a latency/throughput tradeoff. I haven't really see how VPP works, but I don't expect it actually waits for a vector of packets to be completed. Likely it buffers batch 2 while it is still processing batch 1. As soon as it is done, it starts processing batch 2 and begins buffering batch 3. I wouldn't be surprised if this actually reduces variance.
- teleforce 3y agoThis definitely add per-packet latency and processing, that's why the recommended way to run VPP with high performance is to use some kind of network accelerator library, the same way the VPP based solutions like TNSR by Netgate will work best with DPDK for kernel bypass, or use similar technique like XDP with eBPF but XDP does not bypass the kernel hence eBPF is needed. For Linux user space solutions without kernel bypass if the above mentioned network accelerators are not installed (devices in customer's premise, etc), the recommended way is to use Netmap since it enabled direct access to the network interface card (NIC) buffers from user space otherwise you are at the mercy of Linux own notorious sk_buff [1]. Another alternative perhaps for more efficient buffering in Linux is to use PF_ring or the the-kid-on-block IO_ring but not sure they are being currently being utilized in VPP or not. For good introduction on Linux Networking acceleration technology, this presentation is a good start [2]. [1] VPP docs: Create netmap: https://docs.fd.io/vpp/17.04/clicmd_src_vnet_devices_netmap.html https://docs.fd.io/vpp/17.04/clicmd_src_vnet_devices_netmap.... [2] Linux Networking: The meaning of acronyms eBPF, DPDK, XDP, VPP [video]: https://news.ycombinator.com/item?id=38376380 https://news.ycombinator.com/item?id=38376380