3 ms·
I do kernel bypass in production. In my case, I am getting UDP (multicast) packets at a sometimes ferocious rate, that need various kinds of processing done, a
by ncmncm 6y ago
I do kernel bypass in production.
In my case, I am getting UDP (multicast) packets at a sometimes ferocious rate, that need various kinds of processing done, and all captured, with nanosecond timestamps, to disk.
The key is to get the kernel not involved at all. You use NICs from Solarflare (now a Xilinx company, which they used to be a customer of); or Exablaze (now owned by Cisco); Netronome; used to be, Mellanox (now owned by NVidia); or even Napatech (expensive). The NIC driver sets up a ring buffer in DMA memory and just starts dumping packets into it in real time. Each packet gets a little bit of metadata: a nanosecond timestamp, byte count, checksum. The NIC might be filtering by IP address or port, to distribute incoming packets to different ring buffers. The ring buffer is typically a few megabytes, enough for packets to be there for a few ms before they get overwritten.
Your program has to watch this ring buffer for updates, and do whatever it needs to do before the packet gets overwritten. I memcpy them to a big-ass ring buffer, say 8GB of hugepages, and then other unprivileged processes can pick over them for interesting bits, with more leeway for stalls, and can be started and stopped independently.
The process watching the NIC has to be protected against interruptions from the kernel, which involves a mess of kernel boot options -- nohz_full, isolcpus, rcu_nocbs -- because kernels are very jealous of their privilege to stall any process and steal its core for their own purposes. The program needs to do no system calls after startup, and not write to any mmapped memory backed by actual disks (/dev/shm and /dev/hugepages are ok), or the kernel will stall it anyway, boot flags notwithstanding.
Typically each NIC maker has its own kernel-bypass driver and library, often open-source, that understands its ring buffer. Usually they provide a .so your program can LD_PRELOAD to divert regular socket calls into their library, that you will ignore unless you want to, e.g., send out TCP traffic.
NICs have unique features. Intel and others implement a more-or-less portable DPDK interface to their library. Solarflare provides an Onload implementation, and pretty smart hardware filters. ExaNIC has less-smart filters, but delivers packets 120 bytes at a time, so you can start work on a packet before it has all arrived. A Netronome NIC can run eBPF code on a core in the NIC, against packets not even copied to host memory yet. Napatech lets you mess with the filters from the command line, while it's running, and can send packets on a nanosecond-resolution schedule.
Most let you queue up packets and trigger sending certain ones on a dime. You could have a dozen packets with different possible choices, and send only the one you later determine is right.
Lately there is a kernel service, AF_XDP, that is supposed to be a portable, kernel-maintainer approved way to do some of what the proprietary libraries do. I haven't tried it.
Getting reliable nanosecond-resolution timestamps is tricky. Nowadays everything is referenced to atomic clocks on GPS satellites. So, you need a GPS receiver, and a way for the NIC to know what it says. ExaNICs have a receiver on board. Often there is a connector for "PPS" input, expecting a clock rising or falling edge at a known offset from the second boundary. A protocol, PTP, provides ~microsecond resolution, but burns one of your 10Gbps ports. Some switches will process PTP, PPS, or GPS, and tag packets with various non-standard annotations.
If you need to do trickier things, several NICs have FPGAs you can program yourself.
- 10000truths 6y agoI'm curious, what kind of use case do you have that requires nanosecond-precision timestamping for UDP packets? All I can think of are stratum 1 NTP servers, but that doesn't seem to fit the rest of your description.
- Matthias247 6y agoThe use-cases that I'm aware of for precise timestamping on UDP packets are realtime audio/video transmission (ethernet AVB) and static resource allocation on ethernet channels in order to have deterministic delivery delivery guarantees. The latter is interesting for other industrial and automotive bus systems, which want to use ethernet as a replacement for older bus system like CAN. I'm not sure what level of timestamp precision is required for those. The solutions also likely don't require 10Gbit/s.
- knaq 6y agoIt sounds like stock trading. People have been known to process packets in an FPGA to win at that game.
- ncmncm 6y agoTrillions of dollars are traded every day via custom FPGAs. Arguably, it amounts to automated stealing, but is 100% legal. All the NICs have FPGAs on them nowadays, to do regular IP-related stuff, so it is just a matter of reprogramming them.
- ncmncm 6y agoHonestly, the customers don't need nanosecond accuracy, but the NICs offer it, and the nanoseconds in a second fits neatly in 32 bits. In practice, microseconds is better than enough, and all you get from PTP anyway.