4 ms·
you trade a lot of latency to make GPU parallelism work for packet processing/classification. some massively clever work around hiding it — but simply no way to
by pyvpx 3y ago
you trade a lot of latency to make GPU parallelism work for packet processing/classification. some massively clever work around hiding it — but simply no way to avoid it. thus its a niche solution
- touisteur 3y agoWith GPUDirect and active-wait kernels, you can get a tight controlled latency and saturate PCIe bandwidth without touching main memory. StorageDirect if you need to write to (or read from) disk.
- pyvpx 3y agopacket sojurn time is bounded by the latency of the GPU memory architecture. which as I understand has the design dial cranked to ten for parallelism and not so much for expediency
- touisteur 3y agoPeople have been using GPU + DMA for low latency / real-time / high compute intensity applications for some time (using them for adaptive optics of all things). My PhD student's been cranking it to 100/200/400G, with 'just' DPDK, gpudev, and persistent cuda kernels.. Depends on the application, batching policy, compute intensity, etc. But you can put 8 NICs and 8 GPUs in one node (and have them communicate through nvlink, so huge intergpu bandwidth!) which I can't for CPUs. You can maybe also get some unobtainium A100X or CX7+H100 to skimp on PCIe if you're well funded...
- nayuki 3y agoThey disclose their latency in the paper in section 6.4, pages 10 and 11: http://keonjang.github.io/papers/sigcomm10ps.pdf#page=10 http://keonjang.github.io/papers/sigcomm10ps.pdf#page=10 . It's something like 0.4 ms.