4 ms·
Last I looked (several months ago), Linux's UDP stack did not seemed well tuned in its memory management accounting. For background, the mental model of what r
by amluto 2y ago
Last I looked (several months ago), Linux's UDP stack did not seemed well tuned in its memory management accounting.
For background, the mental model of what receiving network data looks like in userspace is almost completely backwards compared to how general-purpose kernel network receive actually works. User code thinks it allocates a buffer (per-socket or perhaps a fancier io_uring scheme), then receives packets into that buffer, then processes them.
The kernel is the other way around. The kernel allocates buffers and feeds pointers to those buffers to the NIC. The NIC receives packets and DMAs them into the buffers, then tells the kernel. But the NIC and the kernel have absolutely no concept of which socket those buffers belong to until after they are DMAed into the buffers. So the kernel cannot possibly map received packets to the actual recipient's memory. So instead, after identifying who owns a received packet, the kernel retroactively charges the recipient for the memory. This happens on a per-packet basis, it involves per-socket and cgroup accounting, and there is no support for having a socket "pre-allocate" this memory in advance of receiving a packet. So the accounting is gnarly, involves atomic operations, and seems quite unlikely to win any speed awards. On a very cursory inspection, the TCP code seemed better tuned, and it possibly also won by generally handling more bytes per operation.
Keep in mind that the kernel can't copy data to application memory synchronously -- the application memory might be paged out when a packet shows up. So instead the whole charging dance above happens immediately when a packet is received, and the data is copied later on.
For quite a long time, I've thought it would be nifty if there was a NIC that kept received data in its own RAM and then allowed it to be efficiently DMAed to application memory when the application was ready for it. In essence, a lot of the accounting and memory management logic could move out of the kernel into the NIC. I'm not aware of anyone doing this.
- fragmede 2y agoRDMA is common for high performance applications but it doesn't work over the Internet.
- Danieru 2y agoIt's a good thing the NIC is connected over pcie then.
- shaklee3 2y agoYou can do GPUdirect over the Internet without RDMA though.
- jpgvm 2y agoGPUDirect relies on the PeerDirect extensions for RDMA and are thus an extension to the RDMA verbs, not a separate an independent thing that works without RDMA.
- shaklee3 2y agoAgain, you can do what I said. You may be using different terminology, but you can do GPUdirect in dpdk without rdma
- jpgvm 2y agoYou can read/write to GPU buffers with gpudev in DPDK yes. It also uses some of the infrastructure that powers GPUDirect (namely the page pinning and address translation). Because you can use the addressable memory in DPDK buffer steering you can have the NIC DMA to/from the GPU and then have a GPU kernel coordinate with your DPDK application. This will be pretty fast on a good lossless datacentre network but probably pretty awful over the Internet. In the DC though it will be beaten by real GPUDirect on RDMA naturally as you don't need the DPDK coordinator and all tx/rx can be driven by the GPU kernel instead. This isn't GPUDirect though, that is an actual product.
- shaklee3 2y agoThis is GPUDirect. GPUDirect is the technology that enables any third-party device to talk to a GPU (like a NIC). > but probably pretty awful over the Internet. In the DC though it will be beaten by real GPUDirect on RDMA naturally It's being used in many places successfully over the internet. RDMA is fine, but completely breaks the abstraction of services. In many places you do not want to know who is sending or what address to send/receive.
- throw0101c 2y ago> RDMA is common for high performance applications but it doesn't work over the Internet. RoCEv2 is routable. * https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet * https://docs.nvidia.com/networking/display/winofv55053000/rocev2 https://docs.nvidia.com/networking/display/winofv55053000/ro... Of course you're going to get horrible latency because of speed-of-light limitations, so the definition of "work" may be weak, but data should be able to be transmitted.
- JoshTriplett 2y ago> For quite a long time, I've thought it would be nifty if there was a NIC that kept received data in its own RAM and then allowed it to be efficiently DMAed to application memory when the application was ready for it. I wonder if we could do a more advanced version of receive-packet steering that sufficiently identifies packets as definitely for a given process and DMAs them directly to that process's pre-provided buffers for later notification? In particular, can we offload enough information to a smart NIC that it can identify where something should be DMAed to?
- amluto 2y agoI don’t think the result would be compatible with the socket or io_uring API, but maybe io_uring could be extended a bit. Basically the kernel would opportunistically program a “flow director” or similar rule to send packets to special rx queue, and that queue would point to (pinned) application memory. Getting this to be compatible with iptables/nftables would be a mess or maybe entirely impossible. I’ve never seen the accelerated steering stuff work well in practice, sadly. The code is messy, the diagnostics are basically nonexistent, and it’s not clear to me that many drivers support it well.
- mgaunard 2y agoMost advanced NICs support flow steering, which makes the NIC write to different buffers depending on the target port. In practice though, you only have a limited amount of these buffers, and it causes complications if multiple processes need to consume the same multicast.
- eptcyka 2y agoMulticast may well be shitcanned to an expensive slow path, given that multicast is rarely used for high bandwidth scenarios, especially when multiple processes need to receive the same packet.
- lokar 2y agoThe main real use of multicast I’ve seen is pretty high packet rate. High frequency traders get multicast feeds of tick data from the exchange.
- derefr 2y agoPresuming that this is a server that has One (public) Job, couldn't you: 1. dedicate a NIC to the application; 2. and have the userland app open a packet socket against the NIC, to drink from its firehose through MMIO against the kernel's own NIC DMA buffer; ...all without involving the kernel TCP/IP (or in this case, UDP/IP) stack, and any of the accounting logic squirreled away in there? (You can also throw in a BPF filter here, to drop everything except UDP packets with the expected specified ip:port — but if you're already doing more packet validation at the app level, you may as well just take the whole firehose of packets and validate them for being targeted at the app at the same time that they're validated for their L7 structure.)
- SSLy 2y ago> 1. dedicate a NIC to the application; you need to respond to ICMPs which have different proto/header number than UDP or TCP.
- derefr 2y agoI mean, under the scheme I outlined, the kernel is still going to do that by default. It's not that the NIC's driver is overridden or anything; the kernel would still be reading the receive buffer from this NIC and triggering per-packet handling — and thus triggering default kernel response-handling where applicable (and so responding to e.g. ICMP ARP messages correctly.) The only thing that's different here, is that there are no active TCP or UDP listening sockets bound to the NIC — so when the kernel is scanning the receive buffer to decide what to do with packets, and it sees a TCP or UDP packet, it's going to look at its connection-state table for that protocol+interface, realize it's empty, and drop the packet for lack of consumer, rather than doing any further logic to it. (It'll bump the "dropped packets" counter, I suppose, but that's it.) But, since there is a packet socket open against the NIC, then before it does anything with the packet, it's going to copy every packet it receives into that packet socket's (userspace-shared) receive-buffer mmap region.
- amluto 2y agoI think DPDK does something like this. The NIC is programmed to aim the packets in question at a specific hardware receive queue, and that queue is entirely owned by a userspace program. A lot of high end NICs support moderately complex receive queue selection rules.
- veber-alex 2y agoHave you looked into NVIDIA VMA? https://docs.nvidia.com/networking/display/vmav9860/introduction+to+vma https://docs.nvidia.com/networking/display/vmav9860/introduc...
- rkagerer 2y agoWhy don't we eliminate the initial step of an app reserving a buffer, keep each packet in its own buffer, and once the socket it belongs to is identified hand a pointer and ownership of that buffer back to the app? If buffers can be of fixed (max) size, you could still allow the NIC to fill a bunch of them in one go.