5 ms·
Zero-copy network transmission with io_uring (2021)
- gigel82 5y agoThat sounds interesting for routing applications. I was just looking into building a custom router using an older PC and a dual-port 2.5GbE NIC. OpnSense sounds like a solid choice, but I'd prefer a Linux solution because the CPU is pretty beefy and wouldn't mind running a few Dockers on the side (yes, I know multi-purposing a router is a bad idea but meh).
- wmf 5y agoThe kernel already does zero-copy routing; io_uring is for UDP and TCP.
- kzemek 5y agoFor a given value of "zero"; the packets are still copied from NIC buffers and wrapped in sk_buff.
- wmf 5y agoMaybe headers are parsed but my understanding is that the whole packet is not copied.
- gigel82 5y agoIt's a NAT router so needs at minimum to masquerade (change the src/dest IP in the packet headers to match the internal / public). I'd also be running fail2ban or crowdsec for the bare intrusion detection.
- jeffbee 5y agoA "beefy CPU" will sit twiddling its thumbs forwarding traffic between 2.5g interfaces. Any CPU on the market can do that in between naps.
- MayeulC 5y agoIt depends if GP wants to do deep packet inspection or other processing on the data. Otherwise, yes, you are right.
- pimeys 5y agoEven with opnsense, put a packet filter and wireguard to the box, and I guarantee you'll use your CPU and memory with a 2.5GbE NIC...
- ahepp 5y agoHave you considered vyos as a Linux alternative to *sense? Additionally, you could set the box up as a hypervisor, pass the NIC through to the router VM, and have the other VMs/containers use a bridge or some more advanced virtual networking. That obviously has potential downsides, but I had a bit of fun doing it before downsizing my server
- dilyevsky 5y agoSo the idea is for it to be used like sendfile?
- rossmohax 5y agolooks like many syscalls will eventually be replaced by io_uring acting like a job queue. Not just network, but any interaction with the kernel.
- foota 5y agoSounds like it, yes, but it is infinitely more flexible and iirc the general architecture also allows chaining, e.g., you can in a sense bind a parameter of a system call to the result of another to-be-made system call. An example might be taking the length from one system call to another or a file descriptor number.
- deleted 5y ago[deleted]
- hamburglar 5y agoOr stuff that requires low latency on a weak platform. When I was at Microsoft we did this with Media Center Extender running on linux on ARM where we needed to stream 1080p MPEG and didn’t have the memory bandwidth for two copies. We added send/recv calls that allowed the memory that the NIC was reading into to be the same memory the mpeg decoder was reading from. It allowed us to get better latency by not having to buffer as much video to keep the decoder fed. Neat that there is a general mechanism for this now.
- cmrdporcupine 5y agoI was involved with trying to do a similar kind of thing with a video streaming box I was tasked on working on for Google Fiber some years ago. Never shipped, but we had to repurpose an old MIPS-based box that was being used as a network box to try to spool up an inbound MPEG-TS video stream and then re-serve it as MPEG DASH segments to the local network. I was trying to do it with zero copies and minimal transitions between user and kernel space, so was a lot of mmap'ing and the like. On weak hardware these kinds of optimizations showed a lot of promise. Unfortunately the whole project was killed and all Fiber projects pulled from our office before we could finish.
- m12k 5y agoIt seems like a lot of performance optimizations that used to require custom kernels can now be implemented using io_uring instead. Does anyone know if any such performance improvements are making their way into places where they can be felt by those of us working higher up the tech stack? (E.g. if nginx, node or similar got faster networking or disk access)
- dan-robertson 5y agoI think the performance optimisations people use these days are not kernel patches so much as: - userspace networking which avoids copying in the kernel and expensive context switches (both advantages of io_uring) and some costs of sockets apis[1] - specialised NICs that can e.g. do TLS in hardware - specialised NICs with on-board FPGAs to which some custom processing may be offloaded - specialised NICs with first class virtualisation support to avoid hypervisor overhead (mostly just a cloud thing) One thing about high performance networking is that there are really two different desires: maximising throughput and minimising latency. Different choices will be made based on what kind of performance is most desired. [1] if you imagine a typical high performance, pre-io_uring, portable web server, you have some socket which you listen on, many threads which call accept on it (with an option so that only one would be woken up at a time with new connections) and a load of fds for open connections with the server calling epoll to wait for inputs from them. And when a packet comes in and makes it through the kernel tcp stack, the kernel has to assign the packet to an appropriate fd and figure out if any thread needs to wake up. But instead with specialised networking configurations, you can just receive the packets in the order they arrive and send them off to their handlers yourself without the overhead of explaining how to do that to the kernel.
- tomnipotent 5y ago> are not kernel patches so much as Intel even offers DDIO (Direct Data I/O), which allows an Intel NIC and CPU to share data without going through main memory.
- drewg123 5y ago
- a-dub 5y agowill this compete with the performance of kernel bypass networking? (and potentially obsolete it someday?)
- ekkeke 5y agoDepends, kernel bypass isn't just about avoiding the kernel, a lot of the cards used are specifically desinged for this and come with much lower latency (as low as 1us to write a packet). Eliminating a kernel copy of the buffer being made will improve the performance significantly, but probably won't get you the same performance as a specialised card which would be what you'd usually go for if it's that important.
- majke 5y agoThere are multiple problems "kernel bypasses" solves. Generally speaking: (A) prevent kernel from touching data and only operate on metadata (to reduce cache misses/memory bandwidth) (B) in routing situations prevent application from copying data (C) busy-polling - save cpu on doing everything in application as opposed to jumping to/from kernel context, for better latency (D) allow manual buffer management For example, splice or tcp_mmap() can be used to "receive" data from networking stack into application managed place. splice and MSG_ZEROCOPY can be used to "transmit" data into networking stack. The problem with MSG_ZEROCOPY is that, while buffers belong to application, the API doesn't tell you when it's done sending and it's possible to repurpose the buffers. This io_uring article seem to indicate this fault has been fixed in io_uring zerocopy API. These API's are pretty much (A), prevent kernel from touching data, and only look at metadata. Due to API limitations it's hard or even impossible to do (B) in application. But with better API's we'll get there. To get (C), you need raw hardware access basically. But the holly grail is (D): the buffer-reuse problem. Ideally you'd want to submit buffer to networking card (or network stack), get data from NIC directly into there, then get a handle to this data in application, perform routing decision, submit that buffer for transmission in some other NIC, and eventually get a completion notification. At which point the buffer could be submitted for RX again. The userspace zerocopy API's are far away from this ideal scenario. So no. Zerocopy kernel API's don't immediately solve all problems typical kernel bypass solution aims for. Having staid that, I don't think many users have the problems true kernel bypass solves.
- deleted 5y ago[deleted]
- gfd 5y agoI wanted to see some benchmark numbers but the first hit on google returns a unresolved 150+ message github issue about how io_ring might not be better than epoll: https://github.com/axboe/liburing/issues/189 https://github.com/axboe/liburing/issues/189 I didn't read the entire thread, but are there still implementation kinks that are being worked out or is the api ready to use now?
- jeffbee 5y agoThere are a lot of rough edges in io_uring, foremost among them is it has been written, in classic Linux style, without regard to the security implications, so despite its youth it already collected four local privilege escalation CVEs. I recommend approaching it skeptically, write your own test and make sure it's actually faster in your use case.
- idealmedtech 5y agoNote: since Alex is a fairly gender neutral name I'm going to use they/them pronouns. I went through and read it, and the submitter is incredibly confrontational and not at all open to feedback on the correctness of their benchmark. Also, when presented with contradictory evidence (their own benchmark where the results show io_uring is faster than epoll on other machines), they essentially dismiss it and says the other users ran their benchmark wrong, or that they can't reproduce their results on their machine, or that the other users used Boost and therefore are invalid. So not an entirely reliable criticism coming from them, in my opinion.
- halpert 5y agoThat’s not my take at all. Many people respond with hand waving arguments: maybe epoll is faster because the benchmark is wrong, or only one file descriptor is used (which wasn’t correct but said anyway), etc. Other people also respond saying they’ve been unable to match epoll performance when trying to add io_uring support to their libraries.
- digikata 5y agoI think io_uring more or less works as advertised now for the key thing it set out to do, but it removes one bottleneck in a larger system of network processing, so you can switch to io_uring, yet potentially without any other changes, a real world application can hit the next bottleneck due some adjacent architectural decision (or performance regressions as implementations are changed to accommodate an expanding envelope of uses).
- scrubs 5y agoMy side project has been to rewrite https://github.com/erpc-io/eRPC https://github.com/erpc-io/eRPC which does RPCs over UDP with some congestion control supposedly quite fast. Paper: https://www.usenix.org/system/files/nsdi19-kalia.pdf https://www.usenix.org/system/files/nsdi19-kalia.pdf. I never got the code to work in AWS; I believe the author focused on Mellanox NICS but that's not really commodity H/W, which is where my interests lay. So I dug into it .. and well I'll have my own library soon. I should be able to send UDP w/o congestion control sometime this week. eRPC uses DPDK (100% user space NIC TX/RX control) plus the author's own other ideas to get performance. Since I'm getting into DPDK way, way late in the game, I hope and believe DPDK is and will be the better way to go than to turn to kernel fix-ups. Getting the kernel out of the way with pinned threads seems -- or more exotic with special H/W if preferred --- is cleaner if one can develop from scratch. DPDK is all day long zero copy, and always has been. This library will be a part of something bigger. For many frameworks, a key architecture point is: I got a RPC/packet/message. Ok, now what? * process it in-place e.g. on the thread that was doing RX? * if I don't delegate am I making back-pressure? * delegate it to another core? How to efficiently do that? Hint: https://blog.acolyer.org/2017/12/04/ffwd-delegation-is-much-faster-than-you-think/ https://blog.acolyer.org/2017/12/04/ffwd-delegation-is-much-... * if I delegate ... to whom? Maybe I'm partitioning? * If I delegate how do I get a response back? In DPDK I believe these are easier to decide and well-manage in code.
- goombacloud 5y agoDepending on the requirements of the RPC you could implement the logic in eBPF code over the XDP framework which can directly execute the code inside the NIC driver and modify the packet buffer to be a response packet that gets send out immediately without ever going through the rest of the kernel nor userspace.
- scrubs 5y agoThat option was not on my radar. >execute the code inside the NIC driver this is ... a NIC+FPGA which can execute code?
- 5y ago
- MichaelMoser123 5y ago> there needs to be a mechanism by which the kernel can tell applications that a given buffer can be reused. MSG_ZEROCOPY handles this by returning notifications via the error queue associated with the socket — a bit awkward, but it works. Let's say they have a circular buffer for the TX queue. Why can't they have an atomic variable for the read end of the circular buffer; one could map this variable into shared memory, the kernel could then move this counter, once the nic has completed to send a frame with the data from the read end of that circular buffer. User space could still query this counter to check for 'buffer full' conditions.