4 ms·
IPC in Rust – A Ping Pong Comparison
- int08h 2y agoSit in a Compare-and-Swap loop on `ping` and `pong` as uint32s. Go even faster in the shared memory case. Zoom zoom.
- amluto 2y agoAt least try to be polite to the cache coherence system, please. Do a loop where you check (with a plain relaxed (when available) read) whether the compare and swap should work and, if not, do your platform’s pause operation (REP NOP on x86) and try again. Only do compare-and-swap if the optimistic read thinks it will work.
- ComputerGuru 2y agoSee also iceoryx2: https://github.com/eclipse-iceoryx/iceoryx2 https://github.com/eclipse-iceoryx/iceoryx2
- dvt 2y agoIPC having a lot of "overhead" is nonsense (other than, like, managing a mutex or semaphore). I've used many header-only IPC libraries (this one[1] worked great on Windows/MacOS). It's way easier to get a hang of instead of jerry-rigging TCP or UDP to do IPC (yuck). [1] https://github.com/jarikomppa/ipc https://github.com/jarikomppa/ipc
- DawsonBruce 2y agoThanks for sharing this. I recently built something similar for a closed source project and am curious how I didn’t run across this library in my research. Briefly: my implementation uses a templated RAII-style wrapper that persists objects in shared memory for as long as an instance of that object remains in scope. Once the last object goes out of scope, shared memory is freed. It works great so far. I’ll probably reference this library to compare implementations and see what I can learn. Thanks again.
- dvt 2y agoIf you need something more powerful, Boost.Interprocess is also a thing[1]. I used it like 10 years ago and seemed to work well. Looking at the docs, it even has a RAII-ish thing called `boost::interprocess::remove_shared_memory_on_destroy`. [1] https://www.boost.org/doc/libs/1_85_0/doc/html/interprocess.html https://www.boost.org/doc/libs/1_85_0/doc/html/interprocess....
- lukeh 2y agoDomain sockets aren't really Linux-specific, even Windows partially supports them now. [1] [1] https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ https://devblogs.microsoft.com/commandline/af_unix-comes-to-...
- Subsentient 2y agoWrite your own. Wrote one fairly recently in good-ole-sepples that uses UNIX domain sockets and pushes around half a gigabyte a second on weak ARM hardware. Domain sockets are great. Shared memory is even better. If you need a portable solution between processes, not threads, I recommend domain sockets and a simple binary protocol for high throughput.
- parhamn 2y agoA named pipe (fifo) would be an interest to test too. I believe theyre a good bit faster than a unix socket.
- i80and 2y agoMy recollection from Advanced Programming in the Unix Environment is that unix sockets are actually often faster than named pipes: You got me curious; according to a quick search, that seems to hold up according to this more recent post (which doesn't note the OS, annoyingly): https://www.yanxurui.cc/posts/server/2023-11-28-benchmark-tcp-uds-namedpipe/#results https://www.yanxurui.cc/posts/server/2023-11-28-benchmark-tc... I'd be interested in more robust data here.
- novafacing 2y agoIIRC for most packets, UDS are actually faster (not by a massive amount) than named pipes: https://stackoverflow.com/questions/1235958/ipc-performance-named-pipe-vs-socket https://stackoverflow.com/questions/1235958/ipc-performance-... and this advantage increases with packet size (from my own testing).
- parhamn 2y agoTIL! Thanks for the link.
- renewiltord 2y agoHave only seen the other way but shmem with a spinloop should be faster.
- 2y ago
- gamozolabs 2y agoI wrote an extremely fast hot polled pipe in Rust for QEMU instrumentation. I’m sure there’s room to improve but it’s effectively bottlenecking on the uarch. https://github.com/MarginResearch/cannoli/blob/main/mempipe/src/lib.rs https://github.com/MarginResearch/cannoli/blob/main/mempipe/... This was specifically designed for one producer (the QEMU processor), and many consumers (processing the trace of instructions and memory accesses). I can't remember what specific tunings I did to help with this specific model. Data structures like this can always be tuned slightly based on your workload, by changing structure shapes, shared cache lines, etc. With about 3-4 consumers QEMU was not really blocking on the reporting of every single instruction executed, which is really cool. This requires a low noise system, just having a browser open can almost half these numbers since there's just more cache coherency bus traffic occuring. If having a browser open doesn't affect your benchmark, it's probably not bottlenecking on the uarch yet ;) https://github.com/MarginResearch/cannoli/blob/main/.assets/perf_graph.png https://github.com/MarginResearch/cannoli/blob/main/.assets/... A little blog on Cannoli itself here: https://margin.re/2022/05/cannoli-the-fast-qemu-tracer/ https://margin.re/2022/05/cannoli-the-fast-qemu-tracer/ Ultimately, mempipe is not really a big thing I've talked about, but it's actually what makes cannoli so good and enables the design in the first place.
- gamozolabs 2y agoAnyways, since you express disappointment in ~1000 cycle cost. That's about right. The latency between cores is actually quite high and there's not much you can do about it, especially on a system like x86 which has extremely strong cache coherency by default. One thing that is really important to understand is that the ownership of cache lines dramatically affects the cost of memory. For IPC, this effectively requires one thread writing to memory (thus, making that cache line modified to that core, and evicted from all other cores). Then, when the polling thread checks in on the line, it will have to demote that cache line from modified, to shared by flushing it out to memory (usually L2 or L3, but also writing out to memory). This causes some memory traffic, and constantly means that the cores are fighting over the same cache line. Since x86 is strongly ordered and caches are coherent, this traffic is extremely expensive. Think of a write as "tell all other cores that you modifed this memory, so they have to evict/invalidate their cache lines". And a read as "tell all cores to flush their cache lines if they're modified, then wait for them to tell me they're done, then I can read the memory". This effectively is a massively blocking operation. The simple act of reading the mailbox/ticket/whatever from another core to check if a message is ready will actually dramatically affect the speed the other core can write to it (as now that write is effectively full latency). There are some tricks you can do to get extremely low latency between cores. One of them, is making sure you're on cores that are physically near each other (eg. on the same processor socket). This is only really relevant on servers, but it's a big thing. You can actually map out the physical processor layout, including on a single die, based on the latency between these cores. It's quite subtle and requires low noise, but it's really cool to map out the grid of cores on the actual silicon due to timing. Another trick that you can do, is have both threads on the same core, thus, using hyperthreads. Hyperthreads share the same core and thus a lot of resources, and are able to actually skip some of the more expensive coherency traffic, as they share the same L1 cache (since L1 is per-core). The lowest latency you will be able to observe for IPC will be on the same core with hyperthreads, but that's often not really useful for _processing_ the data, since performance will not be great on two busy cores. But in theory, you can signal a hyperthread, the hyperthread can then go and raise some other signal, while the original hyperthread still continues doing some relevant work. As long as one of them is blocking/halted, the other won't really be affected by two things on the same thread. Finally, the most reasonable trick, is making sure your tickets/buffers/mailboxes/whatever are _not_ sharing the same cache lines (unless they contain data which is passed at the same time). Once again, the CPU keeps things in sync at cache line levels. So having two pieces of data being hammered by two cores on the same cache line is asking for hundreds of cycles per trivial data access. This can be observed in an extreme case with many core systems, with multiple sockets, fighting over locks. I've done this on my 96C/192T system and I've been able to get single `lock inc [mem]` instructions to take over 15,000 cycles to complete. Which is unreal for a single instruction. But that's what happens when there's 200-500 cycles of overhead every single time that cache line is "stolen" back from other cores. So, effectively, keep in your head which state cache lines will be in. If they're going to be modified on one core, make sure they're not going to be read on another core while still being written. These transitions are expensive, you're only going to get your 3-4 cycle "random L1 data hit performance" if the cache line is being read, and it's in the exclusive, modified, or shared state, and if it's being written, it has to be exclusive or modified. Anything else and you're probably paying hundreds of cycles for the access, and thus, also probably hurting the other side. Ultimately, what you're asking from the CPU is actually extremely complex. Think about how hard it would be for you to manage keeping a copy of a database in sync between hundreds of writers and reads (cores). The CPU is doing this automatically for you under the hood, on every single memory access. It is _not_ free. Thus, you really have to engineer around this problem, batch your operations, find a design that doesn't require as intense of IPC, etc. On more weakly ordered systems you can use some more tricks in page tables to get a bit more control over how cache coherency should be applied for various chunks of memory to get more explicit control.
- bfrog 2y agoiceoryx2 uses shared memory and is implemented in rust if anyone wants something off the shelf
- Sytten 2y agoIt looks nice but the lack of async is sadly a non starter for a lot of us.
- apatheticonion 2y agoWith this sort of thing you'd typically wrap the underlying IPC channel in a system thread and redirect messages to/from it within the local process via an async channel. The dedicated system thread for the channel means the local process will never block on the incoming IPC events and the redirected messages will be buffered/enqueded as determined by the async runtime. For reference, I wrote a utility that wraps an IPC library and exposes an async and sync API: https://github.com/alshdavid/ipc-channel-adapter https://github.com/alshdavid/ipc-channel-adapter
- mutatio 2y agoIs shared memory access naturally async like IO/io_uring? If not, asking for async is misplaced and any implementation would be synonymous with wrapping sync calls in `async fn` wrappers.
- dwattttt 2y agoI think it's as naturally async as anything else here actually. If you're calling a function via IPC over shared memory, you're sending a message, signalling it's there to the other process, and waiting for a response message with the result. The only thing needed to turn this async is to change the "get result over IPC" function from "block until result is available" to "check if result is available, return TRY_AGAIN if it's not"
- tialaramex 2y agoIn practice it's not desirable to go without an explicit wake feature, we need some way to know that now it's likely to succeed. The mechanism needn't be 100% reliable but "Always try again" is essentially exactly as useless as blocking.
- chme 2y agoThere also is splice: https://www.man7.org/linux/man-pages/man2/splice.2.html https://www.man7.org/linux/man-pages/man2/splice.2.html Which allows to move memory from one process to the next using pipes without copying data. So maybe a bit of a mix between shared memory and pipes. Maybe it would be interesting to see that in the comparison as well...
- duped 2y agoThe use case for splice is moving data from one file descriptor to another without reading it, for example port forwarding/mirroring (or taking over sendfile, dumping a file to a socket or socket to file). It doesn't cover IPC where you have messages that are actually being sent from one process to another and presumably, you want to read and write those bytes.
- Arnavion 2y ago>(or taking over sendfile, dumping a file to a socket or socket to file) splice only works if one of the two fds is a pipe. It cannot replace sendfile.
- duped 2y agoYou can do it with two calls to splice
- gpderetta 2y agoThat was the original implementation. I believe this has since been relaxed. I think the kernel will allocate a pipe internally in some cases. The man pages are known to be incomplete. In fact I think sendfile is implemented with the splice machinery now.
- IAmLiterallyAB 2y agoAlso vmsplice
- pdubouilh 2y agoPretty good write up - I reached the same conclusion doing something similar a few years back. For the record, I got even better perfs with shared-memory by avoiding the mutex in `raw_sync::events::BusyEvent` - I used a blocking loop with atomics, and yielded the CPU (`yield_now` in rust) between the iterations of the loop. Downside is that it becomes a bit less consistent. And it is a _lot_ more complex than a unix domain socket :)
- SloopJon 2y agoAfter looking up the code for raw_sync, I came to the same conclusion. My first question was whether the shared memory case is safe on a weaker memory model like ARM or Power. Seeing the mutex, it may be too safe.
- wpollock 2y agoStill learning rust. What's the unwrap() for in the last line of the consumer code: stdout().write(output).unwrap(); Thanks.
- ynik 2y agoIt causes a panic if the write to stdout fails (e.g. if it's a pipe and the other side was closed, or if stdout is redirected to a file and the disk is full). The rust compiler requires handling this potential error in some way, or you get a compiler error. unwrap() "handles" the IO error by crashing the program with a panic, which is the simplest way of avoiding the compiler error. It's fine for such a toy program, but would be inappropriate for production code.
- deleted 2y ago[deleted]
- commandersaki 2y agoFun fact: this is one of the first exercises you come across in the xv6 book.
- WiSaGaN 2y agoWould be good if it had unix domain socket (USS) tested. Compared to normal tcp/udp socket, UDS does not go through network stack in the kernel, which is useless when sender and receiver are on the same machine.