5 ms·

TL;DR: Maximum pipe speed, assuming both programs are written as optimally as possible, is approximately the speed of what one core in your system can read/writ

TL;DR: Maximum pipe speed, assuming both programs are written as optimally as possible, is approximately the speed of what one core in your system can read/write; this is because, essentially, the kernel maps the same physical memory page from one program's stdout to the other's stdin, thus making the operation a zerocopy (or a fast onecopy in slightly less optimal situations). I've known this one for awhile, and it makes writing shell scripts that glue two (or more) things together with pipes to do extremely high performance operations both rewarding and hilarious. Certainly one of the most useful tools in the toolbox.

NortySpock 3y ago
I assume for heterogenous cores (power vs efficiency cores) it bottlenecks on the throughput of the slowest core?
- DiabloD3 3y ago
  Surprisingly no. I'd expect similar performance. In these designs, the actual memory controller that talks to the RAM is part of an internal fabric, and the fabric link between the core and the memory controller is (technically) your upper limit. For both Intel and AMD, the size of the fabric link remains constant to the expected performance of the different cores, as the theoretical usage/performance of the load/store units remain otherwise constant in relation, no matter if it is a big core or a little core. Also, notice: the maximum performance of load-store units is your actual upper limit, period. Some CPUs historically never achieved their maximum theoretical performance because the units were never engaged optimally; sometimes this is because some ports on the load/store units are only accessible from certain instructions (often due to being reserved only for SIMD; this is why memcpy impls often use SSE/AVX, just to exploit this fact). That said, load-store performance usually approaches that core's L2 theoretical maximum, which is greater than what any core generally can get out of its fabric link. Ergo, fabric link is often governing what you're seeing in situations like this. On Intel and AMD's clusters, the memory controller serving their respective core cluster designs requires anywhere from 2 to 4 cores saturating their links to reach peak performance. Also, sibling threads on the same core will compete for access to that link, so it isn't merely threads that get you there, but actual core saturation. On a dummy benchmark like proposed in the linked article, the performance of a single process being piped to another process, either in the situation of "both processes are actually on the same big core, simultaneously hyper-threading", or "two sibling little cores in the same core cluster, being serviced by the same memory controller", the upper limit of performance should approximate optimal usage of memory bandwidth, but in some cases on some architectures this will actually approximate L3 bandwidth (a higher value). Also, as a side note: little cores aren't little. For a little bit more silicon usage, and a little bit less power usage, two little cores approximate one big core /w two threads optimally executing, even in Intel's surprisingly optimal small core design, but very much true in Zen4c. As in, I could buy a "whoops, all little cores" CPU of sufficient size for my desktop, and still be happy (or, possibly, even happier).
bee_rider 3y ago
This is magic system stuff I don’t understand, does it have to go all the way up to the memory or will the caches save us from that trip?
- DiabloD3 3y ago
  Depends entirely on the CPU architecture. The most simple answer I can give is: yes, when its safe; when its not safe, that's part of an entire category of meltdown/spectre family exploits.
jstimpfle 3y ago
AFAIK a severe limitation of pipes is that they can buffer only 64 KB / 16 pages (on x86 Linux). Pretty sure it's generally slower than core-to-memory bandwidth.
- packetlost 3y ago
  64KB is the default, you can increase the buffer size using `fcntl`. You're probably more limited by syscall overhead than anything
packetlost 3y ago
This is why threads aren't nearly as important as many programmers seem to think. Chances are, whatever application you're building can be done in a cleaner way using pipes + processes or green/user-space threads depending on the workload in question. It can be less convenient, but message passing is usually preferable to deadlock hell.
- gpderetta 3y ago
  Message pass enough and you'll easily deadlock as well.
- jstimpfle 3y ago
  Pipes are FIFO data buffers implemented in the kernel. For communication between threads of the same process, you can replace any pipe object by a userspace queue implementation protected by e.g. mutex + condition variable. It is functionally equivalent and has potential to be faster. And if you wrap all accesses in lock/unlock pairs (without locking any other objects in between) there is no danger of introducing any more deadlocks compared to using kernel pipes. Threads are an important structuring mechanism: You can assume that all your threads continue to run, or in the event of a crash, all your threads die. Also, unidirectional pipes aren't exactly sufficient for inter-process / inter-thread synchronisation. They are ok for simple batch processing, but that's about it.
  - gpderetta 3y ago
    Incidentally you can use the exact same setup (plush mmap) for interprocess queues. The advantage of threads is that you can pass pointers to your data through the queue, while that's harder to do between processes and you have to resort to copying data in the queue instead.
    - another2another 3y ago
      >while that's harder to do between processes and you have to resort to copying data in the queue instead. I could be wrong - I've never done it, but I understood that you can even store POSIX mutexes and condition vars in shared mem so that 2 processes (or more?) can process data without copying, so long as they use the both use the same locks stored in the shared memory.
gpderetta 3y ago
Pipes are zero copy only if you use splice or vmsplice. These linux specific syscalls are hard to use (particularly wmsplice) and the vast majority of programs and shell filters (with the notable exception of pv) don't use them and pay for the cost of copying in and out of kernel memory.
- dilyevsky 3y ago
  If you’re using Go it will automatically splice your reader/writer when using io.Copy, etc
  - tucnak 3y ago
    re: https://go.dev/src/net/splice_linux.go https://go.dev/src/net/splice_linux.go very interesting, I didn't know `io` was doing that on linux!
    - deleted 3y ago
      [deleted]
deleted 3y ago
[deleted]
thelastparadise 3y ago
> it makes writing shell scripts that glue two (or more) things together with pipes to do extremely high performance operations both rewarding and hilarious Hilarious because people/teams spend weeks and gobs of money to achieve an inferior result?

NortySpock 3y ago

I assume for heterogenous cores (power vs efficiency cores) it bottlenecks on the throughput of the slowest core?

DiabloD3 3y ago
Surprisingly no. I'd expect similar performance. In these designs, the actual memory controller that talks to the RAM is part of an internal fabric, and the fabric link between the core and the memory controller is (technically) your upper limit. For both Intel and AMD, the size of the fabric link remains constant to the expected performance of the different cores, as the theoretical usage/performance of the load/store units remain otherwise constant in relation, no matter if it is a big core or a little core. Also, notice: the maximum performance of load-store units is your actual upper limit, period. Some CPUs historically never achieved their maximum theoretical performance because the units were never engaged optimally; sometimes this is because some ports on the load/store units are only accessible from certain instructions (often due to being reserved only for SIMD; this is why memcpy impls often use SSE/AVX, just to exploit this fact). That said, load-store performance usually approaches that core's L2 theoretical maximum, which is greater than what any core generally can get out of its fabric link. Ergo, fabric link is often governing what you're seeing in situations like this. On Intel and AMD's clusters, the memory controller serving their respective core cluster designs requires anywhere from 2 to 4 cores saturating their links to reach peak performance. Also, sibling threads on the same core will compete for access to that link, so it isn't merely threads that get you there, but actual core saturation. On a dummy benchmark like proposed in the linked article, the performance of a single process being piped to another process, either in the situation of "both processes are actually on the same big core, simultaneously hyper-threading", or "two sibling little cores in the same core cluster, being serviced by the same memory controller", the upper limit of performance should approximate optimal usage of memory bandwidth, but in some cases on some architectures this will actually approximate L3 bandwidth (a higher value). Also, as a side note: little cores aren't little. For a little bit more silicon usage, and a little bit less power usage, two little cores approximate one big core /w two threads optimally executing, even in Intel's surprisingly optimal small core design, but very much true in Zen4c. As in, I could buy a "whoops, all little cores" CPU of sufficient size for my desktop, and still be happy (or, possibly, even happier).

bee_rider 3y ago

This is magic system stuff I don’t understand, does it have to go all the way up to the memory or will the caches save us from that trip?

DiabloD3 3y ago
Depends entirely on the CPU architecture. The most simple answer I can give is: yes, when its safe; when its not safe, that's part of an entire category of meltdown/spectre family exploits.

jstimpfle 3y ago

AFAIK a severe limitation of pipes is that they can buffer only 64 KB / 16 pages (on x86 Linux). Pretty sure it's generally slower than core-to-memory bandwidth.

packetlost 3y ago
64KB is the default, you can increase the buffer size using `fcntl`. You're probably more limited by syscall overhead than anything

packetlost 3y ago

This is why threads aren't nearly as important as many programmers seem to think. Chances are, whatever application you're building can be done in a cleaner way using pipes + processes or green/user-space threads depending on the workload in question. It can be less convenient, but message passing is usually preferable to deadlock hell.

gpderetta 3y ago
Message pass enough and you'll easily deadlock as well.
jstimpfle 3y ago
Pipes are FIFO data buffers implemented in the kernel. For communication between threads of the same process, you can replace any pipe object by a userspace queue implementation protected by e.g. mutex + condition variable. It is functionally equivalent and has potential to be faster. And if you wrap all accesses in lock/unlock pairs (without locking any other objects in between) there is no danger of introducing any more deadlocks compared to using kernel pipes. Threads are an important structuring mechanism: You can assume that all your threads continue to run, or in the event of a crash, all your threads die. Also, unidirectional pipes aren't exactly sufficient for inter-process / inter-thread synchronisation. They are ok for simple batch processing, but that's about it.
- gpderetta 3y ago
  Incidentally you can use the exact same setup (plush mmap) for interprocess queues. The advantage of threads is that you can pass pointers to your data through the queue, while that's harder to do between processes and you have to resort to copying data in the queue instead.
  - another2another 3y ago
    >while that's harder to do between processes and you have to resort to copying data in the queue instead. I could be wrong - I've never done it, but I understood that you can even store POSIX mutexes and condition vars in shared mem so that 2 processes (or more?) can process data without copying, so long as they use the both use the same locks stored in the shared memory.

gpderetta 3y ago

Pipes are zero copy only if you use splice or vmsplice. These linux specific syscalls are hard to use (particularly wmsplice) and the vast majority of programs and shell filters (with the notable exception of pv) don't use them and pay for the cost of copying in and out of kernel memory.

dilyevsky 3y ago
If you’re using Go it will automatically splice your reader/writer when using io.Copy, etc
- tucnak 3y ago
  re: https://go.dev/src/net/splice_linux.go https://go.dev/src/net/splice_linux.go very interesting, I didn't know `io` was doing that on linux!
  - deleted 3y ago
    [deleted]

deleted 3y ago

[deleted]

thelastparadise 3y ago

> it makes writing shell scripts that glue two (or more) things together with pipes to do extremely high performance operations both rewarding and hilarious Hilarious because people/teams spend weeks and gobs of money to achieve an inferior result?