3 ms·
Actually yes should use vmsplice(). And pv on the other side of the pipe should use splice(). Now that would be a complete zero-copy I/O path, purely limited
by mrb 9y ago
Actually yes should use vmsplice().
And pv on the other side of the pipe should use splice().
Now that would be a complete zero-copy I/O path, purely limited by the CPU, not by memory bandwidth. It would benchmark at hundreds of GB/s :)
- qb45 9y ago> And pv on the other side of the pipe should use splice(). Good catch, I just noticed that it does.
- gpderetta 9y agoSomeone posted an implementation which uses vmsplice on the reddit thread: https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes_so_fast/diu73y0/ https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...
- mrb 9y agoHis code can further be optimized by having only 1 large element in the iovec (vs. many 2-byte "y\n" elements).
- nyanscat 9y agofrom what i can see it's already in a large iovec buffer
- mrb 9y agoCorrect, I misread the code. However bumping the pipe size buffer bumps the speed from 28 to 74 GB/s on my Skylake CPU (i5-6500). Edit: I optimized further and now get 123 GB/s See https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes_so_fast/diua761/ https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...
- gpderetta 9y agonext step is playing tricks with memory mapping so that the buffer is large in virtual memory but only takes a single page in physical memory, fitting in L1 (at least on the read side).