3 ms·

FWIW there is readv() / writev(), splice(), sendfile(), funopen(), and io_buffer() as well. splice() is great when transferring data between pipes and UNIX soc

FWIW there is readv() / writev(), splice(), sendfile(), funopen(), and io_buffer() as well. splice() is great when transferring data between pipes and UNIX sockets with zero-copy, but it is Linux-only. splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal. Sadly on BSDs, for pipes, readv() / writev() is the most performant way to achieve the same if I am not mistaken. Please correct me if I am wrong. At any rate, this is a great article.

messe 1y ago
> sendfile() is file-to-socket (zero-copy as well), and has very high performance as well, for both Linux and BSDs. It only supports file-to-socket, however, and well, to stay relevant, sendmsg() can't be used with pipes in the general case, it is for UNIX domain sockets, INET sockets, and other socket types. On Linux, sendfile supports more than just file to socket, as it's implemented using splice. I've used it for file-to-block-device in the past.
- johnisgood 1y ago
  On BSDs probably not, as they don't have splice, but that is good to know. I wonder if on BSDs it really is readv() and writev() that are the fastest way to achieve the same thing as has been done in the article. Maybe I am missing something. I would like to be corrected.
  - messe 1y ago
    AFAIK, neither OpenBSD nor NetBSD has sendfile. On FreeBSD, I think you're correct regarding it being file-to-socket only.
    - zambal 1y ago
      Indeed, if I'm not mistaken Netflix at least used to use (and commit to kernel) FreeBSD on content servers because of its superior sendfile performance
      - messe 1y ago
        Not only that, but they even contributed patches to allow the FreeBSD kernel to handle the TLS part of SSL_sendfile as well[1]. [1]: https://man.freebsd.org/cgi/man.cgi?ktls(4) https://man.freebsd.org/cgi/man.cgi?ktls(4)
tedunangst 1y ago
Shared memory, like shm_open and fd passing, would be even faster and fully portable.
wavesquid 1y ago
> splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal. Proper use of io_uring should finally have it beat or at least matched.

messe 1y ago

> sendfile() is file-to-socket (zero-copy as well), and has very high performance as well, for both Linux and BSDs. It only supports file-to-socket, however, and well, to stay relevant, sendmsg() can't be used with pipes in the general case, it is for UNIX domain sockets, INET sockets, and other socket types. On Linux, sendfile supports more than just file to socket, as it's implemented using splice. I've used it for file-to-block-device in the past.

johnisgood 1y ago
On BSDs probably not, as they don't have splice, but that is good to know. I wonder if on BSDs it really is readv() and writev() that are the fastest way to achieve the same thing as has been done in the article. Maybe I am missing something. I would like to be corrected.
- messe 1y ago
  AFAIK, neither OpenBSD nor NetBSD has sendfile. On FreeBSD, I think you're correct regarding it being file-to-socket only.
  - zambal 1y ago
    Indeed, if I'm not mistaken Netflix at least used to use (and commit to kernel) FreeBSD on content servers because of its superior sendfile performance
    - messe 1y ago
      Not only that, but they even contributed patches to allow the FreeBSD kernel to handle the TLS part of SSL_sendfile as well[1]. [1]: https://man.freebsd.org/cgi/man.cgi?ktls(4) https://man.freebsd.org/cgi/man.cgi?ktls(4)

tedunangst 1y ago

Shared memory, like shm_open and fd passing, would be even faster and fully portable.

wavesquid 1y ago

> splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal. Proper use of io_uring should finally have it beat or at least matched.