5 ms·
Is the sendfile system call still the fastest way to transfer data locally on Linux systems? For communicating between processes running locally on system. Or,
by ArtWomb 8y ago
Is the sendfile system call still the fastest way to transfer data locally on Linux systems?
For communicating between processes running locally on system. Or, for example, a server running locally on an android device. Unix domain sockets typically beat TCP connections for most large file transfers. But is there anything faster?
And what about transfers over the public net from machine to machine? I usually just use netcat, ssh, or scp.
- Pete_D 8y agoConsider trying rsync instead of scp - I've found it to be faster for multiple small files.
- SteveNuts 8y agoShared memory is faster than sockets
- rurban 8y agoBut unsafe.
- the8472 8y agoThat depends on what you're doing. Sendfile, splice and copy_file_range are the fastest if you want to shovel things back and forth between file descriptors. The latter supports reflink copy if the the underlying filesystem does. And I think there's some special casing in vmsplice that lets you transfer memory pages from process to process over a pipe. At least I've seen some patches for that, not sure if they made it into the kernel. Alternatively there's memfd. But if you want to coordinate processes you'll use a shared memory and futexes. If you just want to hand over an entire file to a different process you can send the file descriptor itself over a unix socket.
- ArtWomb 8y ago>>> you can send the file descriptor itself over a unix socket Oh, that is genius ;) Described in detail here, from Stevens Unix Network Programming http://www.masterraghu.com/subjects/np/introduction/unix_network_programming_v1.3/ch15lev1sec7.html http://www.masterraghu.com/subjects/np/introduction/unix_net...