3 ms·
> Seems quite useless to me. For, what if you want to prefix the file with a header first, or add a trailer? Or what if you want to escape certain parts, or zip
by jsolson 11y ago
> Seems quite useless to me. For, what if you want to prefix the file with a header first, or add a trailer? Or what if you want to escape certain parts, or zip the file before sending? Or if you want to send the file incrementally?
The BSD variant supports headers and trailers, and even if it didn't you're under no obligation to not use other system calls to send data on the socket.
If you're running the kind of workload where sendfile() is valuable you would tend to pre-cache variants that you intended to serve. Consider: why escape the file every time I serve it rather than doing it once?
Even if you did some of these transforms on-the-fly, you can still benefit from sendfile() by caching the transformed version of the file in a tmpfs mount and serving future requests for the same variant from the cached version (with sendfile()).
> If you really want something like sendfile, you could also just find the right library.
Due to the nature of BSD sockets you cannot implement sendfile() in a library. The write() system call (and its vectorized equivalents, along with send(), sendmsg(), etc.) all make a copy of the data to be sent into kernel network buffers[0]. This copy from user mode to kernel mode accounts for the bulk of the computational cost of building a static HTTP server, for example, and can become the bottleneck in terms of maxing out the performance of said server. sendfile() is able to elide this copy by taking advantage of kernel-resident filesystem caches. For any given file the first call to sendfile() will bring the file into memory; assuming a sufficient small corpus (or a sufficiently lucky eviction strategy) the bulk of files served can be served directly from kernel memory.
Also, it's certainly lighter weight than putting the entire HTTP server in the kernel (which, as I understand it, is what Microsoft did at some point for IIS).
[0]: With a sufficiently enlightened memory management system in the kernel one could potentially make these zero-copy for data that was, say, mmap()'d. This would offer the potential for sendfile() like performance from a user-mode library, but would require much more delicate plumbing of memory mapped pages through the networking subsystem.
- amelius 11y agoYou could also say that the aio interface, which has aio_read() and aio_write() calls, is missing an "aio_transfer()" call. Just a quick idea, would mmapped files, combined with the aio_write() interface not give us the same functionality as sendfile()?
- jsolson 11y agoMaybe. It depends on whether or not the kernel is smart enough to recognize that the pages you've referenced in the AIO operation are pages are mmap'd files rather than ordinary heap memory. It would need to do some amount of page table walking to learn that, but certainly for a large enough operation that would be cheaper than simply making the copy. I suspect that Linux, at least, does not do this. That said, I'm curious about the difficulty of plumbing such an optimization. Unfortunately due to other implementation details of the networking stack I work on day to day it wouldn't have any implications for that, but it might have some for one of my side projects.