4 ms·
Yes / no. Sendfile is for files. Sometimes you want to send normal anonymous memory.
by drewg123 8y ago
Yes / no. Sendfile is for files. Sometimes you want to send normal anonymous memory.
- rjsw 8y agoI should have looked at the source before posting, NetBSD can loan pages to the kernel within sosend().
- cryptonector 8y agoWhy do CoW in sosend(2) -- just have such a system call that requires the caller not to touch the memory being loaned until the writes are completed. CoW for memory is way too expensive. This reminds me that CoW filesystems (think ZFS) and writing through mmap() don't play well. You end up having to use msync(2), which many apps assume there is no need for, and msync(2) is often terribly slow (ISRT at least one system ended up doing page-at-a-time sync writes!). Now, what happens if you have a sosend(2) that requires the caller leave the memory alone, and the caller touches it anyways? Undefined behavior. Possible outcomes include: some CRC/hash/MAC will fail to verify, the mod will have come too late and not been included, the mod will have come soon enough to be included.
- tedunangst 8y agoAlso need some way to indicate the send is complete.
- cryptonector 8y agoEh, as usual. If the socket is non-blocking, then you'll find out how much did not fit in the send buffers and you can poll for when the socket becomes writable. EDIT: Oops, no, the memory would not be copied to the send buffers, and since this is for IPC, we don't even need to account for buffering in this path. Also, for IPC, just sharing with the receiver doesn't work: you can't tell when the received will be done with the memory. You'd need the receiver to munmap() the memory when done, else you'll never know when it's done. Though presumably even sosend(2) as-is requires a munmap() on the receive side... but the docs I can find don't mention it, e.g., https://www.freebsd.org/cgi/man.cgi?query=sosend&sektion=9&manpath=freebsd-release-ports https://www.freebsd.org/cgi/man.cgi?query=sosend&sektion=9&m...
- tedunangst 8y agoWhat happens in the case of TCP when the remote end hasn't acked the data yet?
- cryptonector 8y agoI'm assuming this is an IPC case. If not then sosend(9) (ah yes, it's not a system call) would have to devolve into a proper sendmsg(). The memory might still be held onto by the kernel until sent and ack'ed. The mechanism for completion notification (when the data is ack'ed in the non-IPC case, or when the data is unmapped in the IPC case) would be kqueue- or epoll- or whatever-based. In any case, I'm skeptical of CoW for this. The problem is that on today's CPUs any manipulation of memory mappings is just expensive. I'm even more skeptical of loaning w/o CoW -- its semantics rub me very much the wrong way (see ZFS experience with writing through mmaps). So I'm inclined to suspect this path is just not worthwhile for sockets. (sendfile(2) is different because there is no loaning there as the blocks to [read, if not already read, and then] send are in the same address space already.) I'll be very happy to be very wrong about this because indeed, it feels wrong that copying should be faster than loaning, and CoW feels so right.
- drewg123 8y agoYes, as I said above, I should probably have required a new API.
- cryptonector 8y agoIt could just be a new flag.