3 ms·

Also need some way to indicate the send is complete.

cryptonector 8y ago
Eh, as usual. If the socket is non-blocking, then you'll find out how much did not fit in the send buffers and you can poll for when the socket becomes writable. EDIT: Oops, no, the memory would not be copied to the send buffers, and since this is for IPC, we don't even need to account for buffering in this path. Also, for IPC, just sharing with the receiver doesn't work: you can't tell when the received will be done with the memory. You'd need the receiver to munmap() the memory when done, else you'll never know when it's done. Though presumably even sosend(2) as-is requires a munmap() on the receive side... but the docs I can find don't mention it, e.g., https://www.freebsd.org/cgi/man.cgi?query=sosend&sektion=9&manpath=freebsd-release-ports https://www.freebsd.org/cgi/man.cgi?query=sosend&sektion=9&m...
- tedunangst 8y ago
  What happens in the case of TCP when the remote end hasn't acked the data yet?
  - cryptonector 8y ago
    I'm assuming this is an IPC case. If not then sosend(9) (ah yes, it's not a system call) would have to devolve into a proper sendmsg(). The memory might still be held onto by the kernel until sent and ack'ed. The mechanism for completion notification (when the data is ack'ed in the non-IPC case, or when the data is unmapped in the IPC case) would be kqueue- or epoll- or whatever-based. In any case, I'm skeptical of CoW for this. The problem is that on today's CPUs any manipulation of memory mappings is just expensive. I'm even more skeptical of loaning w/o CoW -- its semantics rub me very much the wrong way (see ZFS experience with writing through mmaps). So I'm inclined to suspect this path is just not worthwhile for sockets. (sendfile(2) is different because there is no loaning there as the blocks to [read, if not already read, and then] send are in the same address space already.) I'll be very happy to be very wrong about this because indeed, it feels wrong that copying should be faster than loaning, and CoW feels so right.

Eh, as usual. If the socket is non-blocking, then you'll find out how much did not fit in the send buffers and you can poll for when the socket becomes writable. EDIT: Oops, no, the memory would not be copied to the send buffers, and since this is for IPC, we don't even need to account for buffering in this path. Also, for IPC, just sharing with the receiver doesn't work: you can't tell when the received will be done with the memory. You'd need the receiver to munmap() the memory when done, else you'll never know when it's done. Though presumably even sosend(2) as-is requires a munmap() on the receive side... but the docs I can find don't mention it, e.g., https://www.freebsd.org/cgi/man.cgi?query=sosend&sektion=9&manpath=freebsd-release-ports https://www.freebsd.org/cgi/man.cgi?query=sosend&sektion=9&m...

tedunangst 8y ago
What happens in the case of TCP when the remote end hasn't acked the data yet?
- cryptonector 8y ago
  I'm assuming this is an IPC case. If not then sosend(9) (ah yes, it's not a system call) would have to devolve into a proper sendmsg(). The memory might still be held onto by the kernel until sent and ack'ed. The mechanism for completion notification (when the data is ack'ed in the non-IPC case, or when the data is unmapped in the IPC case) would be kqueue- or epoll- or whatever-based. In any case, I'm skeptical of CoW for this. The problem is that on today's CPUs any manipulation of memory mappings is just expensive. I'm even more skeptical of loaning w/o CoW -- its semantics rub me very much the wrong way (see ZFS experience with writing through mmaps). So I'm inclined to suspect this path is just not worthwhile for sockets. (sendfile(2) is different because there is no loaning there as the blocks to [read, if not already read, and then] send are in the same address space already.) I'll be very happy to be very wrong about this because indeed, it feels wrong that copying should be faster than loaning, and CoW feels so right.