3 ms·

I should add: Even what I wrote above is not the whole story. The deeper you get into kernel internals, the more you realize that the story is nuanced. On Linu

I should add: Even what I wrote above is not the whole story. The deeper you get into kernel internals, the more you realize that the story is nuanced. On Linux, apparently, the kernel does not actually allocate static socket buffers. Instead, the "socket buffer" is actually something like an array of pointers to packets. When the read buffer is empty, it's not taking any space. So on Linux, an idling socket is very cheap (by my understanding, at least). The Windows kernel might do something similar internally (I don't know), but there's no way to avoid the redundant buffer on the userspace side when using IOCP. So basically, it seems that IOCP turns out to be a rather-poor interface for networking in practice, despite at first appearing theoretically superior.

gpderetta 8y ago
I've never used IOCP, but my understanding is that they can sort of emulate readiness notification (as opposed to completion notification) by passing a 0 sized buffer. On the callback then you would perform the (non-blocking) read yourself. In general I think that readiness notification is superior to completion notification for network reads. Completion notification works fine for network writes and it is superior for disk IO (where readiness is ill defined). The biggest advantage of IOCP though is that it basically acts as a dynamically sized threadpool, as it keeps the amount of running threads to a minimum. MacOS does something similar with GCD. I still haven't found a non-hacky way to do the same in Linux (you can sort-of emulate it with either with sched_fifo realtime threads or by using sched_idle to detect idelness).