4 ms·
Hmm, there will still be a buffer in the kernel, that the NIC DMAs into. You will also need to buffer incoming data from a stream protocol like TCP to combine
by kbwt 10y ago
Hmm, there will still be a buffer in the kernel, that the NIC DMAs into.
You will also need to buffer incoming data from a stream protocol like TCP to combine consecutive reads into complete application layer packets.
A better approach can be seen with Registered I/O on Windows, which lets user-mode programs register buffers with the kernel so that they can be locked for the NIC to DMA into: https://technet.microsoft.com/en-us/library/hh997032(v=ws.11).aspx https://technet.microsoft.com/en-us/library/hh997032(v=ws.11...
- tedunangst 10y agoThose DMA buffers are shared. You don't have a different DMA buffer for each of a million sockets.
- kbwt 10y agoYou're right. But there will realistically still be some protocol-level buffering in user-mode for every socket. Although that might be a good use-case for a hash table of buffers, as most of the sockets will not have outstanding partial packets most of the time in typical (frequent small message) applications.
- twic 10y agoThis is something i've wondered about for years. If you have a million sockets, and you've set SO_SNDBUF to 4 kilobytes for all of them, does that mean the kernel has 4 gigabytes of memory sitting around waiting for TCP segments? In the classic C10k scenario, most sockets don't have a segment in flight at any point in time, so their buffers will be empty. It would be a huge waste if they were actually occupying memory.