6 ms·

A shame this has to be done within the kernel. There was never much reason to implement TCP handling in ring zero.

etaioinshrdlu 7y ago
Yes, I have not heard any good arguments for why TCP should be in a kernel, other than, it's convenient, and it's always been done that way, and your apps get to share it. Like using the kernel as a shared library... You could put BitTorrent in the kernel. It makes about as much sense architecturally, but isn't too widely used.
- phicoh 7y ago
  POSIX supports sharing filedescriptors between processes. For example, you can have a process that reads a few bytes from a TCP socket and then passes the socket to another process. Unix tries to model all I/O, including networking, as operations on files. Realistically it is only possible to get this right if the kernel is involved. Of course it is quite possible to come up with different models. But Unix seems to be uniquely powerful in its ability to create complex systems from lots of small processes. The kernel being involved does not imply that all code has to be in the kernel. The FUSE file system interface is a well known way to run filesystem code as user processes. Likewise, there are ways to run device drivers in user space. The disavantage is that the extra context switches cause performance loss. So this approach is use for protocols that are rarely used and do not warrent full kernel implementation.
- cat199 7y ago
  taken to the logical extreme, why do anything in the kernel for that matter?
  - etaioinshrdlu 7y ago
    I think a good rule of thumb should be that a kernel should be responsible making hardware devices safe to use among multiple processes, and little else.
mehrdadn 7y ago
I've never seen anyone explain how keep-alive is supposed to work in user-mode, and this worries me in HTTP/3. If your program blocks on something in user-mode, should the connection really die? Doing it in the kernel avoids that. e.g. I know I want to be able to stop my program and continue it in a debugger without breaking network connections and I would be surprised if other people didn't care for this.
- dooglius 7y ago
  A separate thread? I assume that's all the kernel would be doing anyway, no ring zero magic involved.
  - mehrdadn 7y ago
    A debugger stops every thread though?
    - dooglius 7y ago
      Fair point, I was more addressing the blocking-call concern. There are a number of ways you could get your debugger of choice to not halt a particular thread (this would admittedly require some extra work), but won't a typical browser have an application-level timeout that you'd hit anyway.
      - mehrdadn 7y ago
        Browser? I'm not talking about browsers, I'm talking about every other kind of application that tries to use a userland protocol, because that seems to be the direction network libraries are aiming to go. I have no idea how to tell most (all?) debuggers not to halt particular threads, but it sounds painful. And my point is, there are consequences like this that no one seems to care about at all, but that I would hope would've been addressed before seeing people jump on the ship.
    - beagle3 7y ago
      TCP keep alive has a minimum of 2 hours or so. Debugging can live with that. And debugger ruins everything; If the other side sends something larger than the local receive buffer it will usually disconnect after a while. as it will sense no one on the other end. All the things that debugger can "ruin" should just be parametric - increase buffer sizes / keep alive times when debugging. Also, there are more options than "kernel" and "each process for themselves" - you could have a "network/TCP daemon". QNX successfully does that for disk drivers and file system drivers - and likely network too. So does Minix. It's just that historically, unix/linux/NT don't.
pjc50 7y ago
No, it should be implemented in hardware instead. (This is partly a joke and partly a description of what actually happens with TOE "TCP Offload Engine" network cards.)
toast0 7y ago
Some amount of TCP handling needs to be in the kernel to handle arbitration of the shared resource (namely, connection tuples). Once you've already handled IP defragmentation [1] and looking at the structures to get port numbers, and associate that with a userspace file descriptor/process, you've already got all your cachelines primed, and you may as well finish processing the packet, before handing it to userspace. [1] boo hiss; I wish the spec was simply to truncate overlong packets at the MTU, and indicate that with a flag; the peers could then figure out what to do when a packet arrived that was shorter than its original length. Handling it in-band would mean it was more likely to arrive. Instead we can fragmenting it, which is icky, because defragmentation sucks; or we can drop it and sending an out of band message to the sender, but that message may not make it (and often doesn't). TCP could very easily adapt to 'i sent 1480 bytes of payload, but my peer is only acking 1472 each time, maybe I should send 1472 --- much easier and quicker than I keep sending packets and they don't get acked, maybe i should try sending smaller packets 15 seconds later.
- etaioinshrdlu 7y ago
  Maybe the kernel implementation of TCP could have been more lightweight like UDP. Handle socket and port allocation, and then forward all packets to the relevant process unchanged. The application software is then responsible for reconstructing packets into a reliable stream.

Yes, I have not heard any good arguments for why TCP should be in a kernel, other than, it's convenient, and it's always been done that way, and your apps get to share it. Like using the kernel as a shared library... You could put BitTorrent in the kernel. It makes about as much sense architecturally, but isn't too widely used.

phicoh 7y ago
POSIX supports sharing filedescriptors between processes. For example, you can have a process that reads a few bytes from a TCP socket and then passes the socket to another process. Unix tries to model all I/O, including networking, as operations on files. Realistically it is only possible to get this right if the kernel is involved. Of course it is quite possible to come up with different models. But Unix seems to be uniquely powerful in its ability to create complex systems from lots of small processes. The kernel being involved does not imply that all code has to be in the kernel. The FUSE file system interface is a well known way to run filesystem code as user processes. Likewise, there are ways to run device drivers in user space. The disavantage is that the extra context switches cause performance loss. So this approach is use for protocols that are rarely used and do not warrent full kernel implementation.
cat199 7y ago
taken to the logical extreme, why do anything in the kernel for that matter?
- etaioinshrdlu 7y ago
  I think a good rule of thumb should be that a kernel should be responsible making hardware devices safe to use among multiple processes, and little else.

mehrdadn 7y ago

I've never seen anyone explain how keep-alive is supposed to work in user-mode, and this worries me in HTTP/3. If your program blocks on something in user-mode, should the connection really die? Doing it in the kernel avoids that. e.g. I know I want to be able to stop my program and continue it in a debugger without breaking network connections and I would be surprised if other people didn't care for this.

dooglius 7y ago
A separate thread? I assume that's all the kernel would be doing anyway, no ring zero magic involved.
- mehrdadn 7y ago
  A debugger stops every thread though?
  - dooglius 7y ago
    Fair point, I was more addressing the blocking-call concern. There are a number of ways you could get your debugger of choice to not halt a particular thread (this would admittedly require some extra work), but won't a typical browser have an application-level timeout that you'd hit anyway.
    - mehrdadn 7y ago
      Browser? I'm not talking about browsers, I'm talking about every other kind of application that tries to use a userland protocol, because that seems to be the direction network libraries are aiming to go. I have no idea how to tell most (all?) debuggers not to halt particular threads, but it sounds painful. And my point is, there are consequences like this that no one seems to care about at all, but that I would hope would've been addressed before seeing people jump on the ship.
  - beagle3 7y ago
    TCP keep alive has a minimum of 2 hours or so. Debugging can live with that. And debugger ruins everything; If the other side sends something larger than the local receive buffer it will usually disconnect after a while. as it will sense no one on the other end. All the things that debugger can "ruin" should just be parametric - increase buffer sizes / keep alive times when debugging. Also, there are more options than "kernel" and "each process for themselves" - you could have a "network/TCP daemon". QNX successfully does that for disk drivers and file system drivers - and likely network too. So does Minix. It's just that historically, unix/linux/NT don't.

pjc50 7y ago

No, it should be implemented in hardware instead. (This is partly a joke and partly a description of what actually happens with TOE "TCP Offload Engine" network cards.)

toast0 7y ago

Some amount of TCP handling needs to be in the kernel to handle arbitration of the shared resource (namely, connection tuples). Once you've already handled IP defragmentation [1] and looking at the structures to get port numbers, and associate that with a userspace file descriptor/process, you've already got all your cachelines primed, and you may as well finish processing the packet, before handing it to userspace. [1] boo hiss; I wish the spec was simply to truncate overlong packets at the MTU, and indicate that with a flag; the peers could then figure out what to do when a packet arrived that was shorter than its original length. Handling it in-band would mean it was more likely to arrive. Instead we can fragmenting it, which is icky, because defragmentation sucks; or we can drop it and sending an out of band message to the sender, but that message may not make it (and often doesn't). TCP could very easily adapt to 'i sent 1480 bytes of payload, but my peer is only acking 1472 each time, maybe I should send 1472 --- much easier and quicker than I keep sending packets and they don't get acked, maybe i should try sending smaller packets 15 seconds later.

etaioinshrdlu 7y ago
Maybe the kernel implementation of TCP could have been more lightweight like UDP. Handle socket and port allocation, and then forward all packets to the relevant process unchanged. The application software is then responsible for reconstructing packets into a reliable stream.