13 ms·

Two File Descriptors for Sockets

• pif 3y ago
  Maybe I'm not getting it completely... In the first example, he is using two pipes; in the second example, he is using one socket. And he complains that that they don't work the same!?!
  • greybrunix 3y ago
    I think the point is that a socket is doing both reading and writing using the same file descriptor, which is counterintuitive.
    • rnmmrnm 3y ago
      and that to half-close a socket you need to use some unintuitive syscall.
      • pif 3y ago
        Use two pipes and you won't need to half-close any socket.
    • Veserv 3y ago
      Which is a problem of their own making. If they use two sockets, which would be needed if all three programs were on different machines, then it would work exactly the same as the pipe example. The problem is that they saw TCP sockets are two-way and then thought to optimize the TCP connection/state overhead by using it for double-duty without applying appropriate abstractions to transparently use a two-way channel as two one-way channels.
      • Gabrys1 3y ago
        Some early protocols used two sockets for (maybe?) roughly this reason
        • adontz 3y ago
          If you mean FTP, then no. One socket is for commands, another is for data. Both bidirectional.
          • dfawcus 3y ago
            Compare with what existed when NCP (RFC 33) was in use. An early FTP protocol (452, 454, 542) originally used a server listen port of 3 (and hence a server "return port" of 2*), but 542 allowed temporary use of port 21 as the protocol was changed. A different pair of ports (even or odd depending upon TX or RX end) were used for the data channel, and the port picked was related to the Telnet protocol port in use. RFC 640 then seems to have fixed the use of port 21 instead of 3. Then when TCP came in to use RFC 765 seems to have simplified things somewhat, sticking to port 21. Also see RFC 165 for the/a Telnet Initial Connection Protocol showing the use of the two even/odd ports for forming a bidirectional connection. * Or 4, but I believe the return channel was one port below, not above.
            • adontz 3y ago
              I believe that's a different story explained here https://news.ycombinator.com/item?id=14178993 https://news.ycombinator.com/item?id=14178993
        • dfawcus 3y ago
          Sort of - NCP (RFC 33) had two simplex sockets. Even being receive, odd transmit. That then led to various higher level protocols using pairs of ports. Once TCP took over, that was relaxed.
  • deleted 3y ago
    [deleted]
  • cryptonector 3y ago
    Setting up two sockets would be even more work. The best mitigation is to not close `stdin` when you get EOF on it -- just stop reading from `stdin`.
• murderfs 3y ago
  This is nonsense, pipes are the weird case, not the typical. The socket interface looks largely identical to ttys, which were in the first version of Unix over a decade before Berkeley sockets were introduced. Pipes didn't exist until Research Unix v3.
  • gpderetta 3y ago
    While historically you are correct, having a single file descriptor for reading and writing to distinct streams is confusing and can break code. Bidirectionally buffered wrappers over file descriptors (FILE, std::fstream for example) share the buffer between the read and write side, so you have to use distinct buffered wrappers with dup'd file descriptor anyway. Also the tty is typically connected to stdin, stdout and stderr, that are separate file descriptor (again, a leaky abstraction when they refer to the same description).
  • geertj 3y ago
    Not a lot of application software interacts directly with ttys. I think the more relevant example here is stdio, which uses separate fds for reading and writing, like the pipe example.
• halayli 3y ago
  The difference is that the callee working on the fd it received need to check if fd's type is pipe or socket and call close() or shutdown(fd, 1) accordingly. This is not a pipe vs socket issue only but a penalty for having a vfs abstraction. Files, pipes, sockets, and more all have stream like behavior but they can vary widely and many cases demand leaking the vfs abstraction to check what's the fd type.
  • vacuity 3y ago
    For a new OS, would it be good to circumvent this with a rudimentary runtime type system managed by the kernel? It would probably cost a bit of memory and performance overhead on syscalls, and eat up the syscall space. It could be reflected in the standard OS types in programming languages.
  • ATMLOTTOBEER 3y ago
    When I read beej’s guide to network programming I was confused when he said you have to call close() even after you call shutdown(). The way I understood it was that shutdown gave you granular control to shut down only the reading or writing part of the connection. Reading your comment I am even more lost lol
    • halayli 3y ago
      ah yes that's because close() closes the file descriptor but shutdown does not. You can have a socket that is shutdown on both ends that has a valid fd.
• gary_0 3y ago
  TCP was clearly designed by people who strove for elegance and understood the Unix philosophy, and you almost want to blame something else when all the wonky edge cases show up. Witness all the NODELAY issues, DDoS mitigations, and contortions (and eventual abandonment) that HTTP resorted to. TCP should have Just Worked, but alas, we live in an imperfect universe.
  • Joker_vD 3y ago
    I am fairly certain that TCP and IP were designed and published before UNIX started being widely distributed (or known). And Berkeley sockets appeared in the early eighties, about ten years after TCP.
    • gary_0 3y ago
      It was probably more of a co-evolution as the design of the Internet and Unix solidified, building on earlier established principles. But you can see the same thinking in both designs, the idea of plumbing together simple systems in a flexible way. A very successful approach since both TCP/IP and Unix-like computing are still in use today.
• charcircuit 3y ago
  It sounds like the author forgot to dup the fd before giving it to consume-data. If he would have done that it would have closed the socket only once both fd were closed and the fds would have separate lifetimes like in the pipe example.
  • dfawcus 3y ago
    Nope, and (IMHO) you're missing the point. duping the fd's as you describe would achieve nothing, as both fds will share the same file, and hence while either fd is open, the kernel can not detect a close. The process-data task is on a remote machine, probably listening on a single port, and feeding its output back over the same TCP connection as its input. Some process has then set up the series of connections, by connecting to the remote server, plumbing and doing fork/exec to run consume-data with the socket on stdin, while doing an exec of generate-data having the socket on stdout. Now the processes being run are assumed to be general unix processes, i.e. pipeline processes ala how cat, head, grep etc operate. They generally read from input, write to output, and exit when the input is exhausted. That "exit signal" will then be fed through the pipeline as a series of "EOF" notifications, which FIN can achieve for TCP. However because the output fd from generate-data, and the input fd to consume-data both share the same underlying socket, the mere close of the output fd from generate-data exiting will not result in a FIN being sent to process-data. That is because the same "file" (the socket) is open by consume-data (in read/write mode), and so the kernel can not know that consume-data will not write to the socket. If the UNIX abstraction for the sockets had the ability to split the socket across to "files", not just fds, then the issue would be removed; but it doesn't.
    • charcircuit 3y ago
      >and hence while either fd is open, the kernel can not detect a close. Which is what the author wants. He wants the socket to close once both generate-data and consume-data are done with the socket.
      • dfawcus 3y ago
        No, he wants a way for the close of the FD from generate-data to process-data to trigger a TCP FIN over the connection, such that process-data sees an input EOF. However for that to work, the two FDs (generate-date to process-data, and process-data to consume-data) must not be associated with the same "open file", even though they could (with a change) represent the same TCP socket. i.e. he wants generate-data to exit, closing its output FD, and have that drive EOF all the way through to consume-data. Such that consume-data does not determine it is done until it sees EOF on its input socket. (One can observe a similar issue if one opens FIFO's in read/write mode, but since they can also be opened read-only or write-only, the issue does not arise.)
• tuyiown 3y ago
  FD have been designed for file systems where you definitely wants to share the same data structure for read/write/seek and imply that by giving it the same ID. The fact that seek does not work on socket is just because socket is a sub specialisation of FD manipulation, just as pipes are.
• ceving 3y ago
  It is always a pleasure to read D. J. Bernstein.
• twic 3y ago
  No. Standard input and output should be a single file descriptor, like sockets.
  • ksherlock 3y ago
    Do you even pipe?
• tonyg 3y ago
  Lots of the complexity of TCP at a protocol level comes from treating it as a single duplex connection instead of a pair of simplex connections. And the article here shows that this complexity leaks up to the API level, too.
  • amelius 3y ago
    Isn't there a good reason for it though?
    • jerf 3y ago
      Well, breaking the socket into two pieces would add complexity. It would create a new requirement for non-trivial programs to add bookkeeping to track the two bits together and make it easier for the two bits to become accidentally permanently estranged from each other. It would add new states to the already complicated TCP set of states. Then again, in most of my "non-trivial" network code I already end up cracking the socket into two pieces anyhow, and while I take advantage of the pun that closing either of them closes the other it would be trivial for me to add this behavior to the separated sockets if that's what I wanted. Partially this is a result of my "non-trivial" network code being in Go so I end up running a goroutine each for send and receive, but still I imagine this would be a common structure one way or another in a lot of non-trivial network servers. But the additional states on the TCP socket would definitely bite some people. Even in its current simpler state it still confuses people and there's a post about how some server turned out to be leaking sockets because it didn't properly close them shows up with some frequency on HN.
      • dfawcus 3y ago
        Nah - one would just add a new syscall, to split the fd. int sock_fd, split_pair[2]; char c; int n = socket_split(sock_fd, split_pair); close(sock_fd); n = write(split_pair[1], "a", 1); close(split_pair[1]); n = read(split_pair[0], &c, 1); read(split_pair[0], &c, 1); /* returns 0 / EOF */ The effect being to split the fds, and give an output similar to pipe(2). Moreover it would also duplicate the underlying open file structure, such that one is read only, the other write only, and both referencing the same socket. Then the read fd would reference one "open file", the write-fd the other. (One can observe a similar issue if one opens FIFO's in read/write mode, but since they can also be opened read-only or write-only, the issue does not arise.)
• dmw_ng 3y ago
  Ran into a variant of this when working on an IO loop, ended up with a little "Side" abstraction to wrap bidirectional/unidirectional descriptors to behave identically for the rest of the library. It's an annoying wart but pretty easy to avoid in userspace.. https://github.com/mitogen-hq/mitogen/blob/798032b9/mitogen/core.py#L1960 https://github.com/mitogen-hq/mitogen/blob/798032b9/mitogen/...
• gavinhoward 3y ago
  Honestly, I think a better idea would be to write a program that can act as an intermediary, sending the stdin data through TCP, and writing out the data it receives back through stdout. ...can curl already do that?
  • AdamH12113 3y ago
    I think what's you're looking for is netcat.
    • gavinhoward 3y ago
      Ah, yes, that would work.
  • ksherlock 3y ago
    That's one approach and Daniel Bernstein is aware of it. http://cr.yp.to/ucspi-tcp/tcpclient.html http://cr.yp.to/ucspi-tcp/tcpclient.html
• throwawaaarrgh 3y ago
  That ship sailed years ago. We could have had "pipes between hosts" with a distributed OS design like Plan9. But Linus doesn't care, so we've got what we got. Until somebody forks the kernel and goes their own way for fun, it will never change.
  • deleted 3y ago
    [deleted]
  • bvrmn 3y ago
    These remote pipes are a largely a pipe dream due to very agile crypto scene. It requires a bloated spec to describe policies and be interoperable standard nobody would use. `command1 | ssh host process | command2` works perfectly fine.
    • MisterTea 3y ago
      They're a pipe dream because you're thinking in terms of Unix and ssh. On plan 9 you just use http://man.postnix.pw/9front/8/tlssrv http://man.postnix.pw/9front/8/tlssrv
      • bvrmn 3y ago
        And what prevents to run stunnel on Linux? Is there a way to specify remote address or remote run to use TLS to wrap your application transparently in plan 9? Otherwise plan 9 on the same abstraction level.
        • MisterTea 3y ago
          I'm not sure what you mean but running software remotely is accomplished via http://man.postnix.pw/9front/1/rcpu http://man.postnix.pw/9front/1/rcpu
        • rakoo 3y ago
          stunnel is the tired version of inter-machines communication. It's a basic, raw socket between two processes, and all layers above must be reinvented again and again. Plan 9 with its protocol 9p is the wired version of inter-machines communication. You don't just link two random processes, you have the possibility to interconnect two directory trees, and since most useful software works through a directory tree, this means two processes can be linked by importing another tree on another machine and doing everything locally. It's possible to write something like "echo UNSEEN | imap-search | imap-get-summary" and have actual IMAP stuff done on another machine, and you don't see any of it. It is also possible to use `rcpu` to do what ssh does, and even more: the remote machine has access to the local machine. So for instance you can edit files locally, and have a rcpu connection to run builds on another machine that picks it from your local workstation. No clunky sshfs, no failing NFS, ...
• jchw 3y ago
  I have always had this sort of nagging feeling about the way BSD sockets works: it really is a bit strange that the reading and writing operations are effectively entirely separate. When you write to most FDs, the inverse operation is reading; but sockets don't work that way, reading and writing are effectively working on two different streams! The insight that a TCP connection is basically equivalent to a pipe is a pretty good demonstration of how this could have been better. As another related example, imagine if the same had been done for standard I/O. What if instead of having separate file descriptors, all of the standard I/O happened through one file descriptor? Imagine how much more painful everything would be. Unfortunately, this is now pretty much how a lot of stuff works, but it did not really have to go this way.
• jcalvinowens 3y ago
  An easy counterargument is that sockets support out of band read operations: for example, I can call getsockname() on a socket to obtain meaningful information about it after shutdown(). Pipes have no such analogue. Also, it's a general interface. It's easy to imagine exotic protocols where shutdown() is not so cleanly equivalent to close() as it is for TCP. TBH I find it odd to focus on a preference for one semantic over the other. The real thing we can all probably agree on is that in a perfect world, they ought to be the same.
• numeromancer 3y ago
  He built several utilities for doing what is described here. https://cr.yp.to/ucspi-tcp.html https://cr.yp.to/ucspi-tcp.html These are great for duct-taping connections. I created a whole suite of utilities for another programs for uploading firmware updates. I was able to create one program for handling the uploads with several different comms protocols and devices, without having to handle details of tcp connections, serial connections, UDP, etc. socat is another great tool for this.
• ck45 3y ago
  Interesting parallel that pipes are bidirectional in Plan 9.