3 ms·

Well, breaking the socket into two pieces would add complexity. It would create a new requirement for non-trivial programs to add bookkeeping to track the two b

by jerf 3y ago

Well, breaking the socket into two pieces would add complexity. It would create a new requirement for non-trivial programs to add bookkeeping to track the two bits together and make it easier for the two bits to become accidentally permanently estranged from each other. It would add new states to the already complicated TCP set of states. Then again, in most of my "non-trivial" network code I already end up cracking the socket into two pieces anyhow, and while I take advantage of the pun that closing either of them closes the other it would be trivial for me to add this behavior to the separated sockets if that's what I wanted. Partially this is a result of my "non-trivial" network code being in Go so I end up running a goroutine each for send and receive, but still I imagine this would be a common structure one way or another in a lot of non-trivial network servers. But the additional states on the TCP socket would definitely bite some people. Even in its current simpler state it still confuses people and there's a post about how some server turned out to be leaking sockets because it didn't properly close them shows up with some frequency on HN.

dfawcus 3y ago
Nah - one would just add a new syscall, to split the fd. int sock_fd, split_pair[2]; char c; int n = socket_split(sock_fd, split_pair); close(sock_fd); n = write(split_pair[1], "a", 1); close(split_pair[1]); n = read(split_pair[0], &c, 1); read(split_pair[0], &c, 1); /* returns 0 / EOF */ The effect being to split the fds, and give an output similar to pipe(2). Moreover it would also duplicate the underlying open file structure, such that one is read only, the other write only, and both referencing the same socket. Then the read fd would reference one "open file", the write-fd the other. (One can observe a similar issue if one opens FIFO's in read/write mode, but since they can also be opened read-only or write-only, the issue does not arise.)
- jerf 3y ago
  I wonder if perhaps you replied to the wrong comment? I never questioned that it could be done, I was discussing the results from the point of view of it already being done (and for my discussion's purpose, mandatory).
  - dfawcus 3y ago
    I was disagreeing with your first paragraph. If the splitting is optional, as it would have to be now, and as I proposed it would not add complexity since it would be opt in. Theoretically if it had split from the start, it may have added a minor amount of complexity to some programs, but not a significant amount. I don't get the need to add TCP states, as the TCP machine will not change, and the user space app shouldn't really be tracking/replicatting the TCP states. Anyway, your assumed (and mandatory) point of view was not obvious to me, I interpreted it from the position of "if that was done now". Which would add a trivial amount of logic to the kernel, mainly reusing existing logic, structures, and counters. So I guess we were slightly at cross purposes.
jmalicki 3y ago
> It would add new states to the already complicated TCP set of states. Huh? Half-closed connections already exist. The complexity in the state machine is there, and more TCP users should be aware of it (the source of soo.... many... bugs)
- jerf 3y ago
  While the current "half closed state" and "one of the read/write halves is closed and you don't realize it" may sound similar, they're two different "halves". Splitting the socket fully into two separate file handles would mean that users could close one of them without realizing it, and thus be able to write a new sort of bug where they would be able to ask stack overflow "I'm able to read from my socket but not write from it, what gives?" This wouldn't necessarily manifest as a new TCP state in the kernel state table. You could do this to TCP today at the API level alone. Which is further evidence it's a different "half". And if it sounds silly that anyone would make such a mistake, like many such hypothetical mistakes it's important to remember that in the wild it's not always next to each other. My least favorite networking experience ever involved being three levels deep in abstraction in some Perl libraries, every level of which felt like it can and should make all sorts of helpful little calls, the end result being all kinds of fun to debug as soon as any of them were wrong. The thing that closed one of the halves of the socket and the person confused by the result can be arbitrarily separated in the code. Or, to put it another way, you can't grow the API's complexity without growing the API's complexity. If today closing closes both reading and writing but a new API severs that guarantee and creates at least two new states, that inevitably results in a new way for things to go wrong. You can't have one without the other.
  - jmalicki 3y ago
    > Splitting the socket fully into two separate file handles would mean that users could close one of them without realizing it, and thus be able to write a new sort of bug where they would be able to ask stack overflow "I'm able to read from my socket but not write from it, what gives?" shutdown(sockfd, 1) gives you "I'm able to read from my socket but not write from it, what gives?" The real bugs with TCP come from "why am I not able to download the file because the server did close(sockfd) rather than shutdown(sockfd, 1), and my TCP stack ACK'd, causing the server to RST, causing my machine to give me an error before I read all of the data". This is already waaaay harder than the counterfactual you're imagining, and the bugs are here, today.

by jerf 3y ago

dfawcus 3y ago

Nah - one would just add a new syscall, to split the fd. int sock_fd, split_pair[2]; char c; int n = socket_split(sock_fd, split_pair); close(sock_fd); n = write(split_pair[1], "a", 1); close(split_pair[1]); n = read(split_pair[0], &c, 1); read(split_pair[0], &c, 1); /* returns 0 / EOF */ The effect being to split the fds, and give an output similar to pipe(2). Moreover it would also duplicate the underlying open file structure, such that one is read only, the other write only, and both referencing the same socket. Then the read fd would reference one "open file", the write-fd the other. (One can observe a similar issue if one opens FIFO's in read/write mode, but since they can also be opened read-only or write-only, the issue does not arise.)

jerf 3y ago
I wonder if perhaps you replied to the wrong comment? I never questioned that it could be done, I was discussing the results from the point of view of it already being done (and for my discussion's purpose, mandatory).
- dfawcus 3y ago
  I was disagreeing with your first paragraph. If the splitting is optional, as it would have to be now, and as I proposed it would not add complexity since it would be opt in. Theoretically if it had split from the start, it may have added a minor amount of complexity to some programs, but not a significant amount. I don't get the need to add TCP states, as the TCP machine will not change, and the user space app shouldn't really be tracking/replicatting the TCP states. Anyway, your assumed (and mandatory) point of view was not obvious to me, I interpreted it from the position of "if that was done now". Which would add a trivial amount of logic to the kernel, mainly reusing existing logic, structures, and counters. So I guess we were slightly at cross purposes.

jmalicki 3y ago

> It would add new states to the already complicated TCP set of states. Huh? Half-closed connections already exist. The complexity in the state machine is there, and more TCP users should be aware of it (the source of soo.... many... bugs)

jerf 3y ago
While the current "half closed state" and "one of the read/write halves is closed and you don't realize it" may sound similar, they're two different "halves". Splitting the socket fully into two separate file handles would mean that users could close one of them without realizing it, and thus be able to write a new sort of bug where they would be able to ask stack overflow "I'm able to read from my socket but not write from it, what gives?" This wouldn't necessarily manifest as a new TCP state in the kernel state table. You could do this to TCP today at the API level alone. Which is further evidence it's a different "half". And if it sounds silly that anyone would make such a mistake, like many such hypothetical mistakes it's important to remember that in the wild it's not always next to each other. My least favorite networking experience ever involved being three levels deep in abstraction in some Perl libraries, every level of which felt like it can and should make all sorts of helpful little calls, the end result being all kinds of fun to debug as soon as any of them were wrong. The thing that closed one of the halves of the socket and the person confused by the result can be arbitrarily separated in the code. Or, to put it another way, you can't grow the API's complexity without growing the API's complexity. If today closing closes both reading and writing but a new API severs that guarantee and creates at least two new states, that inevitably results in a new way for things to go wrong. You can't have one without the other.
- jmalicki 3y ago
  > Splitting the socket fully into two separate file handles would mean that users could close one of them without realizing it, and thus be able to write a new sort of bug where they would be able to ask stack overflow "I'm able to read from my socket but not write from it, what gives?" shutdown(sockfd, 1) gives you "I'm able to read from my socket but not write from it, what gives?" The real bugs with TCP come from "why am I not able to download the file because the server did close(sockfd) rather than shutdown(sockfd, 1), and my TCP stack ACK'd, causing the server to RST, causing my machine to give me an error before I read all of the data". This is already waaaay harder than the counterfactual you're imagining, and the bugs are here, today.