3 ms·

It's not clear if the author knows that there's a shared file structure in the kernel, and file descriptors simply point to this. When you open a new file you

by rwmj 3y ago

It's not clear if the author knows that there's a shared file structure in the kernel, and file descriptors simply point to this. When you open a new file you get a new file descriptor as well, so there's a 1-1 correspondence normally. But dup/dup2 exposes the indirection by only creating a new file descriptor that points to the old file (so this way you end up with multiple fds pointing to a single file structure). More interestingly this applies across processes, eg. after fork. Here's the file struct in Linux (f_pos is the file offset): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/fs.h#n986 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... And here's the fd array in Linux (note file descriptors are indexes into fd_array which is an array of pointers to struct file): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/fdtable.h#n46 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

usrnm 3y ago
The author is talking about a cross-platform interface standardised by POSIX, it's not really the same thing as the details of one particular implementation. It isn't about how, but why
- epcoa 3y ago
  The gp isn’t only talking about a particular implementation. The concept and terminology of an “open file description” (represented by a struct file in Linux and BSD and others) in contrast to a “file descriptor” are standardized/documented in POSIX.
- pjmlp 3y ago
  This is part of POSIX. => The dup() function provides an alternative interface to the service provided by fcntl() using the F_DUPFD command. The call dup(fildes) shall be equivalent to: => fcntl(fildes, F_DUPFD, 0); -- https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ Going to fcntl() docs, => The new file descriptor shall refer to the same open file description as the original file descriptor, and shall share any locks -- https://pubs.opengroup.org/onlinepubs/9699919799/functions/fcntl.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f... Which implies being exactly the same kernel data structure for the file description, with all its current state.
nly 3y ago
I love that the Linux kernel codebase is approachable enough that these things are easy to find
gpderetta 3y ago
I was bitten by this recentishly and I had to learn the difference between file descriptors and descriptions. I had a program that was misbehaving when talking to a tty, but not when the output was piped through cat. Turns the program would set O_NONBLOCK (a description level property) on its input, but of course the shell was dup'ing the input tty fd to output as well, implicitly making the output non-blocking as well. Setting the output to a different description (i.e. the pipe) would workaround that.
- 392 3y ago
  For a long time `npm` would do that to your stdout/in, which didn't impact anything in interactive shells because your shell resets it every time it shows the prompt, but run those same commands in a bash script and later processes will crash with strange error messages because no one has ever tested with stdout being nonblocking. So much for separate processes being independent.
- deleted 3y ago
  [deleted]
senderista 3y ago
Well, the author of epoll(2) apparently didn't know that either, so...
JdeBP 3y ago
I am certain that M. Siebenmann has known about file descriptions for decades. The problem with a "here is what Linux does" response is that it actually bespeaks of the respondent not knowing enough. Linux isn't even relevant, as M. Siebenmann is talking of design decisions made 20 years before Linux was even conceived. Whereas there were contemporary operating systems that used everything from file control blocks to file handles, and did not have these semantics. Nor indeed did they have the higher-level command processor semantics that they enabled. The question M. Siebenmann was posing is in part whether it was inevitable that multi-process operating system designs ended up with these particular shared file pointer semantics, or just an accident of the fact that we got the idea of shared "standard" I/O streams long before we got the idea of forcing all sequential-access write operations to always append with a flag.

by rwmj 3y ago

usrnm 3y ago

The author is talking about a cross-platform interface standardised by POSIX, it's not really the same thing as the details of one particular implementation. It isn't about how, but why

epcoa 3y ago
The gp isn’t only talking about a particular implementation. The concept and terminology of an “open file description” (represented by a struct file in Linux and BSD and others) in contrast to a “file descriptor” are standardized/documented in POSIX.
pjmlp 3y ago
This is part of POSIX. => The dup() function provides an alternative interface to the service provided by fcntl() using the F_DUPFD command. The call dup(fildes) shall be equivalent to: => fcntl(fildes, F_DUPFD, 0); -- https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ Going to fcntl() docs, => The new file descriptor shall refer to the same open file description as the original file descriptor, and shall share any locks -- https://pubs.opengroup.org/onlinepubs/9699919799/functions/fcntl.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f... Which implies being exactly the same kernel data structure for the file description, with all its current state.

nly 3y ago

I love that the Linux kernel codebase is approachable enough that these things are easy to find

gpderetta 3y ago

I was bitten by this recentishly and I had to learn the difference between file descriptors and descriptions. I had a program that was misbehaving when talking to a tty, but not when the output was piped through cat. Turns the program would set O_NONBLOCK (a description level property) on its input, but of course the shell was dup'ing the input tty fd to output as well, implicitly making the output non-blocking as well. Setting the output to a different description (i.e. the pipe) would workaround that.

392 3y ago
For a long time `npm` would do that to your stdout/in, which didn't impact anything in interactive shells because your shell resets it every time it shows the prompt, but run those same commands in a bash script and later processes will crash with strange error messages because no one has ever tested with stdout being nonblocking. So much for separate processes being independent.
deleted 3y ago
[deleted]

senderista 3y ago

Well, the author of epoll(2) apparently didn't know that either, so...

JdeBP 3y ago

I am certain that M. Siebenmann has known about file descriptions for decades. The problem with a "here is what Linux does" response is that it actually bespeaks of the respondent not knowing enough. Linux isn't even relevant, as M. Siebenmann is talking of design decisions made 20 years before Linux was even conceived. Whereas there were contemporary operating systems that used everything from file control blocks to file handles, and did not have these semantics. Nor indeed did they have the higher-level command processor semantics that they enabled. The question M. Siebenmann was posing is in part whether it was inevitable that multi-process operating system designs ended up with these particular shared file pointer semantics, or just an accident of the fact that we got the idea of shared "standard" I/O streams long before we got the idea of forcing all sequential-access write operations to always append with a flag.