3 ms·

> For example, from a filesystem driver's perspective, "file descriptors" don't exist! Instead, read(2) and write(2) calls are stateless That seems hard to bel

> For example, from a filesystem driver's perspective, "file descriptors" don't exist! Instead, read(2) and write(2) calls are stateless That seems hard to believe. It would inefficient to look for a file for each e.g. write(). I guess a filesystem defines its own implementation-defined open-file handle and the kernel translates that into an (integer) FD. > each call getting passed a kernel-side io-handle struct that must get re-evaluated for matching permissions on each IO operation. (You can do some up-front evaluation during the open(2) call, but, given that a file's permissions might change while you have a descriptor open to it, there's not much point.) That's not how it works. File modes are checked when you open the file. Once you have an open file (a file handle), the file modes no longer matter. File modes are more a "PATH" thing (there aren't per-filesystem variations and I assume permission checking is not done in file system code), although the file system must allocate the space to save the file mode bits and implement the VFS API.

codys 10y ago
Yep, this is misleading. "file descriptors" taken _literally_ don't exist (they are just the interface exposed to user space programs). Linux uses `struct file` internally as the generic file tracking, and multiple file descriptors can refer to a single `struct file`. `struct file` in turn refers to a `struct inode`, which represents an actual "something" on disk. Each file system driver stores a bunch of info in the `struct inode`. In addition to that tracking, there is generally a bunch of fs-specific data structures to handle lookup of names (in the typical case, these are essentially cached versions of what is stored on disk).
loeg 10y ago
> That seems hard to believe. It would inefficient to look for a file for each e.g. write(). I guess a filesystem defines its own implementation-defined open-file handle and the kernel translates that into an (integer) FD. At least in BSD, the syscall layer calls into the kernel virtual filesystem, which translates file descriptors into file objects. File objects have fo_read/fo_write methods implemented by different kinds of file object. Files backed by filesystems have an associated "vnode" (virtual inode object). Then the fo_read/fo_write layer invokes the VOP_READ or VOP_WRITE (for example) method on the vnode. VOP ("vnode operation") methods are implemented by individual filesystems. Multiple user-level file descriptors can refer to a single vnode. The filesystem's inode object hangs off a pointer from the generic vnode. Vnodes are unique (1:1) per real file (inode). So: read(fd, ...) -> sys_read -> kern_readv -> fget() (translate fd number into struct file object; check that the descriptor was opened read/write as appropriate for the operation) dofileread() -> fo_read() -> vn_io_fault() (fo_read method for vnodes) -> vn_read -> VOP_READ(vnode, uio, ioflags, ucred) (vnode for the file, an io descriptor ("uio"), io flags, and user credentials) So yeah, at the filesystem layet (VOP_READ), you no longer have the fd (or even the file object) in the BSD model. Linux's model is similar, but not completely identical. They call "vnodes" "inodes" instead, and may not have the intermediary file layer.
derefr 10y ago
> there aren't per-filesystem variations and I assume permission checking is not done in file system code There are! Keep in mind that "filesystems" includes things like NFS and SMB. The local kernel has no ultimate authority over security policy of a remotely-mounted filesystem, right? In open(2), the kernel has to ask the filesystem what's what, because the filesystem might know something the kernel doesn't. Now, the interpretation of stat(2) values (UID/GID, ACLs, etc.) are up to the VFS—but the filesystem is the one that created that stat struct when the kernel called its stat(2) impl, and it expects to get that stat-struct passed back with no changes to open(2), and then make the decision for itself what those stat-struct members mean. Which is to say, it's perfectly possible to write a filesystem that says you have 0000 permissions on a file, but which still lets you open(2), read(2), write(2), readdir(2), etc. that file! It's up to the filesystem to enforce file permissions or ACLs, and it can do that however it wishes; stat(2) is just an indication, in a common VFS language, of the policy the filesystem is (probably) going to enforce. It's not a baton passed to the kernel to do the enforcement for it. Linux has no equivalent to NT's kernel-object ACLs. > File modes are more a "PATH" thing (there aren't per-filesystem variations and I assume permission checking is not done in file system code), although the file system must allocate the space to save the file mode bits and implement the VFS API. Ah, sorry, I didn't mean file permissions; was a typo. I mean things like, the process on the other end of a pipe closing its write end, will cause your read(2) call to that pipe's FD to fail, because the IO permissions (not file permissions) on the FD have changed between the two successive read(2) calls. When your open(2) impl gets called, you receive a stat(2) [that you previously created yourself when the kernel called your stat(2) implementation], and a set of open(2) flags, compare the two, and decide whether to grant each requested permission from open(2) given the stat-struct. Essentially, the open(2) impl is a pure mask-function on the requested flags, to determine what permissions actually end up put into the descriptor. (Conveniently, the kernel then returns a permissions error if it doesn't get returned the perms it asks for. But it could always end up with more perms than it asked for!) Then, later, the kernel can modify that set of IO permissions without telling you, and your next read(2) or write(2) might get called with different IO permissions. Another interesting fact: in the VFS struct file_operations (where you put your pointers to your filesystem's implementations of file operations), there is no member representing close(2). No FS-driver-level function gets called by the kernel in response to close(2)! Instead: • There is a flush(2) that gets passed a file struct, to indicate to the FS that a given file's handle has been closed—but this is only there so that, if the file is part of a filesystem with synchronous-commit (e.g. NFS in sync mode), closing the file will trigger a flush of the entire device. This is stateless and idempotent; a given file-struct might get flush(2)ed any number of times. It's there to ask the the file's extents' backing store to checkpoint itself, not to do anything with the file itself. • There is a release(2) that gets called when all handles to a file have been closed—i.e. when the kernel's "open(2) refcount" on the file-struct drops to zero. If the filesystem, say, caches some things about the file when you open(2), you can release that cache-entry on release(2). Notice that neither of these operations has semantics that would let you clean up local state allocated in a table keyed off anything passed to open(2), because there's no call that happens 1:1 with open(2) calls. Thus, you really can't key local state to a struct-file in a way where you can later look it up again. And there's nowhere inside a struct-file to stash a key for your local state, either. So, like I said, read(2) and write(2) are "stateless."

codys 10y ago

Yep, this is misleading. "file descriptors" taken _literally_ don't exist (they are just the interface exposed to user space programs). Linux uses `struct file` internally as the generic file tracking, and multiple file descriptors can refer to a single `struct file`. `struct file` in turn refers to a `struct inode`, which represents an actual "something" on disk. Each file system driver stores a bunch of info in the `struct inode`. In addition to that tracking, there is generally a bunch of fs-specific data structures to handle lookup of names (in the typical case, these are essentially cached versions of what is stored on disk).

loeg 10y ago

> That seems hard to believe. It would inefficient to look for a file for each e.g. write(). I guess a filesystem defines its own implementation-defined open-file handle and the kernel translates that into an (integer) FD. At least in BSD, the syscall layer calls into the kernel virtual filesystem, which translates file descriptors into file objects. File objects have fo_read/fo_write methods implemented by different kinds of file object. Files backed by filesystems have an associated "vnode" (virtual inode object). Then the fo_read/fo_write layer invokes the VOP_READ or VOP_WRITE (for example) method on the vnode. VOP ("vnode operation") methods are implemented by individual filesystems. Multiple user-level file descriptors can refer to a single vnode. The filesystem's inode object hangs off a pointer from the generic vnode. Vnodes are unique (1:1) per real file (inode). So: read(fd, ...) -> sys_read -> kern_readv -> fget() (translate fd number into struct file object; check that the descriptor was opened read/write as appropriate for the operation) dofileread() -> fo_read() -> vn_io_fault() (fo_read method for vnodes) -> vn_read -> VOP_READ(vnode, uio, ioflags, ucred) (vnode for the file, an io descriptor ("uio"), io flags, and user credentials) So yeah, at the filesystem layet (VOP_READ), you no longer have the fd (or even the file object) in the BSD model. Linux's model is similar, but not completely identical. They call "vnodes" "inodes" instead, and may not have the intermediary file layer.

derefr 10y ago

> there aren't per-filesystem variations and I assume permission checking is not done in file system code There are! Keep in mind that "filesystems" includes things like NFS and SMB. The local kernel has no ultimate authority over security policy of a remotely-mounted filesystem, right? In open(2), the kernel has to ask the filesystem what's what, because the filesystem might know something the kernel doesn't. Now, the interpretation of stat(2) values (UID/GID, ACLs, etc.) are up to the VFS—but the filesystem is the one that created that stat struct when the kernel called its stat(2) impl, and it expects to get that stat-struct passed back with no changes to open(2), and then make the decision for itself what those stat-struct members mean. Which is to say, it's perfectly possible to write a filesystem that says you have 0000 permissions on a file, but which still lets you open(2), read(2), write(2), readdir(2), etc. that file! It's up to the filesystem to enforce file permissions or ACLs, and it can do that however it wishes; stat(2) is just an indication, in a common VFS language, of the policy the filesystem is (probably) going to enforce. It's not a baton passed to the kernel to do the enforcement for it. Linux has no equivalent to NT's kernel-object ACLs. > File modes are more a "PATH" thing (there aren't per-filesystem variations and I assume permission checking is not done in file system code), although the file system must allocate the space to save the file mode bits and implement the VFS API. Ah, sorry, I didn't mean file permissions; was a typo. I mean things like, the process on the other end of a pipe closing its write end, will cause your read(2) call to that pipe's FD to fail, because the IO permissions (not file permissions) on the FD have changed between the two successive read(2) calls. When your open(2) impl gets called, you receive a stat(2) [that you previously created yourself when the kernel called your stat(2) implementation], and a set of open(2) flags, compare the two, and decide whether to grant each requested permission from open(2) given the stat-struct. Essentially, the open(2) impl is a pure mask-function on the requested flags, to determine what permissions actually end up put into the descriptor. (Conveniently, the kernel then returns a permissions error if it doesn't get returned the perms it asks for. But it could always end up with more perms than it asked for!) Then, later, the kernel can modify that set of IO permissions without telling you, and your next read(2) or write(2) might get called with different IO permissions. Another interesting fact: in the VFS struct file_operations (where you put your pointers to your filesystem's implementations of file operations), there is no member representing close(2). No FS-driver-level function gets called by the kernel in response to close(2)! Instead: • There is a flush(2) that gets passed a file struct, to indicate to the FS that a given file's handle has been closed—but this is only there so that, if the file is part of a filesystem with synchronous-commit (e.g. NFS in sync mode), closing the file will trigger a flush of the entire device. This is stateless and idempotent; a given file-struct might get flush(2)ed any number of times. It's there to ask the the file's extents' backing store to checkpoint itself, not to do anything with the file itself. • There is a release(2) that gets called when all handles to a file have been closed—i.e. when the kernel's "open(2) refcount" on the file-struct drops to zero. If the filesystem, say, caches some things about the file when you open(2), you can release that cache-entry on release(2). Notice that neither of these operations has semantics that would let you clean up local state allocated in a table keyed off anything passed to open(2), because there's no call that happens 1:1 with open(2) calls. Thus, you really can't key local state to a struct-file in a way where you can later look it up again. And there's nowhere inside a struct-file to stash a key for your local state, either. So, like I said, read(2) and write(2) are "stateless."