5 ms·

For a multi-threaded system I don't actually see how any of that is solved by moving things that SHOULD NOT belong in the kernel to the kernel. Edited to add t

For a multi-threaded system I don't actually see how any of that is solved by moving things that SHOULD NOT belong in the kernel to the kernel. Edited to add this paragraph: I feel I should expand on the above line. It's actually the reason why most of the post this is replying to is invalid. Outside of the single thread context __all__ caches are potentially invalid, even the in-kernel cache could be invalidated between different submissions. So all clients have to be coded to accept a 'fail' case and behave properly anyway, OR you allow for one to 'win'. This is why (man 2 open) open() has been extended over time to include the O_EXCL flag for failing in the case of a file already existing. Similarly the (man 2 rename) rename() has been extended (fairly recently) with RENAME_NOREPLACE, which fails in the case of the file already existing. Anything which might be subject to concurrency issues already requires such handling, or accepting that the results might be undefined (E.G. one version of the rename to target filename working in an 'atomic' way but no assurance about the ordering of operations from parallel threads/processes). Anything not facing a user should already be creating filenames in it's preferred case and normalization. Anything facing the user (accepting user input as a filename) should either be doing the same (if it's part of a sanitized filename) OR is directly part of a UI interacting with a user, which must already handle cases such as renaming an existing file to one that might also exist. I really don't buy that anything of actual value is being gained here for use cases that seem very rare and a complexity cost in kernel that is very high.

comex 8y ago
O_EXCL and RENAME_NOREPLACE are examples of why you want case insensitivity to be in the kernel. They're atomic operations: if you use O_EXCL, you could get a successful file creation, or you could get EEXIST, but not something in between. But you can't implement a case insensitive version of O_EXCL in userland. As I said, the best you can do is scan first to see whether a conflicting file already exists, but that doesn't help if a conflicting file is created in between the scan and when you create your file. The kernel is in a better position; since all readers and writers must go through the kernel's filesystem routines, those routines can use mutexes or other concurrency mechanisms (that they need anyway to protect their data structures) to implement higher-level atomic semantics. That's how the kernel implements regular O_EXCL, and it's how it can implement case-insensitive O_EXCL as well, without having to deal with race conditions. In fact it already does that for vfat and other case-insensitive filesystems. (A FUSE filesystem similarly gatekeeps all accesses and thus can do the job in userspace, but FUSE is slow.)
- nybble41 8y ago
  > But you can't implement a case insensitive version of O_EXCL in userland. So leave O_EXCL in the kernel in its current case-sensitive form. In userspace, implement normalization and case-folding as you see fit and pass the normalized, case-folded filename to open(). Ignore any files other programs may create with non-normalized names. Keep an xattr or lookup table with the unfolded "presentation" version of the name for directory listings.

comex 8y ago

O_EXCL and RENAME_NOREPLACE are examples of why you want case insensitivity to be in the kernel. They're atomic operations: if you use O_EXCL, you could get a successful file creation, or you could get EEXIST, but not something in between. But you can't implement a case insensitive version of O_EXCL in userland. As I said, the best you can do is scan first to see whether a conflicting file already exists, but that doesn't help if a conflicting file is created in between the scan and when you create your file. The kernel is in a better position; since all readers and writers must go through the kernel's filesystem routines, those routines can use mutexes or other concurrency mechanisms (that they need anyway to protect their data structures) to implement higher-level atomic semantics. That's how the kernel implements regular O_EXCL, and it's how it can implement case-insensitive O_EXCL as well, without having to deal with race conditions. In fact it already does that for vfat and other case-insensitive filesystems. (A FUSE filesystem similarly gatekeeps all accesses and thus can do the job in userspace, but FUSE is slow.)

nybble41 8y ago
> But you can't implement a case insensitive version of O_EXCL in userland. So leave O_EXCL in the kernel in its current case-sensitive form. In userspace, implement normalization and case-folding as you see fit and pass the normalized, case-folded filename to open(). Ignore any files other programs may create with non-normalized names. Keep an xattr or lookup table with the unfolded "presentation" version of the name for directory listings.