3 ms·

It's great that file names in the user/kernel ABI are treated as unstructured NUL terminated byte streams so I can do what I want with my file names even if you

It's great that file names in the user/kernel ABI are treated as unstructured NUL terminated byte streams so I can do what I want with my file names even if you don't like it. And you can do what you want with yours, including not creating ones you think are idiotic, or using filesystems with code or options that restrict what names can be used.

CorrectHorseBat 5y ago
But is there any real use case for that? For me I've only encountered this when something else went wrong, I'd rather have an error at that time than later trying to find out what this garbage is and how to remove it. So why not give a mount option for this behavior?
- throwawaylinux 5y ago
  > But is there any real use case for that? Compatibility, at least. Which is actually a big one and is basically never broken in Linux. > So why not give a mount option for this behavior? Not sure, maybe just nobody yet cared enough to code it up and submit it for inclusion. It's never caused me problems.
  - CorrectHorseBat 5y ago
    Compatibility with what? Is there any software that relies on this behavior?
    - throwawaylinux 5y ago
      > Compatibility with what? With existing applications and filesystem images. > Is there any software that relies on this behavior? Possibly.
      - CorrectHorseBat 5y ago
        Possibly, so possibly even not. The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior.
        throwawaylinux 5y ago
        > Possibly, so possibly even not. Possibly so. > The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior. Certainly not something like this by default though. To be clear, this wouldn't somehow solve shell / scripting / terminal issues with file names. There are many other special characters and escape sequences and other whitespace like spaces that can trip up incorrectly written programs. These can certainly not all be removed by the kernel so the incremental advantage of just filtering out a couple of such cases doesn't seem like it would be very big.
patrec 5y ago
Can you give a plausible use case? Filenames can't be arbitrary bytes anyway, since they cannot contain '\0' and '/'. What's a realistic example where it's really useful to be able to stuff arbitrary bytes into a filename, just not '/' or '\0' and where C or URL escaping would somehow be onerous enough to justify all the other problems these pathological filenames create? Do you really think disallowing pathological filenames (at least as a mount option) would be more expensive than the countless security exploits allowing them has already caused or massive tax nearly all software that tries to deal with filenames robustly needs to pay for it? Forget shell scripting, almost no software can afford to just pretend filenames are arbitrary bytes. They typically still somehow need to be displayed to and be editable by end users somewhere along the line, and this means (in unix-based systems) some conversion to-and-from utf-8. Which is going to cause problems[1]. And even if you don't directly need to handle this yourself (but you do, even for a simple shell script or command-line utility or a library that wants to provide an error message with a filename), there is now a whole lot of extra bloat and complexity and edge cases no one handles in practice. With weirdo types like special filename strings, which are neither bytes or proper unicode like python's unicode surrogate encoding (which effectively leaks into all text handling). And of course different languages and eco-systems solve it differently (e.g. whereas python bends its general unicode string for this, Rust has a OsString). [1] Even the utf-8 compatible subset causes problems of course. E.g. if you have a terminal program that needs to display untrusted filenames to an end user, you now have to deal with problems like terminal escape injection via filenames.
- throwawaylinux 5y ago
  Compatibility. And a mount option seems fine if you don't need compatibility. That does not relieve applications of the requirement to robustly handle paths and file names though. > Forget shell scripting, almost no software can afford to just pretend filenames are arbitrary bytes. Much non-script software can actually treat file names as arbitrary bytes and just pass them through its typical input and output mechanisms. Shells and terminals are very special classes of application, and they need a lot of I/O sanitization whether or not the filesystem restricts file names.

CorrectHorseBat 5y ago

But is there any real use case for that? For me I've only encountered this when something else went wrong, I'd rather have an error at that time than later trying to find out what this garbage is and how to remove it. So why not give a mount option for this behavior?

throwawaylinux 5y ago
> But is there any real use case for that? Compatibility, at least. Which is actually a big one and is basically never broken in Linux. > So why not give a mount option for this behavior? Not sure, maybe just nobody yet cared enough to code it up and submit it for inclusion. It's never caused me problems.
- CorrectHorseBat 5y ago
  Compatibility with what? Is there any software that relies on this behavior?
  - throwawaylinux 5y ago
    > Compatibility with what? With existing applications and filesystem images. > Is there any software that relies on this behavior? Possibly.
    - CorrectHorseBat 5y ago
      Possibly, so possibly even not. The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior.
      - throwawaylinux 5y ago
        > Possibly, so possibly even not. Possibly so. > The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior. Certainly not something like this by default though. To be clear, this wouldn't somehow solve shell / scripting / terminal issues with file names. There are many other special characters and escape sequences and other whitespace like spaces that can trip up incorrectly written programs. These can certainly not all be removed by the kernel so the incremental advantage of just filtering out a couple of such cases doesn't seem like it would be very big.

patrec 5y ago

Can you give a plausible use case? Filenames can't be arbitrary bytes anyway, since they cannot contain '\0' and '/'. What's a realistic example where it's really useful to be able to stuff arbitrary bytes into a filename, just not '/' or '\0' and where C or URL escaping would somehow be onerous enough to justify all the other problems these pathological filenames create? Do you really think disallowing pathological filenames (at least as a mount option) would be more expensive than the countless security exploits allowing them has already caused or massive tax nearly all software that tries to deal with filenames robustly needs to pay for it? Forget shell scripting, almost no software can afford to just pretend filenames are arbitrary bytes. They typically still somehow need to be displayed to and be editable by end users somewhere along the line, and this means (in unix-based systems) some conversion to-and-from utf-8. Which is going to cause problems[1]. And even if you don't directly need to handle this yourself (but you do, even for a simple shell script or command-line utility or a library that wants to provide an error message with a filename), there is now a whole lot of extra bloat and complexity and edge cases no one handles in practice. With weirdo types like special filename strings, which are neither bytes or proper unicode like python's unicode surrogate encoding (which effectively leaks into all text handling). And of course different languages and eco-systems solve it differently (e.g. whereas python bends its general unicode string for this, Rust has a OsString). [1] Even the utf-8 compatible subset causes problems of course. E.g. if you have a terminal program that needs to display untrusted filenames to an end user, you now have to deal with problems like terminal escape injection via filenames.

throwawaylinux 5y ago
Compatibility. And a mount option seems fine if you don't need compatibility. That does not relieve applications of the requirement to robustly handle paths and file names though. > Forget shell scripting, almost no software can afford to just pretend filenames are arbitrary bytes. Much non-script software can actually treat file names as arbitrary bytes and just pass them through its typical input and output mechanisms. Shells and terminals are very special classes of application, and they need a lot of I/O sanitization whether or not the filesystem restricts file names.