6 ms·

More importantly, we need to get rid of the ability to put line feeds, tabs in file names and also disallow odd starting characters such as tab, dash and $ I w

More importantly, we need to get rid of the ability to put line feeds, tabs in file names and also disallow odd starting characters such as tab, dash and $ I wish someone would add a mount option for that and have eg fedora be a trailblazer to fix the few apps that break

Tepix 5y ago
It would be almost trivial to create a fuse filesystem which completely hides these files if they exist (and doesn't allow the creation of new ones).
- patrec 5y ago
  And how would that be even remotely useful? Unless something changed recently, FUSE has so much overhead it's only useful for niche applications and prototyping.
- laumars 5y ago
  It would be trivial to code but any such wrapper would add overhead to file system operations. FUSE is a fantastic set of APIs (I’ve used it personally) and performs remarkably well considering it is constantly swapping memory between kernel and user space but for wide spread adoption any feature like this would need to be part of the native file system options.
patrec 5y ago
A thousand times this. There is absolutely no reason to allow newlines in filenames, and it is pathetic that there isn't even yet a mount option to disallow totally idiotic filenames (at the minimum I don't want programs to create filenames with newlines or invalid utf-8).
- throwawaylinux 5y ago
  It's great that file names in the user/kernel ABI are treated as unstructured NUL terminated byte streams so I can do what I want with my file names even if you don't like it. And you can do what you want with yours, including not creating ones you think are idiotic, or using filesystems with code or options that restrict what names can be used.
  - CorrectHorseBat 5y ago
    But is there any real use case for that? For me I've only encountered this when something else went wrong, I'd rather have an error at that time than later trying to find out what this garbage is and how to remove it. So why not give a mount option for this behavior?
    - throwawaylinux 5y ago
      > But is there any real use case for that? Compatibility, at least. Which is actually a big one and is basically never broken in Linux. > So why not give a mount option for this behavior? Not sure, maybe just nobody yet cared enough to code it up and submit it for inclusion. It's never caused me problems.
      - CorrectHorseBat 5y ago
        Compatibility with what? Is there any software that relies on this behavior?
        throwawaylinux 5y ago
        > Compatibility with what? With existing applications and filesystem images. > Is there any software that relies on this behavior? Possibly.
        CorrectHorseBat 5y ago
        Possibly, so possibly even not. The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior.
        throwawaylinux 5y ago
        > Possibly, so possibly even not. Possibly so. > The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior. Certainly not something like this by default though. To be clear, this wouldn't somehow solve shell / scripting / terminal issues with file names. There are many other special characters and escape sequences and other whitespace like spaces that can trip up incorrectly written programs. These can certainly not all be removed by the kernel so the incremental advantage of just filtering out a couple of such cases doesn't seem like it would be very big.
laumars 5y ago
Thats not a bad suggest per se but it would only work for Linux and thus you still have other POSIX systems that wouldn’t follow suit. So the advice here of not parsing ls is still prudent.
cranekam 5y ago
Why? The name of a file is none of the filesystem’s business. If users choose names that make using software difficult it’s on them. It’s not like there aren’t ways to handle any kind of “weird” character in a file name, as the linked article states. Furthermore, if the kernel/filesystem starts prohibiting certain characters this is more code to maintain and test. User space programs that previously worked fine will stop working. All of this just to prevent someone shootings themselves in the foot by misunderstanding how filenames should be manipulated.
- tomrod 5y ago
  In practical reality, the name of the file is the filesystem's business. It would be nice if it operated similar to cloud filesystems, where you could version based on GUID that is disconnected from the filename, but the practical reality is that users and developers have long accepted the local operational mode.
- laumars 5y ago
  If you’re looking at file system in the purist possible sense, you’d be right. But equally a lot of other meta data is stored beyond the inode that file systems “understand”. And there is already precedence of file systems having code to intelligently handle file names (eg options for case sensitivity). So pragmatically it’s not an unreasonable suggestion to house any code defining legal file names in the file system driver too. The biggest argument against that in my view isn’t down to testing but DRY methodologies: if the code sits in the kernel then it should work against all file systems and not just supported ones.
- charcircuit 5y ago
  Try and name a file "(//^-^//)" You can't because certain characters are prohibited.
- unilynx 5y ago
  I already can’t use NUL and slashes in a file name. And win32 limits me even more. It’s always been a compromise And the amount of feet that have been shot by weird file names is staggering. Programs will stop working, but that’s why we need a bleeding edge distribution to find them. In the short term things will break, in the long term quality will increase. Just like memory protection broke some DOS apps in the short term
jbverschoor 5y ago
Nah.. we need to use object graphs as streams instead of whitespace "(un)parsable" text. The output to the console (ui) or gui (ui) can be different, but the data should be structured
- notreallyserio 5y ago
  Sounds like Powershell to me. I'm down, as long as the syntax is as simple and terse as on UNIX-based systems and not what Microsoft did (were they paid by the character for flag names?)
  - gerdesj 5y ago
    Absolutely. For example: why can't "Get-TrustAuthorityKeyProviderClientCertificateCSR" simply be "takpccc" as $DEITY intended? If your keyboard's tab key is still legibly labelled then you aren't trying hard enough or have an eidetic memory and fast typing skills!
    - notreallyserio 5y ago
      Amusingly HN cuts off the end of the command you typed, I assume using css overflow attributes (don't have an easy way to tell on my UA). I assume it stops at "cate"[0]. I see this sort of chopping a lot, which naturally makes sharing PS commands frustrating -- although there may be workarounds like using `backticks`. 0: Nope, had to paste it to see it ends with "cateCSR".
    - emj 5y ago
      That is basically integrations, there is never going to be nice integrations to my Cobol mainframe linked to a Springboot fuzzbuzz. As is stated in other comments the big issue is usually about being cross platform, and that is a subset of the ls problem: Most of the time you have control over your inputs, until you haven't. This is true for every language even Python which is obnoxious about that. What I mean is that you will always hit edgecases in integrations and you never have time to write new ones. I always felt that powershell was tab unfriendly, the Get- prefix is hard to get used to. I may be wrong that they have a good way to deal with one-off integrations in a sane manner.
ezoe 5y ago
I hope you won't be in the position of handling the non-ascii file names. Whitespaces, symbols and other complicated glyphs are widely used in file name since Windows 95.
ravenstine 5y ago
Or... ls could just escape filenames. I'm not sure why it seemingly doesn't in the year 2021.
- harry8 5y ago
  ls -Q --quoting-style=xxx ?
- dspillett 5y ago
  From “man ls”: -b, --escape print C-style escapes for nongraphic characters I'd also add: -1 list one file per line. Avoid '\n' with -q or -b to make sure you can easily split the list by just the EOL, in case for some reason it thinks it is talking to a terminal and tries for format things for a human.

Tepix 5y ago

It would be almost trivial to create a fuse filesystem which completely hides these files if they exist (and doesn't allow the creation of new ones).

patrec 5y ago
And how would that be even remotely useful? Unless something changed recently, FUSE has so much overhead it's only useful for niche applications and prototyping.
laumars 5y ago
It would be trivial to code but any such wrapper would add overhead to file system operations. FUSE is a fantastic set of APIs (I’ve used it personally) and performs remarkably well considering it is constantly swapping memory between kernel and user space but for wide spread adoption any feature like this would need to be part of the native file system options.

patrec 5y ago

A thousand times this. There is absolutely no reason to allow newlines in filenames, and it is pathetic that there isn't even yet a mount option to disallow totally idiotic filenames (at the minimum I don't want programs to create filenames with newlines or invalid utf-8).

throwawaylinux 5y ago
It's great that file names in the user/kernel ABI are treated as unstructured NUL terminated byte streams so I can do what I want with my file names even if you don't like it. And you can do what you want with yours, including not creating ones you think are idiotic, or using filesystems with code or options that restrict what names can be used.
- CorrectHorseBat 5y ago
  But is there any real use case for that? For me I've only encountered this when something else went wrong, I'd rather have an error at that time than later trying to find out what this garbage is and how to remove it. So why not give a mount option for this behavior?
  - throwawaylinux 5y ago
    > But is there any real use case for that? Compatibility, at least. Which is actually a big one and is basically never broken in Linux. > So why not give a mount option for this behavior? Not sure, maybe just nobody yet cared enough to code it up and submit it for inclusion. It's never caused me problems.
    - CorrectHorseBat 5y ago
      Compatibility with what? Is there any software that relies on this behavior?
      - throwawaylinux 5y ago
        > Compatibility with what? With existing applications and filesystem images. > Is there any software that relies on this behavior? Possibly.
        CorrectHorseBat 5y ago
        Possibly, so possibly even not. The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior.
        throwawaylinux 5y ago
        > Possibly, so possibly even not. Possibly so. > The kernel actually does "break compatibility" from time to time if there's no software relying on the behavior. Certainly not something like this by default though. To be clear, this wouldn't somehow solve shell / scripting / terminal issues with file names. There are many other special characters and escape sequences and other whitespace like spaces that can trip up incorrectly written programs. These can certainly not all be removed by the kernel so the incremental advantage of just filtering out a couple of such cases doesn't seem like it would be very big.

laumars 5y ago

Thats not a bad suggest per se but it would only work for Linux and thus you still have other POSIX systems that wouldn’t follow suit. So the advice here of not parsing ls is still prudent.

cranekam 5y ago

Why? The name of a file is none of the filesystem’s business. If users choose names that make using software difficult it’s on them. It’s not like there aren’t ways to handle any kind of “weird” character in a file name, as the linked article states. Furthermore, if the kernel/filesystem starts prohibiting certain characters this is more code to maintain and test. User space programs that previously worked fine will stop working. All of this just to prevent someone shootings themselves in the foot by misunderstanding how filenames should be manipulated.

tomrod 5y ago
In practical reality, the name of the file is the filesystem's business. It would be nice if it operated similar to cloud filesystems, where you could version based on GUID that is disconnected from the filename, but the practical reality is that users and developers have long accepted the local operational mode.
laumars 5y ago
If you’re looking at file system in the purist possible sense, you’d be right. But equally a lot of other meta data is stored beyond the inode that file systems “understand”. And there is already precedence of file systems having code to intelligently handle file names (eg options for case sensitivity). So pragmatically it’s not an unreasonable suggestion to house any code defining legal file names in the file system driver too. The biggest argument against that in my view isn’t down to testing but DRY methodologies: if the code sits in the kernel then it should work against all file systems and not just supported ones.
charcircuit 5y ago
Try and name a file "(//^-^//)" You can't because certain characters are prohibited.
unilynx 5y ago
I already can’t use NUL and slashes in a file name. And win32 limits me even more. It’s always been a compromise And the amount of feet that have been shot by weird file names is staggering. Programs will stop working, but that’s why we need a bleeding edge distribution to find them. In the short term things will break, in the long term quality will increase. Just like memory protection broke some DOS apps in the short term

jbverschoor 5y ago

Nah.. we need to use object graphs as streams instead of whitespace "(un)parsable" text. The output to the console (ui) or gui (ui) can be different, but the data should be structured

notreallyserio 5y ago
Sounds like Powershell to me. I'm down, as long as the syntax is as simple and terse as on UNIX-based systems and not what Microsoft did (were they paid by the character for flag names?)
- gerdesj 5y ago
  Absolutely. For example: why can't "Get-TrustAuthorityKeyProviderClientCertificateCSR" simply be "takpccc" as $DEITY intended? If your keyboard's tab key is still legibly labelled then you aren't trying hard enough or have an eidetic memory and fast typing skills!
  - notreallyserio 5y ago
    Amusingly HN cuts off the end of the command you typed, I assume using css overflow attributes (don't have an easy way to tell on my UA). I assume it stops at "cate"[0]. I see this sort of chopping a lot, which naturally makes sharing PS commands frustrating -- although there may be workarounds like using `backticks`. 0: Nope, had to paste it to see it ends with "cateCSR".
  - emj 5y ago
    That is basically integrations, there is never going to be nice integrations to my Cobol mainframe linked to a Springboot fuzzbuzz. As is stated in other comments the big issue is usually about being cross platform, and that is a subset of the ls problem: Most of the time you have control over your inputs, until you haven't. This is true for every language even Python which is obnoxious about that. What I mean is that you will always hit edgecases in integrations and you never have time to write new ones. I always felt that powershell was tab unfriendly, the Get- prefix is hard to get used to. I may be wrong that they have a good way to deal with one-off integrations in a sane manner.

ezoe 5y ago

I hope you won't be in the position of handling the non-ascii file names. Whitespaces, symbols and other complicated glyphs are widely used in file name since Windows 95.

ravenstine 5y ago

Or... ls could just escape filenames. I'm not sure why it seemingly doesn't in the year 2021.

harry8 5y ago
ls -Q --quoting-style=xxx ?
dspillett 5y ago
From “man ls”: -b, --escape print C-style escapes for nongraphic characters I'd also add: -1 list one file per line. Avoid '\n' with -q or -b to make sure you can easily split the list by just the EOL, in case for some reason it thinks it is talking to a terminal and tries for format things for a human.