3 ms·

This is where the wisdom of Unix’s “tools, not policy” approach pays off. Treat file names as opaque byte strings, and don’t assume or impose any particular enc

This is where the wisdom of Unix’s “tools, not policy” approach pays off. Treat file names as opaque byte strings, and don’t assume or impose any particular encoding on them. Let user space programs like your file manager GUI worry about how to resolve encoding details.

eesmith 5y ago
Isn't the issue here that APFS doesn't enforce policy and instead lets the tools resolve encoding details, resulting in differences by the different tools? Most tools expect Form D. Disk Utility doesn't normalize to Form D. Terminal uses a different mechanism. Spotlight doesn't support Form C names. If anything, it seems like an argument against the X design principle of "tools, not policy", as increases the possibility of bugs, with no apparent advantage. The essay links to another essay by the author at https://eclecticlight.co/2021/05/08/explainer-unicode-normalization-and-apfs/ https://eclecticlight.co/2021/05/08/explainer-unicode-normal... , including the observations: > In its original specification, APFS was billed as being non-normalising (the original phrase referred to filenames as just being a ‘bag of bytes’). ... > When APFS was first released, chaos ensued, not so much with languages based on Roman alphabets, but worst of all in Korean, where there are a lot more normalisations. ... > More likely and more serious are the conflicts which can occur when using different methods of accessing non-Mac file systems. Thomas Tempelmann has demonstrated this using a share on a NAS, which he mounted first using NFS, then created a file with a name in Form C. When mounted via SMB on a Mac, as that filename is un-normalised, it can’t be accessed, as he has described here.
- 10000truths 5y ago
  > Isn't the issue here that APFS doesn't enforce policy and instead lets the tools resolve encoding details, resulting in differences by the different tools? The issue isn’t that APFS doesn’t enforce policy, per se. It’s that combined with the fact that its predecessor (HFS+) did, and the transition wreaked havoc on programs that came to rely on file names always being normalized Unicode.
  - eesmith 5y ago
    When the file system doesn't normalize names, then wouldn't the problem be worse? That is, wouldn't different tools be built with different normalizations (or none at all)? Because that's my experience from Unix-based systems where some programs generate Latin-1 filenames and others UTF-8 names. And that doesn't even touch normalizations. What would your tool-not-policy approach be?

eesmith 5y ago

Isn't the issue here that APFS doesn't enforce policy and instead lets the tools resolve encoding details, resulting in differences by the different tools? Most tools expect Form D. Disk Utility doesn't normalize to Form D. Terminal uses a different mechanism. Spotlight doesn't support Form C names. If anything, it seems like an argument against the X design principle of "tools, not policy", as increases the possibility of bugs, with no apparent advantage. The essay links to another essay by the author at https://eclecticlight.co/2021/05/08/explainer-unicode-normalization-and-apfs/ https://eclecticlight.co/2021/05/08/explainer-unicode-normal... , including the observations: > In its original specification, APFS was billed as being non-normalising (the original phrase referred to filenames as just being a ‘bag of bytes’). ... > When APFS was first released, chaos ensued, not so much with languages based on Roman alphabets, but worst of all in Korean, where there are a lot more normalisations. ... > More likely and more serious are the conflicts which can occur when using different methods of accessing non-Mac file systems. Thomas Tempelmann has demonstrated this using a share on a NAS, which he mounted first using NFS, then created a file with a name in Form C. When mounted via SMB on a Mac, as that filename is un-normalised, it can’t be accessed, as he has described here.

10000truths 5y ago
> Isn't the issue here that APFS doesn't enforce policy and instead lets the tools resolve encoding details, resulting in differences by the different tools? The issue isn’t that APFS doesn’t enforce policy, per se. It’s that combined with the fact that its predecessor (HFS+) did, and the transition wreaked havoc on programs that came to rely on file names always being normalized Unicode.
- eesmith 5y ago
  When the file system doesn't normalize names, then wouldn't the problem be worse? That is, wouldn't different tools be built with different normalizations (or none at all)? Because that's my experience from Unix-based systems where some programs generate Latin-1 filenames and others UTF-8 names. And that doesn't even touch normalizations. What would your tool-not-policy approach be?