2 ms·

Honestly? Convince operating systems to have a switch that enforces UTF-8 path names, and then convince distros to flip that switch by default for new installs.

Honestly? Convince operating systems to have a switch that enforces UTF-8 path names, and then convince distros to flip that switch by default for new installs. That is to say, we need to move the world from a probably to a definitely state. The reality is that file names are "stringy" in nature--people expect to be able to do display them--and that means you need to have some up-front agreement on how to interpret those strings. In practice, on Unix systems, everyone has generally agreed that this is UTF-8, to the point that trying to not be UTF-8 generally causes interesting breaks in the system. It would be great if we could actually get the operating system to help enforce these rules, rather than placing the blame on other software for not correctly handling situations where the correct solution is itself incredibly ambiguous.

Joker_vD 4y ago
While I am all for an OS monoculture, it's still not there yet... and besides, my idea of what OS precisely must be the sole survivor is different from yours, and yours is different from somebody else's, etc. So I am afraid we can either a) indulge ourselves in wishful thinking, b) actively try to extinguish platforms that don't match our ideals, c) make an effort to be actually cross-platform, and not in "let's just build a tiny Linux model in a bottle for us to use and pretend the rest of the environment is not there" kind.
jefftk 4y ago
On a system with that switch flipped, what should happen if you plug in a USB drive or untar an archive that has non-utf8 filenames?
- deathanatos 4y ago
  For a legacy/broken FS that permits non-utf8 filenames, and has them: have the FS driver map them into UTF-8 as best it can by making some sort of compromise. E.g., use the PUA to map malformed sequences in/out. For untar'ing a tar archive: error out by default, but provide a flag or option to permit untarring using some sort of escaping to map the malformed names back into Unicode. I think here I'd map to something printable, though, like "\xnn" or something.
tpolzer 4y ago
You can use ZFS as a root file system, and it actually has such a switch (called "utf8only").

Joker_vD 4y ago

While I am all for an OS monoculture, it's still not there yet... and besides, my idea of what OS precisely must be the sole survivor is different from yours, and yours is different from somebody else's, etc. So I am afraid we can either a) indulge ourselves in wishful thinking, b) actively try to extinguish platforms that don't match our ideals, c) make an effort to be actually cross-platform, and not in "let's just build a tiny Linux model in a bottle for us to use and pretend the rest of the environment is not there" kind.

jefftk 4y ago

On a system with that switch flipped, what should happen if you plug in a USB drive or untar an archive that has non-utf8 filenames?

deathanatos 4y ago
For a legacy/broken FS that permits non-utf8 filenames, and has them: have the FS driver map them into UTF-8 as best it can by making some sort of compromise. E.g., use the PUA to map malformed sequences in/out. For untar'ing a tar archive: error out by default, but provide a flag or option to permit untarring using some sort of escaping to map the malformed names back into Unicode. I think here I'd map to something printable, though, like "\xnn" or something.

tpolzer 4y ago

You can use ZFS as a root file system, and it actually has such a switch (called "utf8only").