4 ms·

Are these two names the same case insensitively? Pick and PICK. Hint: https://en.wikipedia.org/wiki/Dotted_and_dotless_I https://en.wikipedia.org/wiki/Dotted_

Are these two names the same case insensitively? Pick and PICK. Hint: https://en.wikipedia.org/wiki/Dotted_and_dotless_I https://en.wikipedia.org/wiki/Dotted_and_dotless_I How about Straße, STRASSE and strasse? For an American audience, how about Résumé and RESUME and resume? How about Ⅸ and IX? Hint: https://en.wikipedia.org/wiki/Unicode_equivalence#Normalization https://en.wikipedia.org/wiki/Unicode_equivalence#Normalizat... The answer to all that is that it depends on the user and their locale settings. So to get this right you would have to pass the locale settings into system calls that go into the filesystem so it can do the right thing on every call. But then what happens if the locale later gets changed? Before some file names may be considered equivalent, but with the new one they are different and things break. Ok how about a system wide locale setting that can't be changed? Are you going to require a disk reformat each time a different user is on the system, or a multilingual user who travels? The right solution is exactly what Unix did in the first place. The filesystem stores names exactly as is without having to deal with locales. User space is then responsible for doing the right thing for a user. For example a word processor can treat Résumé and Resume as the same for Americans in file selection dialogs. User space already has to deal with all this, for example by sorting file names. Here is how hard that is http://www.unicode.org/reports/tr10/ http://www.unicode.org/reports/tr10/ and a simple example is if z comes before or after ö. Or when showing a list of Swedish names to a German user do you use Swedish or German sort order? A similar analogy is dealing with time. The best approach is to store the time as UTC in the filesystem and let user space show it in whatever way is relevant for the user. On the other hand if you store local time in the filesystem, you will end up in a world of hurt trying to work around that. (UTC to local time is a lossy conversion making local time back to UTC hard to work out.)

archagon 12y ago
But half the time, I'm having to deal with file access and creation in the terminal. That's as an important user experience to me (a power user) as anything in the GUI. Does Unix change the way the terminal handles files depending on the locale? My understanding is that it does not. (If it did, maybe that would be the best solution for everyone, but I imagine a lot of geeks would be up in arms about how the terminal experience is not consistent...)
- woodman 12y ago
  The terminal is in userland right? So yeah, locale makes a difference based one how the terminal software is designed to handle it.
- rogerbinns 12y ago
  There are multiple layers at play. For example someone looking a file selection dialog in a gui has at least these layers, lowest level first: - Filesystem implementation (actual on disk bytes) - Filesystem compatibility (adjustments to OS semantics) - Generic filesystem layer in the kernel - System call interface (between userland and kernel) - Userland low level library (eg system call wrappers) - Higher level libraries (eg libc) - User interface libraries - App level modules & libraries The problem with the filesystem being case insensitive is that is the first level above, where there is virtually no context or user information. That is the hardest place to work out if the user considers Résumé and RESUME to be the same. The lower down the list the easier it gets. The important point is that how you store things and how you present does not need to be the same. Command line tools typically use the libc level of things. It is perfectly possible to make that level so the system appears case insensitive (another poster mentioned Windows doing that). Your Unix command line experience is very much affected by your locale. For example the ls command does the following: - Sorts filenames - Displays sizes (different locales have different conventions for floating point representation, digit grouping etc) - Displays times Environment variables like $LANG, $LC_*, $TZ etc control this in Unix. Individual programs can also alter behaviour. For example bash can be configured to do case insensitive filename completion. TLDR: while case insensitivity is a nice user experience for humans, it is awful for the lower layers including filesystems. The closer a program is to a human, the easier it is to give that nice user experience. There is no requirement for a case insensitive user experience to then make filesystems also implement that. And as my examples show, they couldn't do it correctly anyway.
  - woodman 12y ago
    Don't forget the additional fun that terminal emulators add in the journey from libc to your eyeballs. Some of the most difficult bugs I've worked on involved grep with the color flag and automatic build systems...

archagon 12y ago

But half the time, I'm having to deal with file access and creation in the terminal. That's as an important user experience to me (a power user) as anything in the GUI. Does Unix change the way the terminal handles files depending on the locale? My understanding is that it does not. (If it did, maybe that would be the best solution for everyone, but I imagine a lot of geeks would be up in arms about how the terminal experience is not consistent...)

woodman 12y ago
The terminal is in userland right? So yeah, locale makes a difference based one how the terminal software is designed to handle it.
rogerbinns 12y ago
There are multiple layers at play. For example someone looking a file selection dialog in a gui has at least these layers, lowest level first: - Filesystem implementation (actual on disk bytes) - Filesystem compatibility (adjustments to OS semantics) - Generic filesystem layer in the kernel - System call interface (between userland and kernel) - Userland low level library (eg system call wrappers) - Higher level libraries (eg libc) - User interface libraries - App level modules & libraries The problem with the filesystem being case insensitive is that is the first level above, where there is virtually no context or user information. That is the hardest place to work out if the user considers Résumé and RESUME to be the same. The lower down the list the easier it gets. The important point is that how you store things and how you present does not need to be the same. Command line tools typically use the libc level of things. It is perfectly possible to make that level so the system appears case insensitive (another poster mentioned Windows doing that). Your Unix command line experience is very much affected by your locale. For example the ls command does the following: - Sorts filenames - Displays sizes (different locales have different conventions for floating point representation, digit grouping etc) - Displays times Environment variables like $LANG, $LC_*, $TZ etc control this in Unix. Individual programs can also alter behaviour. For example bash can be configured to do case insensitive filename completion. TLDR: while case insensitivity is a nice user experience for humans, it is awful for the lower layers including filesystems. The closer a program is to a human, the easier it is to give that nice user experience. There is no requirement for a case insensitive user experience to then make filesystems also implement that. And as my examples show, they couldn't do it correctly anyway.
- woodman 12y ago
  Don't forget the additional fun that terminal emulators add in the journey from libc to your eyeballs. Some of the most difficult bugs I've worked on involved grep with the color flag and automatic build systems...