3 ms·

I get the problem, but don't you need canonical equivalence to make sure you don't allow files with filenames that look the same but which happen to have a diff

I get the problem, but don't you need canonical equivalence to make sure you don't allow files with filenames that look the same but which happen to have a different unicode byte sequence? Or is the problem specifically with NFC? I don't see how byte equality could be preserved the other normalisations.

restalis 12y ago
With or without Unicode's own meddling, characters may still look the same. Think of non-Latin alphabets that have letters defined in an identical fashion to those in Latin alphabet. Heck, even without other alphabets, the Latin letter O and the digit 0 are valid characters for filenames in any system I know of, and although looking the same they are yet different. (Having a bar, a point, or some other distinctive mark inside 0 is just a hack, the common description for the aspect of this digit makes no distinction from letter O.) This problem is not new, definitely not brought up by this Unicode thing.
- MrBuddyCasino 12y ago
  True, we have seen this issue with fraudulent domain names, where they can pose a security issue. To me it seems normalisation at least reduces the attack surface, while not solving the issue completely. I am not qualified to judge whether or not the complexity increase is worth it. But what are the downsides - what is so bad about NFC?
ploxiln 12y ago
Well in my case, I don't have to worry about filename spoofing, or even filenames being too similar, for my own directory of records/archives. The strangeness of a normalizing filesystem, is that you can try to open a filename (with accent, NFC), and it works. Then, a readdir() doesn't list that exact filename byte sequence (you happen to get one that's extremely similar, but NFD). On linux it always does, there's no normalization, exactly what you open is exactly what's listed. This really is an awefully nice simplification, for this layer of abstraction. But you could argue that I'm just accustomed to the tradeoffs of it by this point.

restalis 12y ago

With or without Unicode's own meddling, characters may still look the same. Think of non-Latin alphabets that have letters defined in an identical fashion to those in Latin alphabet. Heck, even without other alphabets, the Latin letter O and the digit 0 are valid characters for filenames in any system I know of, and although looking the same they are yet different. (Having a bar, a point, or some other distinctive mark inside 0 is just a hack, the common description for the aspect of this digit makes no distinction from letter O.) This problem is not new, definitely not brought up by this Unicode thing.

MrBuddyCasino 12y ago
True, we have seen this issue with fraudulent domain names, where they can pose a security issue. To me it seems normalisation at least reduces the attack surface, while not solving the issue completely. I am not qualified to judge whether or not the complexity increase is worth it. But what are the downsides - what is so bad about NFC?

ploxiln 12y ago

Well in my case, I don't have to worry about filename spoofing, or even filenames being too similar, for my own directory of records/archives. The strangeness of a normalizing filesystem, is that you can try to open a filename (with accent, NFC), and it works. Then, a readdir() doesn't list that exact filename byte sequence (you happen to get one that's extremely similar, but NFD). On linux it always does, there's no normalization, exactly what you open is exactly what's listed. This really is an awefully nice simplification, for this layer of abstraction. But you could argue that I'm just accustomed to the tradeoffs of it by this point.