4 ms·
NTFS, by comparison, does the "bag of wchar_t" instead of the "bag of byte" for filenames.
by klodolph 9y ago
NTFS, by comparison, does the "bag of wchar_t" instead of the "bag of byte" for filenames.
- rbanffy 9y ago> NTFS, by comparison, does the "bag of wchar_t" I threw up a little.
- aidenn0 9y agoThis is largely due to microsoft being an early adopter of unicode. In the 90s, unicode was limited to 16 bits, so switching to fixed-length 16-bit characters made a lot of sense. Of course, it didn't take long for that to change, but MS had already put a lot of resources into switching to UCS-2.
- dietrichepp 9y agoThis timeline also meant that wchar_t was fixed as 16 bits on Windows for compatibility, but ended up as 32 bits on macOS and Unix. It can't really be removed from the C or C++ standards, but there's nothing you can do with wchar_t that's both useful and portable. Well-meaning developers learn how to support Unicode with wchar_t only to discover that wchar_t is the worst way to support Unicode. https://www.moria.us/articles/wchar-is-a-historical-accident/ https://www.moria.us/articles/wchar-is-a-historical-accident...
- rspeer 9y agoIt doesn't help that many developers are introduced to the basics of Unicode by a popular Joel on Software article from 2003, which recommended the use of wchar_t. [1] It was a very important article for 2003, but it should be honorably retired. Its plan of action isn't as good as the modern "UTF-8 Everywhere", and even its motivating examples are becoming less relevant: when was the last time you went to the 'Encoding' menu in your web browser and guessed which codepage the page author meant to use? When was the last time you worried about whether your e-mail was 7-bit clean? [1] https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/ https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...
- cryptonector 9y agoHFS+'s use of NFD also comes from it being an early adopter of Unicode. When I argued strenuously for n-p/n-i behavior in ZFS (see elsewhere in this thread) I was told that gee, Unicode doesn't describe n-i string comparison the way I was proposing, so we can't do it :( However, fast n-i does follow from the spec, so eventually it got done. I can't take credit for the code, though I did contribute an optimization.
- cryptonector 9y ago"Bag of code units" would be a better description (if it's correct anyways). This is not very good. You want normalization-preserving/insensitive behavior instead.
- klodolph 9y agoWhat about "bag of wchar_t" doesn't preserve normalization? Or am I not sure what you're trying to say? It's no more a "bag of code units" than Linux filesystems store a "bag of code units". Windows will barf back whatever wchar_t array you give it, just like Linux will barf back whatever char array you give it.
- cryptonector 9y agoI'm saying "code unit" is the correct Unicode term for "8-bit bytes" (UTF-8) or "16-bit values" (UTF-16).
- klodolph 9y agoYou're right that "code unit" is the correct Unicode term for parts of encoded Unicode strings, which is exactly why it's the wrong term in this case, because the whole point is the filenames don't have to be Unicode. So I can take the byte string [0xFF, 0xFF, 0xFF] and use that as a filename on Linux, or I can take the sequence of 16-bit values [0xD800, 0xD800, 0xD800] and use that as a filename on NTFS. They're not made of code units because they're not Unicode strings.