3 ms·
"Bag of code units" would be a better description (if it's correct anyways). This is not very good. You want normalization-preserving/insensitive behavior ins
by cryptonector 9y ago
"Bag of code units" would be a better description (if it's correct anyways).
This is not very good. You want normalization-preserving/insensitive behavior instead.
- klodolph 9y agoWhat about "bag of wchar_t" doesn't preserve normalization? Or am I not sure what you're trying to say? It's no more a "bag of code units" than Linux filesystems store a "bag of code units". Windows will barf back whatever wchar_t array you give it, just like Linux will barf back whatever char array you give it.
- cryptonector 9y agoI'm saying "code unit" is the correct Unicode term for "8-bit bytes" (UTF-8) or "16-bit values" (UTF-16).
- klodolph 9y agoYou're right that "code unit" is the correct Unicode term for parts of encoded Unicode strings, which is exactly why it's the wrong term in this case, because the whole point is the filenames don't have to be Unicode. So I can take the byte string [0xFF, 0xFF, 0xFF] and use that as a filename on Linux, or I can take the sequence of 16-bit values [0xD800, 0xD800, 0xD800] and use that as a filename on NTFS. They're not made of code units because they're not Unicode strings.