4 ms·
Old versions use UCS-2. New versions use UTF-16. (correctness not enforced) This is also how Java and OS X were updated. The kernel generally uses the 16-bit e
by burfog 9y ago
Old versions use UCS-2. New versions use UTF-16. (correctness not enforced) This is also how Java and OS X were updated.
The kernel generally uses the 16-bit equivalent of the old Pascal string, that being a 16-bit count of 16-bit pieces of UTF-16 data. This allows a 16-bit NUL to get into various places that make the Win32 API choke.
- masklinn 9y ago> Old versions use UCS-2. New versions use UTF-16. (correctness not enforced) This is also how Java and OS X were updated. I usually call that ucs2-plus-surrogates, to make it clear that you may encounter unpaired surrogates, and thus invalid paths if you assume proper UTF-16.
- leeter 9y agoThis is actually kinda painful with NTFS as NTFS doesn't really care what's in a path other than the directory separators it's all binary. This means that different applications using different Unicode normalization will result in odd things happening. To me the right answer is NTFS should normalize all paths the same way internally but they have yet to implement it because it would break legacy systems that have un-normalized paths (I assume).
- freeone3000 9y agoPrecisely. Programs that internally use ShiftJIS, for instance, would stop functioning on UTF-16 enforced compatability or normalization. They're currently "broken" (as in, operating incorrectly) but in a way that works.
- leeter 9y agoCan you explain why this would be the case? In theory this shouldn't be an issue because any ShiftJIS conversion to Unicode should be reversible.
- TazeTSchnitzel 9y agoI think it's not a perfect round-trip due to differences in how the two standards encode certain characters. You will get a correct conversion either way, but the result of a round-trip might not be bit-identical.
- rspeer 9y agoIt should be reversible, sure. Just let me know when you find a standardized, correct, one-to-one mapping between JIS codepoints and Unicode codepoints. Also make sure it hasn't changed in the lifetime of the oldest software that anyone is using.
- freeone3000 9y agoThe apps I'm thinking of write bytes, not strings. If you enforced UTF-16 compatibility, they'd have to say what encoding they're using (they don't) or convert it themselves (they don't) - and changing either of these would require at least an application recompile. The reason they currently work is because bytes out == bytes in, so they can read the files they create, despite what mojibake the user sees.