4 ms·
Another example of UTF-16 brokenness: it is byte-order dependent, that is, programmer must care about big-endian vs little-endian encoding of 16-bit numbers. Th
by dmytrish 13y ago
Another example of UTF-16 brokenness: it is byte-order dependent, that is, programmer must care about big-endian vs little-endian encoding of 16-bit numbers. This caused another hack: using non-text BOM (Byte-Order Mark) symbols to denote what endianness is used.
I think it's very ugly and may be ok for Windows-x86 worlds, but not acceptable for variety of platforms connected to the Internet.
UTF-8 avoids endianness problems completely.
- asveikau 13y agoA legit complaint. Though I'll say, in early Unicode before UTF-8 existed, this was probably seen as less of a problem than using the older non-Unicode charsets, and probably rightly so. Also, if you keep a clean separation between serialization and in-memory representation, the BOM hack is not such a big deal. Serialize code can write a BOM, and de-serialize can do the char swapping and omit the BOM in RAM, then in-memory it's always host byte order and without BOM. Or you can just have the on-disk/over-the-wire format be UTF-8, which is common today on platforms where UTF-16 is used in RAM. (I'll repeat a point that seems to be lost on a lot of people in this thread, that Microsoft adopted 16-bit chars for NT before UTF-8 existed, so that was not an option.)