3 ms·
UTF16 was an extension of the original "obvious" Unicode encoding, back when Unicode started and was defined as fitting the world's languages into a 16-bit spec
by chungy 5y ago
UTF16 was an extension of the original "obvious" Unicode encoding, back when Unicode started and was defined as fitting the world's languages into a 16-bit spec (what is now the Basic Multilingual Plane). UTF16 allowed (most) older UCS2 documents to be upgraded for free to UTF16, much the way that ASCII documents are also valid UTF8.
Some of the weirdness in the Unicode spec even comes from the need for backwards compatibility. 17 planes and 1,114,078 total usable codepoints... these numbers would not have been arrived if the system had the foresight that 16-bit wasn't enough in the first place. They were derived from reassigning private use area codepoints into surrogate pairs for UTF16. Unicode would probably have rather (maybe should have) started out as a 32-bit spec and avoided this mess from the get-go.
- Dylan16807 5y agoWell, probably not 32 exactly. They had a 31 bit spec at the time but decided they could simplify to 16 bits. Also UTF-8 is naturally optimized for 31 bits.
- CountSessine 5y agoYou know I’d love to read a book about the Unicode standardization process. Unicode and the “16-bits is enough for all modern scripts” business must be one of the biggest failures of requirements analysis in computing history. And the fact that at the same time they were upsetting so many of their partners with Asian unification to me means there must have been a real issue with personalities and project responsibilities.