I think what they did not realize is that once a universal encoding won the battle of standards, it would have to be all-inclusive. Operating systems and platfo

I think what they did not realize is that once a universal encoding won the battle of standards, it would have to be all-inclusive. Operating systems and platforms like Java settled on Unicode so they won't have to deal with a myriad of different encodings, but the moment you support only one encoding, you'll eventually have to accommodate everyone, and it's not enough to include all commonly used characters in the world's major languages. You'll have to make a go at the long tail. You can't leave out people who need to encode rare languages on computers underserved, so for all the scholars recording Egyptian and Luwian hieroglyphs, Sumerian and Akkadian cuneiform and Tangut script. You'd probably have to eventually support as-of-yet unencoded scripts such as Classical Yi, Jurchen, Aztec hieroglyphs and so on. You'd have to add alphabets and syllabaries for small minority languages such as Lisu, Coptic, Cherokee and Vai. And you'd have to support the rather widely used writing systems of India, each with its own 128 characters. Combine this with the over 28,000 CJK characters Unicode did end up allocating, the 11,184 discrete codepoints they Unicode did end up adding for Koran Hangul [1], you've already used 40,000 characters. Add all the long-tail scripts and you're past 65,535 well before we even reach Emoji or extremely rare CJKV characters from Mojikyo[1]. "In other words, given that the limitation to 65,536 character codes genuinely does satisfy all the world's modern communication needs with a safety factor of about four, then one can decide up-front that preserving a pure 16-bit architecture has a higher design priority than publicly encoding every extinct or obscure character form. Then the sufficiency of 16 bits for the writing technology of the future becomes a matter of our active intention, rather than passive victimization by writing systems of the past." The engineers behind Unicode made the classic software engineering mistake of looking at their needs today and taking a factor of safety that's way too small. It's the same thing that happened with the IPv4 address range. A simple back-of-the-envelope calculation would have told us it's not going to be enough, but it was easy to forget back in the 1980s that one day children in Africa would also have personal computers connected to the grand Internet, that would need an IP address. The fact that by version 1.0.1 (the first version of Unicode to include the CJK characters), they already used almost half of their space when they were planning to use less than a quarter, should have already prepared them to the future. No, the reasoning behind 16 bits was always crazy. You can't design future-proof standards with a low safety factor and condescending estimation of usage (Yi Syllabary? Who would ever use that?). But we should probably be happy about this. A 32-bit encoding would have certainly been rejected by just about everyone as too wasteful, and software engineers back in the 80's and 90's were still unable to give up merging their understanding of codepoints with glyhs, so a multi-byte encoding (which is what we ended up having eventually with UTF-8 and UTF-16) wasn't a popular idea either. Fortunately enough, Unicode is still much better than the mess it preceded, and the fix for the 16-bit limitation, as painful as it was, has proven far easier than IPv6. [1] a.k.a The "Korean Mess": http://unicode.org/pipermail/unicode/2015-June/002038.html http://unicode.org/pipermail/unicode/2015-June/002038.html [2] https://en.wikipedia.org/wiki/Mojikyo https://en.wikipedia.org/wiki/Mojikyo