3 ms·
Many Kanji and Chinese glyphs are distinct. There are also many variants of Chinese, among them simplified and traditional. They don't have distinct Unicode co
by wffurr 2y ago
Many Kanji and Chinese glyphs are distinct. There are also many variants of Chinese, among them simplified and traditional. They don't have distinct Unicode codepoints due to Han Unification which tried to cram them all into UTF-16.
You have to know the intended locale or the text to disambiguate and select the correct glyphs.
- Onavo 2y agoWell, the world is on UTF-8 now, they can extend the standard and put the Japanese characters there. There's plenty of space since it's variable length.
- wffurr 2y agoSure but "they" can't undo the billions of already authored documents and the thousands of overlapping fonts.
- hnfong 2y agoIt's actually what they did. I heard from people involved in the CJK-unification project that the Japanese representatives insisted this be done as a condition for their participation. Given that there were thousands of such characters, some characters slipped through and the same code point was used by both Chinese and Japanese. The Japanese have been bickering about the handful of those cases ever since.
- syncsynchalt 2y agoWeirdly, there _are_ separate codepoints for simplified vs traditional chinese characters, cf. 丟 (traditional) vs 丢 (simplified) [1]. This contributes to the annoyance of Japanese users, it feels like the unicode consortium went out of their way to mangle only their language. [1] One major reason is that pre-Unicode charsets exist which encode both traditional and simplified characters, and one of the primary goals of Unicode is to support round-trip mapping from any charset into Unicode and back without loss of information.