3 ms·
That kind of problem is much rarer, because Unicode has separate codepoints for traditional/simplified Chinese.
by lmm 3y ago
That kind of problem is much rarer, because Unicode has separate codepoints for traditional/simplified Chinese.
- klausa 3y agoIf only the article you're replying to had some example of characters that use the same code points for the same characters in Traditional and Simplified Chinese.
- lmm 3y agoThe codepoint mentioned in the article, U+5203, is a perfect example: in most of the world it will display in Simplified Chinese, whereas there's a counterpart U+2F81E that (in practice if not in theory) can be used to reliably display in Traditional Chinese. Only the Japanese rendering doesn't have a codepoint and is forced to use out-of-band methods of indicating the desired appearance.
- adastra22 3y agoIMEs do not and should not select U+2F81E, however. That core point exists merely for obscure Unicode compatibility requirements.
- lmm 3y agoThat's the Unicode consortium's opinion. Many people believe they're wrong about semantics and that Han unification was a mistake. Regardless of the Unicode consortium's intentions, these compatibility codepoints allow one to write text that displays Traditional Chinese correctly without additional out-of-band information - but there is sadly no equivalent for Japanese.
- adastra22 3y agoExcept then you get those characters rendered as little squares for Unknown Char on a wide variety of platforms. There’s good reason to stick to the basic multilingual plane.
- lmm 3y ago> you get those characters rendered as little squares for Unknown Char on a wide variety of platforms. That can happen in the BMP too. It's a well-known problem with well-known solutions. (E.g. you can ship fonts that support all the codepoints you use).
- rfoo 3y agoIt isn't. The problem used to be default installed macOS with language set to English put Japanese first (as in CJK) in the language preferences list so they display Simplified Chinese / Traditional Chinese without lang tag wrong. It looks like now Apple flipped the default to Simplified Chinese first maybe.
- lifthrasiir 3y agoOnly for some, because some source character sets had separate codes for them and Unicode had to guarantee a safe round-trip conversion ("source separation principle").
- j16sdiz 3y agoMost of them. Some have "minor" different with the same codepoint.
- adastra22 3y agoIt’s called Unihan for a reason. Using different code points is the exception, not the rule.