4 ms·
Unicode is currently limited to 21 bits for compatibility with UTF-16. Eventually we might manage to exhaust all available codepoint space, and with that we'd h
by TazeTSchnitzel 11y ago
Unicode is currently limited to 21 bits for compatibility with UTF-16. Eventually we might manage to exhaust all available codepoint space, and with that we'd have to move to yet another encoding with a whole new kind of surrogate pairs. Though UTF-8 could originally handle 31 bits, that's no longer the case.
- plugnburn 11y agoSo I see 2 steps here: dropping UTF-16 altogether (well, already, because there are plenty of extended codepoints above 0xFFFF), and when approaching the 31-bit limit - inventing something like "zero-width codepoint joiner" to compose codes of arbitrary length. For example, in a hypothetical alien language, a hypothetical character "rjou" would have a code 0x2300740457 (all the previous codes are exhausted). We can't express this with a single code, so actually we split it into 2-byte parts and write "#" (0x0023), joiner, "t" (0x0074), joiner and "ї" (cyrillic letter yi, 0x0457). As we have a joiner between these codes, we know that we must interpret and display them not as a "#tї" sequence but as a single alien "rjou" character. I think you get the idea.
- TazeTSchnitzel 11y ago> dropping UTF-16 altogether (well, already, because there are plenty of extended codepoints above 0xFFFF) UTF-16 can handle stuff above U+FFFF just fine, it encodes that with surrogate pairs. Are you thinking about UCS-2? The 21-bit limit for Unicode comes from the limits of UTF-16's surrogate pairs.