4 ms·

I think the suggested alternative though is having UTF-8 take care of this. UTF-8 already has different codepoints for every other language, like I can type ru

I think the suggested alternative though is having UTF-8 take care of this. UTF-8 already has different codepoints for every other language, like I can type russian (День), greek (Ημέρα), english, even egyption hieroglyphs (𓀃) all of them in this one text field, and they all render right for both of us if we have a suitable font. If they don't render, it's unambiguous. It's only chinese and japanese where I can type some characters, and depending on if you have a chinese or japanese font _first_ in your system, it might render wrong, and either way it's ambiguous to the computer without further metadata. The computer doesn't need metadata to know that "α" is "α", why does it need metadata to know if 直 is the chinese (http://www.hanzi5.com/bishun/76f4.html http://www.hanzi5.com/bishun/76f4.html) or japanese (https://kanjivg.tagaini.net/viewer.html?kanji=%E7%9B%B4 https://kanjivg.tagaini.net/viewer.html?kanji=%E7%9B%B4) character, two different looking characters that share the same codepoint.

yorwba 4y ago
Unicode does not have different codepoints for every language. Leaving aside scripts that aren't encoded at all, all languages using the Latin script get a single set of glyphs, and if you want Comic Sans MS or Carolingian miniscule or fine-grained control over whether "a" has a hook at the top or not, you need to specify a font that renders it the way you want, just like with 直. Because Unicode is about the meaning of characters, not their appearance. If there were a context where the two different shapes of 直 have different meaning, Unicode would add a new codepoint to distinguish them. In fact "ɑ" without hook does have a separate codepoint because it is used in linguistics to mark a vowel different from "a".
- lozenge 4y ago
  If CYRILLIC SMALL LETTER A deserves a code point despite being so similar to LATIN SMALL LETTER A, why doesn't a Japanese character that actually looks different from a Chinese one get a codepoint? And the idea that something can have the "same meaning" across two languages is very wooly.
  - astrange 4y ago
    Why don't Japanese characters in seal script or handwriting get different codepoints from mincho font characters? They look different too. Sometimes looking visually different doesn't matter because if you know how to write them by stroke order, you'll still be able to read it (which is how handwriting works, I think; I'm pretty bad at reading that…) Hanzi simplification in the Mainland also complicated things, since I doubt they wanted to make all of those into different characters.

yorwba 4y ago

Unicode does not have different codepoints for every language. Leaving aside scripts that aren't encoded at all, all languages using the Latin script get a single set of glyphs, and if you want Comic Sans MS or Carolingian miniscule or fine-grained control over whether "a" has a hook at the top or not, you need to specify a font that renders it the way you want, just like with 直. Because Unicode is about the meaning of characters, not their appearance. If there were a context where the two different shapes of 直 have different meaning, Unicode would add a new codepoint to distinguish them. In fact "ɑ" without hook does have a separate codepoint because it is used in linguistics to mark a vowel different from "a".

lozenge 4y ago
If CYRILLIC SMALL LETTER A deserves a code point despite being so similar to LATIN SMALL LETTER A, why doesn't a Japanese character that actually looks different from a Chinese one get a codepoint? And the idea that something can have the "same meaning" across two languages is very wooly.
- astrange 4y ago
  Why don't Japanese characters in seal script or handwriting get different codepoints from mincho font characters? They look different too. Sometimes looking visually different doesn't matter because if you know how to write them by stroke order, you'll still be able to read it (which is how handwriting works, I think; I'm pretty bad at reading that…) Hanzi simplification in the Mainland also complicated things, since I doubt they wanted to make all of those into different characters.