3 ms·

Aside from all of the other issues mentioned, for some languages it's not clear what language something is purely based on codepoints. For languages that have

by cyphar 5y ago

Aside from all of the other issues mentioned, for some languages it's not clear what language something is purely based on codepoints. For languages that have Latin-derived writing systems, it's not uncommon to use English letters (without diacritics) to write the language -- how would that be handled? In addition, thanks to Han unification (though this would've been a problem anyway -- loads of characters would've been unified anyway) all similar CJK Hanzi/Kanji/漢字 characters are mapped to the same codepoint regardless of language. This means that for some sentences it is entirely possible for you to not know whether a sentence fragment is Chinese or Japanese without more context and a native-like understanding of the language. Also in many languages English words are written verbatim meaning that you can have sentences like (my Japanese is not perfect, this is just an example): > あの芸人のYouTube動画を見たの？面白すぎるww And (at least in Japanese) there are loads of other usage of Latin characters aside from loan words that would be too unwieldy to write in katakana -- "w" is like "lol", and fair few acronyms (BGM = Background Music, CM = Advertisement, TKG = 卵かけご飯 = (Raw) Egg on Rice). There are other languages that have similar "issues", but unfortunately I can't give any more examples because the only other language I speak (Serbian) writes everything (even people's names) phonetically. As an aside -- if anyone ever has to support CJK languages (in subtitles for instance), please make sure you use the right fonts. While Unicode has encoded Han characters with the same codepoint, in different languages the characters are often drawn differently and you need to use the right font for the corresponding language (and area -- Mandarin speakers in different areas write some characters differently -- 返 is different in every CJK locale). Many media players and websites do not handle this correctly and it is fairly frustrating -- the net result is that Japanese is often displayed using Chinese fonts which makes it uncomfortable to read (it's still obvious what the character is, it's just off-putting).

kingcharles 5y ago
Yeah, for the same reason that CJK designers often use absolutely abhorrent fonts for English words on packaging and printed media, is the same reason Westerners use terrible fonts for CJK. They are working with a language and culture they have no knowledge of, and think that if the characters look sorta right, then they must be readable and look good. I've made so many horrible localization errors in the past because I had to translate things into 30 different languages and I can only barely read a handful of them, so I just copy and paste whatever the translators give me. Incidently, I saw Japanese text recently that was quoting both English AND Arabic in the same sentence. And this was in a block of vertical text. That is literally a worst-case scenario I think. You have RTL-vertical text also containing RTL- and LTR-horizontal text. And unlike English, which when placed into vertical Japanese text, you can essentially choose whether you want the characters going vertically or sideways - you can't do that with Arabic as the letters must be joined together, I don't believe you can break them down and stack them on top of each other. Why can't we all just convert to Esperanto? ;)
- divingdragon 5y ago
  > Incidently, I saw Japanese text recently that was quoting both English AND Arabic in the same sentence. And this was in a block of vertical text. That is literally a worst-case scenario I think. Look up Mongolian script and you might change your mind :)
  - kingcharles 5y ago
    Oh Lord. I just looked it up on Wikipedia. It's a vertical-only language for a start. How do you even discuss a vertical-only language in a language which is horizontal? Wikipedia has to write all the script horizontally, which would be like a page talking about English stacking all the letters vertically - it's weird. From Wikipedia: "Computer operating systems have been slow to adopt support for the Mongolian script, and almost all have incomplete support or other text rendering difficulties."

by cyphar 5y ago

kingcharles 5y ago

Yeah, for the same reason that CJK designers often use absolutely abhorrent fonts for English words on packaging and printed media, is the same reason Westerners use terrible fonts for CJK. They are working with a language and culture they have no knowledge of, and think that if the characters look sorta right, then they must be readable and look good. I've made so many horrible localization errors in the past because I had to translate things into 30 different languages and I can only barely read a handful of them, so I just copy and paste whatever the translators give me. Incidently, I saw Japanese text recently that was quoting both English AND Arabic in the same sentence. And this was in a block of vertical text. That is literally a worst-case scenario I think. You have RTL-vertical text also containing RTL- and LTR-horizontal text. And unlike English, which when placed into vertical Japanese text, you can essentially choose whether you want the characters going vertically or sideways - you can't do that with Arabic as the letters must be joined together, I don't believe you can break them down and stack them on top of each other. Why can't we all just convert to Esperanto? ;)

divingdragon 5y ago
> Incidently, I saw Japanese text recently that was quoting both English AND Arabic in the same sentence. And this was in a block of vertical text. That is literally a worst-case scenario I think. Look up Mongolian script and you might change your mind :)
- kingcharles 5y ago
  Oh Lord. I just looked it up on Wikipedia. It's a vertical-only language for a start. How do you even discuss a vertical-only language in a language which is horizontal? Wikipedia has to write all the script horizontally, which would be like a page talking about English stacking all the letters vertically - it's weird. From Wikipedia: "Computer operating systems have been slow to adopt support for the Mongolian script, and almost all have incomplete support or other text rendering difficulties."