6 ms·
Can't blame them, given how mediocre computer representation continues to be. Just hop onto any Wikipedia page about Chinese characters, and be perplexed at th
by anigbrowl 3y ago
Can't blame them, given how mediocre computer representation continues to be.
Just hop onto any Wikipedia page about Chinese characters, and be perplexed at the non-displayability of many characters, eg https://en.wiktionary.org/wiki/Appendix:Chinese_radical/%E4%B9%99 https://en.wiktionary.org/wiki/Appendix:Chinese_radical/%E4%...
Unicode just isn't a good solution for pictographic languages where new characters can be invented the same way new words can be coined in alphabetic languages. In practice this doesn't happen that often, but there's no reason they can't be; there's an annual creative kanji contest in Japan: https://sousaku-kanji.com/archive.html https://sousaku-kanji.com/archive.html
- jacquesm 3y agoThat's super interesting: so even the 'well meaning' Unicode committee got a basic part of writing culturally very wrong: the assumption was that these are Alphabets, rather than living things. Alphabets change as well over time, but much more slowly than the words. Thanks for pointing this out.
- Ekaros 3y agoI think emojis were a mistake to include in Unicode. But they actually prove that we could continue adding characters, if the design allowed enough foresight in space designated to these evolving sets.
- Macha 3y agoTo an extent, the fact that they made so much software care about non-BMP characters that otherwise wouldn't and the fact that they made English speaking users care about characters outside Latin-1 makes it worthwhile. Even from an English speaking country, running into software that wouldn't accept the € sign was a problem until the early 10s, and I think displaying emojis drives American companies more than displaying the euro sign or kanji. So I'll give emojis in Unicode a wide pass on criticism for their effect in making everyone actually support all of unicode, and making people give up things like MySQL's utf8mb3.
- wk_end 3y agoAFAIK the construction of new kanji is a novelty, not something of any real importance in communication. Japan has embraced UTF-8, and before that they embraced Shift-JIS - a homegrown encoding that also didn't support constructing new characters. Is there any encoding that works well for this particular case? AFAIK there aren't any of any prominence, which suggests that it's a non-issue - surely if constructing new characters was important to them, Japanese or Chinese or Taiwanese computer scientists and standards organizations would push for some kind of encoding that enabled it.
- hfhdjdks 3y agoYou're correct. Parent comment says something true (some / many characters don't have a representation in most fonts), but the inference it makes is wrong (this is a bottleneck for communication)
- Macha 3y agoAnd specifically these characters that are not present in mainstream fonts are getting so far down the long tail that most natives wouldn't know how to write them, so would result in e.g. hiragana in Japanese if they knew the word at all. To some extent computers have even increased the variety of kanji usage as you'll get textbooks that describe a word as "usually written in kana" because it's infrequent enough that an average Japanese person wouldn't be confident to handwrite it from scratch, and so twenty years ago that meant they had to use Hiragana. But in these days of computer IMEs when they type it in phonetically and get a list of options presented to them, they're confident enough to pick one, and so the kanji gets used a lot more.
- anigbrowl 3y agoso far down the long tail that most natives wouldn't know how to write them True, but if we can't even display them then chunks of language are going to effectively disappear because people will have less and less opportunity to encounter them.
- anigbrowl 3y ago
- hfhdjdks 3y agoDoes this come from actual experience trying to communicate in chinese or japanese? At least for chinese (but I'm pretty sure it applies to japanese too), what you're saying doesn't ring as relevant (i.e. it is true that some characters don't have a graphic representation in many fonts, but it doesn't cause problems for communication / creation of new words) Plenty of fonts can represent a vast amount of characters (search around, but even free fonts can have >50k), much more than even highly literate people could ever use (a very educated person might know 8k characters, but would use much less actively). Every new release of Unicode releases new characters that are slowly incorporated into fonts...but for many years these are on the truly long tail of characters, nothing that any common person would use. There might be a few exceptions with new characters (the new japanese emperor characters comes to mind), but the thing to bear in mind is that new words don't necessarily involve new characters. A big chunk of nouns in chinese are composed of multiple characters, so making a new combination can generate a new word (电视 is television...electric + vision). This is probably even easier in japanese were they can use katakana to spell an English word and create a new japanese word.
- earthboundkid 3y agoYes. Han Unification is something people read about online and decide is bad or whatever, but it actually isn’t relevant to using Japanese with computers. People use Shift-JIS and fax machines because legacy technology takes a long time to die everywhere and longer in Japan. It’s not more complicated than that.
- numpad0 3y agoThis is correct: as much as it's wrong, Han Unification isn't a problem so long you stick to one language to support and remove all conflicting fonts for the other two locales, which are rarely needed anyway in Asia, it just comes back when you try to support multiple CJK languages at the same time. Unicode coverage of each languages are totally fine for 99% of use cases, the Unification problem is just it's reusing the same address space for all CJK languages.
- 3y ago
- mostlylurks 3y ago> In practice this doesn't happen that often The primary reason that this doesn't happen that often is that neither Chinese nor Japanese use Chinese characters in a primarily pictographic manner anymore. Only a small subset of the characters are pictographic (and even they are only vaguely so), and unlike in the distant past, most words are actually compositions of several characters, so new words can be represented via either combinations of existing characters if applicable, or via borrowing existing characters for their pronunciation (or in the case of Japanese, using the syllabic characters).
- anigbrowl 3y agomost words are actually compositions of several characters For that matter most characters are combinations of other ones. Briefly, there are 27 different stroke shapes and 214 different 'radicals' in kanji that are recombined in various ways to form new characters. Japanese kanji overlap heavily with Chinese Kangxi, but they're gone in somewhat different directions over the last few centuries. (I'm interested in both languages but less familiar with Chinese than Japanese because I can't speak the former at all.) https://en.wikipedia.org/wiki/List_of_kanji_radicals_by_stroke_count https://en.wikipedia.org/wiki/List_of_kanji_radicals_by_stro... I'm not a linguist, but I don't think you can understand languages that use Chinese characters if you're not willing to look at the components those characters are built out of. I mean, you could just learn them by sight and remember the ~2000 most common ones, but that seems like both a more shallow and difficult approach. Anyway, a lot of the more obscure characters are not displayed properly on most web pages like the Wikipedia example above because they're in one of the CJK extension tables, and this seems like a problem, because few people are going to want to dig through the unicode charts to find them. This isn't a problem with unicode as such, but with the way browsers and operating systems implement it. https://unicode.org/charts/ https://unicode.org/charts/
- Macha 3y agoYou couldn't simply do an encoding that generates kanji procedurally from the list of component radicals because of the differing placements, and sometimes significant modifications of the radicals to fit in the kanji. Take the kanji 勉 which is made up of the radicals 免 and 力. You could maybe have a "right half" slot for the 力 radical which would figure out the stretching but the fact that the leg of 免 extends underneath it is something that only exists for that kanji and then you'll need to special case it.
- elFarto 3y agoActually Unicode has something for character creation: 'IDS', or Ideographic Description Sequence. For example, the Chinese character for the word Biang[1] can be describe with: ⿺辶⿳穴⿲月⿱⿲幺言幺⿲長馬長刂心 [1] https://en.wikipedia.org/wiki/Biangbiang_noodles#Chinese_character_for_bi.C3.A1ng https://en.wikipedia.org/wiki/Biangbiang_noodles#Chinese_cha...