11 ms·

> This led me down a rabbit hole of trying to find a font that could render all of the language scripts. I went on Google Fonts to find this perfect font and fo

by arp242 3y ago

> This led me down a rabbit hole of trying to find a font that could render all of the language scripts. I went on Google Fonts to find this perfect font and found that one did not exist. This seems like a rather odd example of "technological disparity"; it's just modular and "one huge font file" isn't in wide use because it's unwieldy. Install/add an additional Chinese font, or Tamil font, etc. as needed. Most use cases don't need "all the scripts" and a modular approach is much better as fonts are large: NotoSans-Regular is 590K; NotoSansCJK-Regular is 26M, etc. In total Noto Fonts is 373M on my system. And it contains hundreds of thousands of different glyphs, and requires expertise on dozens of writing systems. Creating such a font is a significant effort, which is why few "universal" fonts exist, and why there are many "for this script" fonts. --- I wonder how one could design a better Morse code for Chinese; Japanese Morse code used Kana, rather than Kanji, and Hangul can be composed from smaller blocks. As near as I can find, Chinese is kind of an outlier here. Any system I can think of would probably be equally difficult to use or error-prone (due to either operator error or line noise mangling things).

masklinn 3y ago
> Hangul can be composed from smaller blocks Not “can be”, “is”. Hangul was designed, it’s alphabetic and morpho-syllabic features were intentional.
- arp242 3y ago
  What is even the point of pedantic "well, duh" comments like this? "English {can be,is} written using the Latin character set" "The symbol 'e' {can be,is} used as a vowel" "Italian {can be,is} spoken in Rome" "Python {can be,is} used to program a computer" "A car {can be,is} used to drive on the road"
  - smallnix 3y ago
    Because not everyone knows Hangul and could learn something wrong.
    - Asraelite 3y ago
      But it's not even strictly correct from a technical point of view. In Unicode, Hangul syllables can either be encoded directly as a single codepoint, or composed from multiple codepoints each representing a single jamo. One could conceivably devise a Morse code encoding that simply uses the Unicode codepoints of the syllables. It would be a bad way to do it, but it's an option. Composition isn't a hard requirement for Hangul encodings.
iainmerrick 3y ago
I wonder how one could design a better Morse code for Chinese In terms of efficiency, I guess you might start by ranking characters by frequency, and build a Huffman code? Then think about adding parity bits or sync symbols or whatnot. It's hard to imagine people learning it, rather than painstakingly looking up each character in a book, but I suppose it'd be similar to learning any other numeric mapping. Wikipedia says "Chinese expert telegraphers used to remember several thousands of codes of the most frequent use".
- pmoriarty 3y ago
  Please forgive my extremely ignorant question, but could Chinese be written phonetically, maybe using the IPA or some existing/custom script? If so, that should require much less memorization than memorizing thousands of Chinese characters, and a "morse code" could be made from the phonetic script.
  - akasakahakada 3y ago
    Absolutely no since everything about Chinese is not based on pronunciation. Even the meaning of a word change by the pitch. If you don't specify the pitch in the phonetic transcript, the level of uniqueness in pronunciation of vocabulary is so much lower than other languages. Same may say Japanese can express their language solely in pronunciation (Kana) but this is not true. There are easily 20+ meanings for the same pronunciation and are even hard to distinguish them with context provided. Just open any Japanese dictionary then you will see that it is a completely broken language if without the expressive power of Kanji (or Chinese characters).
    - crooked-v 3y ago
      For a simple example in practice, many Japanese names can be written multiple ways based on the kanji used, ending up with different meanings and different implications, while being pronounced exactly the same. For example, the common name Michiko can be written in many different ways with many different meanings, such as: 美智子 — "beautiful wise child" 美千子 — "child of a thousand beauties" 見知子 — "child of recognition" 道子 — "child of the way" 路子 — "child of the road" 倫子 — "child of morals" 皆子 — "child of all" 通子 — "child of passage"
oefrha 3y ago
Missing glyphs would be a much smaller problem if people stop creating/using text renderers that don’t support font fallback and system font store. There are still a good number of GUI frameworks that assume one font or even one Latin font is enough. Fyne (golang) is an example; the advice, if you want to display CJK text, is to bundle a single font with all the glyphs you need…
wodenokoto 3y ago
You could use pinyin or bopomofo as a base. Written Chinese can be thought of as a syllable alphabet with 100s of ways to write each syllable. For a fluent reader it is easier to read with those contextual hints, but strictly speaking it is not necessary. Spoken Chinese works just fine without them. Morse code usually has its own vernacular so it is easy to get around the lack of characters.
- arp242 3y ago
  > You could use pinyin or bopomofo as a base. Ah yeah, that would be the obvious solution today, but neither were invented yet in the 1880s – I was kind of thinking from that viewpoint: "how could they have done better in 1880?", rather than "how could we do better today?" The best I could think of is a UTF-8 like scheme, where a prefix selects the number of digits where less digits represent more common characters. I'm not entirely sure how well that would work in practice as I operated a telegraph exactly once in my life (as a child). I know figure/letter/Cyrillic shifts in telegraphs could cause some problems if the shift got lost or was garbled, but I suppose by prefixing every character solves this – but then again, why didn't they do this for Baudot code/ITA?
  - wodenokoto 3y ago
    I hadn’t considered late 18-hundreds as a context, but I’m gonna stick to my guns and say wade-Giles. Wades incarnation goes back to the 1860s. Since you don’t need the alphabet as an intermediate you can can encode each initial and final from the table[1] as an individual character to balance the number of encodings per symbol with the number of symbols per word. Since Morse typically encodes simple messages with its own vernacular I think you can safely drop encoding the tones. [1] https://en.m.wikipedia.org/wiki/Wade–Giles_table https://en.m.wikipedia.org/wiki/Wade–Giles_table
geokon 3y ago
Pretty sure Unifont will do what the author wants. Coding or system wide and you never gotten tofu. (+ the crispness of bitmap fonts)
Decabytes 3y ago
I actually ran into this myself when I was testing my C program for UTF-8 support. Ended up just installing noto CJK but I wondered why there wasn’t a universal font I could use. Now I’m wondering why I can’t just fuse a font that has cjk chats to a font that has Latin ones.
- uranusjr 3y ago
  You totally can, tools such as fontforge even make this pretty straightforward.
- numpad0 3y ago
  There's a Unicode quirk/mistake called Han Unification that similar CJK letters, not always close enough, are assigned same code points. As the result, each of CJK languages(Chinese, Hong Kong/Taiwan Chinese, Korean, Japanese, Vietnamese ...) has to be displayed in its own fonts. It must have been much less of issue when UTF-8 specs were formalized in early 90s, as memory was expensive, and electronic communications between CJK cultures were minimal; computers just had to have the correct font for a single specific language, and others didn't even need to render, let alone correctly. Being able to display similar letters outside of a specific language must have been a bonus. Today it's a much pronounced annoyance as software is globally distributed, but until the Unicode Consortium figures out how to normalize this CJK situation, the font switching hack has to go on and there just can't be a single font that could render everything.
deleted 3y ago
[deleted]

by arp242 3y ago

masklinn 3y ago

> Hangul can be composed from smaller blocks Not “can be”, “is”. Hangul was designed, it’s alphabetic and morpho-syllabic features were intentional.

arp242 3y ago
What is even the point of pedantic "well, duh" comments like this? "English {can be,is} written using the Latin character set" "The symbol 'e' {can be,is} used as a vowel" "Italian {can be,is} spoken in Rome" "Python {can be,is} used to program a computer" "A car {can be,is} used to drive on the road"
- smallnix 3y ago
  Because not everyone knows Hangul and could learn something wrong.
  - Asraelite 3y ago
    But it's not even strictly correct from a technical point of view. In Unicode, Hangul syllables can either be encoded directly as a single codepoint, or composed from multiple codepoints each representing a single jamo. One could conceivably devise a Morse code encoding that simply uses the Unicode codepoints of the syllables. It would be a bad way to do it, but it's an option. Composition isn't a hard requirement for Hangul encodings.

iainmerrick 3y ago

I wonder how one could design a better Morse code for Chinese In terms of efficiency, I guess you might start by ranking characters by frequency, and build a Huffman code? Then think about adding parity bits or sync symbols or whatnot. It's hard to imagine people learning it, rather than painstakingly looking up each character in a book, but I suppose it'd be similar to learning any other numeric mapping. Wikipedia says "Chinese expert telegraphers used to remember several thousands of codes of the most frequent use".

pmoriarty 3y ago
Please forgive my extremely ignorant question, but could Chinese be written phonetically, maybe using the IPA or some existing/custom script? If so, that should require much less memorization than memorizing thousands of Chinese characters, and a "morse code" could be made from the phonetic script.
- akasakahakada 3y ago
  Absolutely no since everything about Chinese is not based on pronunciation. Even the meaning of a word change by the pitch. If you don't specify the pitch in the phonetic transcript, the level of uniqueness in pronunciation of vocabulary is so much lower than other languages. Same may say Japanese can express their language solely in pronunciation (Kana) but this is not true. There are easily 20+ meanings for the same pronunciation and are even hard to distinguish them with context provided. Just open any Japanese dictionary then you will see that it is a completely broken language if without the expressive power of Kanji (or Chinese characters).
  - crooked-v 3y ago
    For a simple example in practice, many Japanese names can be written multiple ways based on the kanji used, ending up with different meanings and different implications, while being pronounced exactly the same. For example, the common name Michiko can be written in many different ways with many different meanings, such as: 美智子 — "beautiful wise child" 美千子 — "child of a thousand beauties" 見知子 — "child of recognition" 道子 — "child of the way" 路子 — "child of the road" 倫子 — "child of morals" 皆子 — "child of all" 通子 — "child of passage"

oefrha 3y ago

Missing glyphs would be a much smaller problem if people stop creating/using text renderers that don’t support font fallback and system font store. There are still a good number of GUI frameworks that assume one font or even one Latin font is enough. Fyne (golang) is an example; the advice, if you want to display CJK text, is to bundle a single font with all the glyphs you need…

wodenokoto 3y ago

You could use pinyin or bopomofo as a base. Written Chinese can be thought of as a syllable alphabet with 100s of ways to write each syllable. For a fluent reader it is easier to read with those contextual hints, but strictly speaking it is not necessary. Spoken Chinese works just fine without them. Morse code usually has its own vernacular so it is easy to get around the lack of characters.

arp242 3y ago
> You could use pinyin or bopomofo as a base. Ah yeah, that would be the obvious solution today, but neither were invented yet in the 1880s – I was kind of thinking from that viewpoint: "how could they have done better in 1880?", rather than "how could we do better today?" The best I could think of is a UTF-8 like scheme, where a prefix selects the number of digits where less digits represent more common characters. I'm not entirely sure how well that would work in practice as I operated a telegraph exactly once in my life (as a child). I know figure/letter/Cyrillic shifts in telegraphs could cause some problems if the shift got lost or was garbled, but I suppose by prefixing every character solves this – but then again, why didn't they do this for Baudot code/ITA?
- wodenokoto 3y ago
  I hadn’t considered late 18-hundreds as a context, but I’m gonna stick to my guns and say wade-Giles. Wades incarnation goes back to the 1860s. Since you don’t need the alphabet as an intermediate you can can encode each initial and final from the table[1] as an individual character to balance the number of encodings per symbol with the number of symbols per word. Since Morse typically encodes simple messages with its own vernacular I think you can safely drop encoding the tones. [1] https://en.m.wikipedia.org/wiki/Wade–Giles_table https://en.m.wikipedia.org/wiki/Wade–Giles_table

geokon 3y ago

Pretty sure Unifont will do what the author wants. Coding or system wide and you never gotten tofu. (+ the crispness of bitmap fonts)

Decabytes 3y ago

I actually ran into this myself when I was testing my C program for UTF-8 support. Ended up just installing noto CJK but I wondered why there wasn’t a universal font I could use. Now I’m wondering why I can’t just fuse a font that has cjk chats to a font that has Latin ones.

uranusjr 3y ago
You totally can, tools such as fontforge even make this pretty straightforward.
numpad0 3y ago
There's a Unicode quirk/mistake called Han Unification that similar CJK letters, not always close enough, are assigned same code points. As the result, each of CJK languages(Chinese, Hong Kong/Taiwan Chinese, Korean, Japanese, Vietnamese ...) has to be displayed in its own fonts. It must have been much less of issue when UTF-8 specs were formalized in early 90s, as memory was expensive, and electronic communications between CJK cultures were minimal; computers just had to have the correct font for a single specific language, and others didn't even need to render, let alone correctly. Being able to display similar letters outside of a specific language must have been a bonus. Today it's a much pronounced annoyance as software is globally distributed, but until the Unicode Consortium figures out how to normalize this CJK situation, the font switching hack has to go on and there just can't be a single font that could render everything.

deleted 3y ago

[deleted]