7 ms·

The ascendency of the CJK market, followed by Google Chrome, paved the way for UTF-8 everywhere. The more interesting thing is why basically no one uses Easter

The ascendency of the CJK market, followed by Google Chrome, paved the way for UTF-8 everywhere. The more interesting thing is why basically no one uses Eastern ideograms in the West, except maybe the Korean ideogram for crying (ㅠㅠ) and rarely, other kaomoji-like stuff. Some kanji also tell visual stories, and most children learn them just fine, so it’s not as simple as accessibility. Borrowing kanji was also anticipated by many sci fi writers and yet is not to be.

nneonneo 6y ago
Out of curiosity, which Korean ideograph would that be? Korean doesn’t use ideograms (much) anymore; they use an alphabet packed into syllabic blocks. The character I can think of that kind of matches the description is 囧, which is a Chinese character.
- masklinn 6y ago
  > Out of curiosity, which Korean ideograph would that be? Yu. Having two of them looks like a crying face. Although tha (ಥ) is also a common component of crying face (ಥ_ಥ). They're talking about kaomoji which use various non-latin or fullwidth symbols (though you're right that they're largely not ideograms) to compose pretty extensive "smileys" e.g. the look of disapproval uses kannada, denko uses greek and katakana, ...
  - mattnewton 6y ago
    Shameless plug ア( ͡° ͜ʖ ͡°)ア The Gboard keyboard on android has a tab for many of these common “emoticon” faces / character sequences. If you open the emoji picker on the keyboard and then tap the far right bottom tab icon “:-)” They can get very elaborate though, these are just very basic common faces.
    - masklinn 6y ago
      > The Gboard keyboard on android has a tab for many of these common “emoticon” faces / character sequences. If you open the emoji picker on the keyboard and then tap the far right bottom tab icon “:-)” iOS also has that on the standard Japanese "Kana" keyboard (and possibly others), under the "^_^" key.
      - reificator 6y ago
        Windows has this as well, just hit Windows + ; and go to the ;-) tab.
- kevin_thibedeau 6y ago
  It's a jamo component used to compose full hangul characters. https://en.wikipedia.org/wiki/List_of_Hangul_jamo https://en.wikipedia.org/wiki/List_of_Hangul_jamo
jandrese 6y ago
I'm a little sad that the cute Japanese 「quote characters」 have not gotten traction. I'd love to be able to use those in code.
- Isthatablackgsd 6y ago
  Those are called Corner Bracket. It took me a while to find out that I have to have CJK font installed in my computer to use the corner bracket. And the file size of CJK font family are huge! More than 100MB.
  - boogies 6y ago
    I see them and the only CJK font on my PC is GNU Unifont, which is only ~12MB for the TTF version IIRC, and smaller for other formats.
    - Isthatablackgsd 6y ago
      Oh GNU Unifont is new for me, thanks for sharing that information. I used other source for CJK (the one that are 100MB)to ensure that I have every single possible uncommon character/glyph installed without chasing for more fonts. One source that have it all in one file. I discovered this because other sources don't always have a full set.
    - account42 6y ago
      GNU Unifont is a bitmap font with a tiny resolution. Nice to have as a last level fallback, but too ugly for any characters you actually see more than once in a fortnight.
- andrewl-hn 6y ago
  First of all, they ARE getting traction. Many Youtubers and Twitch streamers started to use them in stream / video titles. I haven't seen corner barackets at all ten years ago, and these days I see them in use at least once a week. Some programming languages also start adopting them, too. Raku is the one I know (it allows French and German quotes, too). Maybe Julia, too? I think some language communities tend to be more open to widespread Unicode usage in source code than others.
SpicyLemonZest 6y ago
It's accessibility in the sense that computer input methods popular in the West can't generate them. As far as I know, there's no way to get my computer or phone keyboards to produce 水 without switching to one of the CJK input modes.
- layoutIfNeeded 6y ago
  On Windows you can use Alt + numpad keys for entering the character code. https://en.m.wikipedia.org/wiki/Alt_code https://en.m.wikipedia.org/wiki/Alt_code
- nneonneo 6y ago
  On Mac, at least, the "Emoji keyboard" accessible through Cmd+Ctrl+Space in all standard text controls makes it possible to add basically any character in Unicode if you know its name. For example, you can type "water" to get ⽔ (along with other characters, like the water droplet emojis). I use this often to type the Greek beta symbol, for example.
  - tzs 6y ago
    ...and then when you look it up that way, if you note that it is U+2F54, and if you have your your input source set to "Unicode Hex Input", you can enter ⽔ by holding down the option key (⌥) and typing 2f54. As far as I know this only works for characters in the BMP, but for such characters it may be faster than picking them from that cmd-ctrl-space popup. If you don't have that input source available, you can add it via the "Input Sources" tab of Keyboard preferences. There you can also enable showing the input menu in the menu bar giving you an easy way to switch between "Unicode Hex Input" and your normal input source.
josefx 6y ago
> followed by Google Chrome I will bite, wtf has Chrome to do with UTF-8? As far as I can find the last browser to struggle with it was IE5, IE6 was released almost a decade before Chrome was a thing.
- belval 6y ago
  And even then that's not true, parts of the Chromium projects still use UTF-16 internally.
  - csande17 6y ago
    Aren't JavaScript strings defined to be UTF-16? So parts of every browser will use UTF-16, forever!
    - chungy 6y ago
      UCS-2, actually, but for practical purposes, everyone treats them as UTF-16 as long as they are also valid UTF-16.
      - chrismorgan 6y ago
        I’d quibble over that characterisation. JavaScript uses neither UCS-2 nor UTF-16. Rather, its strings are a sequence of 16-bit Unicode code points, with surrogates handled according to UTF-16 rules, saving that unmatched surrogates are permitted to remain (that is, JavaScript strings aren’t necessarily valid Unicode). There is no requirement whatsoever that the browser actually use 16-bit code units to represent strings. This is what Simon Sapin achieved for Servo with the WTF-8 encoding, which extends UTF-8 to allow surrogates, including unmatched surrogates: it makes it able to represent these strings with 8-bit code units, commonly halving memory usage and improving the speed of various sorts of linear processing (though at the cost of random access speed which becomes O(n)).
CountSessine 6y ago
The more interesting thing is why basically no one uses Eastern ideograms in the West ¯\_(ツ)_/¯ I don't know! (Yeah - I know - it's not an ideogram)
tasogare 6y ago
> basically no one uses Eastern ideograms in the West That’s false. There is a lot of students of CJK languages in European universities as Japanese and more recently Korean as popular language. Chinese and Japanese can be learn in some high-school too. There are publishers publishing books containing such characters (e.g. You Feng in Paris). And of course the diaspora and heritage learners... That a lot of people, even more in countries with significant East-Asian population like Australia or the US.

nneonneo 6y ago

Out of curiosity, which Korean ideograph would that be? Korean doesn’t use ideograms (much) anymore; they use an alphabet packed into syllabic blocks. The character I can think of that kind of matches the description is 囧, which is a Chinese character.

masklinn 6y ago
> Out of curiosity, which Korean ideograph would that be? Yu. Having two of them looks like a crying face. Although tha (ಥ) is also a common component of crying face (ಥ_ಥ). They're talking about kaomoji which use various non-latin or fullwidth symbols (though you're right that they're largely not ideograms) to compose pretty extensive "smileys" e.g. the look of disapproval uses kannada, denko uses greek and katakana, ...
- mattnewton 6y ago
  Shameless plug ア( ͡° ͜ʖ ͡°)ア The Gboard keyboard on android has a tab for many of these common “emoticon” faces / character sequences. If you open the emoji picker on the keyboard and then tap the far right bottom tab icon “:-)” They can get very elaborate though, these are just very basic common faces.
  - masklinn 6y ago
    > The Gboard keyboard on android has a tab for many of these common “emoticon” faces / character sequences. If you open the emoji picker on the keyboard and then tap the far right bottom tab icon “:-)” iOS also has that on the standard Japanese "Kana" keyboard (and possibly others), under the "^_^" key.
    - reificator 6y ago
      Windows has this as well, just hit Windows + ; and go to the ;-) tab.
kevin_thibedeau 6y ago
It's a jamo component used to compose full hangul characters. https://en.wikipedia.org/wiki/List_of_Hangul_jamo https://en.wikipedia.org/wiki/List_of_Hangul_jamo

jandrese 6y ago

I'm a little sad that the cute Japanese 「quote characters」 have not gotten traction. I'd love to be able to use those in code.

Isthatablackgsd 6y ago
Those are called Corner Bracket. It took me a while to find out that I have to have CJK font installed in my computer to use the corner bracket. And the file size of CJK font family are huge! More than 100MB.
- boogies 6y ago
  I see them and the only CJK font on my PC is GNU Unifont, which is only ~12MB for the TTF version IIRC, and smaller for other formats.
  - Isthatablackgsd 6y ago
    Oh GNU Unifont is new for me, thanks for sharing that information. I used other source for CJK (the one that are 100MB)to ensure that I have every single possible uncommon character/glyph installed without chasing for more fonts. One source that have it all in one file. I discovered this because other sources don't always have a full set.
  - account42 6y ago
    GNU Unifont is a bitmap font with a tiny resolution. Nice to have as a last level fallback, but too ugly for any characters you actually see more than once in a fortnight.
andrewl-hn 6y ago
First of all, they ARE getting traction. Many Youtubers and Twitch streamers started to use them in stream / video titles. I haven't seen corner barackets at all ten years ago, and these days I see them in use at least once a week. Some programming languages also start adopting them, too. Raku is the one I know (it allows French and German quotes, too). Maybe Julia, too? I think some language communities tend to be more open to widespread Unicode usage in source code than others.

SpicyLemonZest 6y ago

It's accessibility in the sense that computer input methods popular in the West can't generate them. As far as I know, there's no way to get my computer or phone keyboards to produce 水 without switching to one of the CJK input modes.

layoutIfNeeded 6y ago
On Windows you can use Alt + numpad keys for entering the character code. https://en.m.wikipedia.org/wiki/Alt_code https://en.m.wikipedia.org/wiki/Alt_code
nneonneo 6y ago
On Mac, at least, the "Emoji keyboard" accessible through Cmd+Ctrl+Space in all standard text controls makes it possible to add basically any character in Unicode if you know its name. For example, you can type "water" to get ⽔ (along with other characters, like the water droplet emojis). I use this often to type the Greek beta symbol, for example.
- tzs 6y ago
  ...and then when you look it up that way, if you note that it is U+2F54, and if you have your your input source set to "Unicode Hex Input", you can enter ⽔ by holding down the option key (⌥) and typing 2f54. As far as I know this only works for characters in the BMP, but for such characters it may be faster than picking them from that cmd-ctrl-space popup. If you don't have that input source available, you can add it via the "Input Sources" tab of Keyboard preferences. There you can also enable showing the input menu in the menu bar giving you an easy way to switch between "Unicode Hex Input" and your normal input source.

josefx 6y ago

> followed by Google Chrome I will bite, wtf has Chrome to do with UTF-8? As far as I can find the last browser to struggle with it was IE5, IE6 was released almost a decade before Chrome was a thing.

belval 6y ago
And even then that's not true, parts of the Chromium projects still use UTF-16 internally.
- csande17 6y ago
  Aren't JavaScript strings defined to be UTF-16? So parts of every browser will use UTF-16, forever!
  - chungy 6y ago
    UCS-2, actually, but for practical purposes, everyone treats them as UTF-16 as long as they are also valid UTF-16.
    - chrismorgan 6y ago
      I’d quibble over that characterisation. JavaScript uses neither UCS-2 nor UTF-16. Rather, its strings are a sequence of 16-bit Unicode code points, with surrogates handled according to UTF-16 rules, saving that unmatched surrogates are permitted to remain (that is, JavaScript strings aren’t necessarily valid Unicode). There is no requirement whatsoever that the browser actually use 16-bit code units to represent strings. This is what Simon Sapin achieved for Servo with the WTF-8 encoding, which extends UTF-8 to allow surrogates, including unmatched surrogates: it makes it able to represent these strings with 8-bit code units, commonly halving memory usage and improving the speed of various sorts of linear processing (though at the cost of random access speed which becomes O(n)).

CountSessine 6y ago

The more interesting thing is why basically no one uses Eastern ideograms in the West ¯\_(ツ)_/¯ I don't know! (Yeah - I know - it's not an ideogram)

tasogare 6y ago

> basically no one uses Eastern ideograms in the West That’s false. There is a lot of students of CJK languages in European universities as Japanese and more recently Korean as popular language. Chinese and Japanese can be learn in some high-school too. There are publishers publishing books containing such characters (e.g. You Feng in Paris). And of course the diaspora and heritage learners... That a lot of people, even more in countries with significant East-Asian population like Australia or the US.