5 ms·

Excellent post, thank you! By the way, I've sometimes heard that it was the popularity of emoji that finally made American and Western European coders learn ab

Excellent post, thank you! By the way, I've sometimes heard that it was the popularity of emoji that finally made American and Western European coders learn about the particulars of unicode. Is this just a jokingly told anecdote, or did emoji really help?

Manishearth 10y ago
I find it pretty accurate, but that's just anecdotal from how I've seen software I use change in that time period, and how I've seen attitudes of people online change. I'm sure plenty of Western programmers were aware of this stuff, though. Just not enough.
klibertp 10y ago
Western European? I don't know - there are many languages in that part of the world which use some non-standard characters. Germany (not sure if it's still the case), France, Austria, Italy, Spain... Anyway, I always thought UNICODE ignorance is strictly an anglo-sphere thing - most of the rest of the world is very aware.
- 1wd 10y ago
  Western Europe uses more characters than ASCII like umlauts (äöü), accents (àèé), €¢, German ß and a few more, but large parts are still comfortably stuck in a codepage-style mindset thinking "ISO Latin-1 is good enough". (Also know as or confused with ISO/IEC 8859-1, CP-1252 or "Ansi".) https://en.wikipedia.org/wiki/Code_page#ISO.2FIEC_8859-related_code_pages https://en.wikipedia.org/wiki/Code_page#ISO.2FIEC_8859-relat...
- throwawayish 10y ago
  Central European languages can get more or less by with Latin-1, although even there many characters are already missing, iirc French, Czech, Dutch and Finnish. Special characters of many regional dialects are missing as well. FWIW (more or less proper) Unicode support seems to be pretty much an implicit, hard requirement today for almost anything facing users.
  - 1wd 10y ago
    Many government databases still just use some codepage and won't be changing anytime soon though. You can't get your name in a Swiss passport without first changing it into something ISO 8859-15 can represent. http://www.24heures.ch/suisse/Les-noms-de-l-Est-mutiles-lors-de-la-naturalisation/story/25231514 http://www.24heures.ch/suisse/Les-noms-de-l-Est-mutiles-lors...
Freak_NL 10y ago
Emoji may have helped popularize support for Unicode characters outside of the Basic Multilingual Plane (The first 2¹⁶ code points in Unicode), because all but a few fall outside of it. Before emoji any code point outside of the BMP tended to have a niche audience. When I wrote my master's thesis in 2009, support for non-BMP characters was still sketchy (although XeLaTeX and PDF-viewer Evince could handle them neatly on the Linux desktop). I doubt that emoji helped further popularize Unicode in Europe, because in my experience (anecdotal of course) developers were already well aware of the benefits of Unicode (and UTF-8 in particular for the web) before emoji got adopted from Japanese telephones and embraced en-masse. I suspect that in Europe the need for Unicode arose sooner. For Dutch for example; with the introduction of ISO-8859-15 in 1999 (subtly different from ISO-8859-1) you now often had two character encoding standards in mixed use for a single language, and when you had to render documents with names of people from various countries, things just got confusing (in the US diacritics tend to get dropped instead). UTF-8 offered a clean break from this. It seems plausible that emoji may have helped Americans break the remnants of the ASCII barrier though.
- Manishearth 10y ago
  (where "niche audience" includes speakers of the most spoken language in the world, but I assume you're talking about the West specifically)
  - Freak_NL 10y ago
    I was under the assumption that for the average Chinese speaker the Basic Multilingual Plane sufficed. Chinese characters (as used in Chinese, Japanese, Korean, and Vietnamese) were certainly added beyond the BMP as well (the CJK Unified Ideographps Extensions), but to my knowledge these all tend to be rather rare (think characters from Buddhist sutras, linguistic research, historical use, et cetera, but also from names not yet provided for in the BMP).
    - Manishearth 10y ago
      Huh. IIRC they're a bit more common than that, but yes, not very common. Or perhaps you're just talking about Chinese where I'm also considering Kanji and others. Which isn't from the most spoken language in the world, but still, top 10!

Manishearth 10y ago

I find it pretty accurate, but that's just anecdotal from how I've seen software I use change in that time period, and how I've seen attitudes of people online change. I'm sure plenty of Western programmers were aware of this stuff, though. Just not enough.

klibertp 10y ago

Western European? I don't know - there are many languages in that part of the world which use some non-standard characters. Germany (not sure if it's still the case), France, Austria, Italy, Spain... Anyway, I always thought UNICODE ignorance is strictly an anglo-sphere thing - most of the rest of the world is very aware.

1wd 10y ago
Western Europe uses more characters than ASCII like umlauts (äöü), accents (àèé), €¢, German ß and a few more, but large parts are still comfortably stuck in a codepage-style mindset thinking "ISO Latin-1 is good enough". (Also know as or confused with ISO/IEC 8859-1, CP-1252 or "Ansi".) https://en.wikipedia.org/wiki/Code_page#ISO.2FIEC_8859-related_code_pages https://en.wikipedia.org/wiki/Code_page#ISO.2FIEC_8859-relat...
throwawayish 10y ago
Central European languages can get more or less by with Latin-1, although even there many characters are already missing, iirc French, Czech, Dutch and Finnish. Special characters of many regional dialects are missing as well. FWIW (more or less proper) Unicode support seems to be pretty much an implicit, hard requirement today for almost anything facing users.
- 1wd 10y ago
  Many government databases still just use some codepage and won't be changing anytime soon though. You can't get your name in a Swiss passport without first changing it into something ISO 8859-15 can represent. http://www.24heures.ch/suisse/Les-noms-de-l-Est-mutiles-lors-de-la-naturalisation/story/25231514 http://www.24heures.ch/suisse/Les-noms-de-l-Est-mutiles-lors...

Freak_NL 10y ago

Emoji may have helped popularize support for Unicode characters outside of the Basic Multilingual Plane (The first 2¹⁶ code points in Unicode), because all but a few fall outside of it. Before emoji any code point outside of the BMP tended to have a niche audience. When I wrote my master's thesis in 2009, support for non-BMP characters was still sketchy (although XeLaTeX and PDF-viewer Evince could handle them neatly on the Linux desktop). I doubt that emoji helped further popularize Unicode in Europe, because in my experience (anecdotal of course) developers were already well aware of the benefits of Unicode (and UTF-8 in particular for the web) before emoji got adopted from Japanese telephones and embraced en-masse. I suspect that in Europe the need for Unicode arose sooner. For Dutch for example; with the introduction of ISO-8859-15 in 1999 (subtly different from ISO-8859-1) you now often had two character encoding standards in mixed use for a single language, and when you had to render documents with names of people from various countries, things just got confusing (in the US diacritics tend to get dropped instead). UTF-8 offered a clean break from this. It seems plausible that emoji may have helped Americans break the remnants of the ASCII barrier though.

Manishearth 10y ago
(where "niche audience" includes speakers of the most spoken language in the world, but I assume you're talking about the West specifically)
- Freak_NL 10y ago
  I was under the assumption that for the average Chinese speaker the Basic Multilingual Plane sufficed. Chinese characters (as used in Chinese, Japanese, Korean, and Vietnamese) were certainly added beyond the BMP as well (the CJK Unified Ideographps Extensions), but to my knowledge these all tend to be rather rare (think characters from Buddhist sutras, linguistic research, historical use, et cetera, but also from names not yet provided for in the BMP).
  - Manishearth 10y ago
    Huh. IIRC they're a bit more common than that, but yes, not very common. Or perhaps you're just talking about Chinese where I'm also considering Kanji and others. Which isn't from the most spoken language in the world, but still, top 10!