3 ms·

On the plus side, this is really great work. On the minus side I sometimes think people working on unicode, a system intended to allow for all the world's writt

by bane 11y ago

On the plus side, this is really great work. On the minus side I sometimes think people working on unicode, a system intended to allow for all the world's written languages to be represented by a single encoding, have lost their minds sometimes. I understand that emoji are used in-line with text, but they aren't text. Once you start going the path of encoding pictures into a text representation you're going to start missing things that people need in these kinds of pictoral representations. I'm sympathetic in a sense, I wrote a paper in college demonstrating why emoji/emoticons are a natural extension of text (to a point), because they can allow encoding emotion and intent, which natural language punctuation don't really allow for. But at the same time, is it really necessary to have half a dozen fish, trains, several food items an alien head, a full set of numbers from 1-10, squares of different sizes, and other non-emotional iconography? And then the argument is that this is just intended to encode Japanese carrier emoji. Great, let's spend brain cycles building into unicode icons used by phone carriers in one country. I'm already bothered by the separate encodings I've seen from time to time for the exact same script, but in slightly different font variations. That's supposed to be handled by the font that's representing the script, not by the encoding. This really needs to be a separate encoding that's not unicode. I figure at some point, unicode will simply turn into a generic image format with a bunch of extra character encoding baggage that people will complain about.

aw3c2 11y ago
Do emojis always look the same or are there different sets? I often wonder if the emotion that I feel some icon perfectly shows, will be displayed the same way at the recipient. I know that fonts and icon sets are usually different per app. Is it just as unpredictable here?
- masklinn 11y ago
  Emoji are like letters, they have a basic meaning but the exact representation depends on the font. So there are "different sets" in the sense that U+1F46E "POLICE OFFICER" may not look the same across all systems, in the same way "A" will not always look the same unless you specify the font to use.
masklinn 11y ago
> On the plus side, this is really great work. On the minus side I sometimes think people working on unicode, a system intended to allow for all the world's written languages to be represented by a single encoding, have lost their minds sometimes. Why? It's a useful and convenient pictographic addition to text, and it brings attention to astral characters and combining characters whose handling is routinely screwed up by western developers (let alone anglo-centric ones) who don't routinely deal with them and don't care to test for these cases. I mean it took until 2010 for MySQL to start not destroying astral characters at all (with the introduction of "utf8mb4" in 5.5(.3)). And oddly enough, one of the big world-wide changes to text content between 2006 (MySQL 5.1) and 2010 was emoji spreading outside of japan starting around 2008. I'd like to say it's a coincidence, but that's not really likely. > And then the argument is that this is just intended to encode Japanese carrier emoji. Great, let's spend brain cycles building into unicode icons used by phone carriers in one country. That's an argument nobody is making because emoji escaped japan back in 2008 when people outside japan started unlocking the emoji iOS keyboard and using emoji elsewhere. > I'm already bothered by the separate encodings I've seen from time to time for the exact same script, but in slightly different font variations. That's supposed to be handled by the font that's representing the script, not by the encoding. That doesn't really work, because now you can't mix the two languages anymore, this is actually a big issue with han unification, mixing Chinese and Japanese in the same text becomes a pain in the ass because they can't look right without a bunch of font-based hackery, which expects the client to have all the right fonts in the first place. > This really needs to be a separate encoding that's not unicode. And so you couldn't mix emoji and text, now that would be convenient and absolutely wouldn't lead to proprietary emoji implementations in private unicode fields at all. Well it would actually because that's how emoji were first integrated in unicode in the first place.
- bane 11y ago
  You talk about this issue like text and images have never shared space next to one another in the same gui control on a computer screen. More importantly, your history is all wrong. Emoticons have been used in the west since the telegraph and kaomojis (emojis) showed up in the 80s and have been known outside of Japan not long after. These images are simply an interpretation of common emojis into graphical form, they literally map something like ^_^ to a smiley face. Somewhere along the line, it was thought to be a good idea to let people just choose from common emoticons and have it map to the character implementation rather than doing the reverse (because remembering things is hard). On the receiving end, the ^_^ is simply replaced with whatever image the chat app has chosen to represent that. Later, DoCoMo, KDDI and Softbank decided to further formalize the emoji codec mapping as part of Shift-JIS (and ISO-2022-JP) and that's why we have encodings for snail, minidisc and chicken leg, but not mosque, pork chop and blue jeans. Here's the mapping table https://docs.google.com/viewer?url=http%3A%2F%2Fwww.unicode.org%2F~scherer%2Femoji4unicode%2Fsnapshot%2Femojidata.pdf https://docs.google.com/viewer?url=http%3A%2F%2Fwww.unicode.... What the logic was to do this is anybody's guess, but it was probably for bandwidth efficiency. It would be more appropriate for there to be an entirely separate "expressions" standard for encoding various emoticon systems (there's several) and providing a standard iconography. You can already mix and match fonts and languages with images in most rich-text controls. Unicode is not the correct place to do this. And amazingly, text-controls, as we have already established, can support fonts and images next to each other. So extending a text control to support unicode, images and expressions, would seem more logical than shoehorning shift-jis, which is not a human written language, into unicode. Here's the original proposal that started this madness, so you can read the rationale. There's lots of people to blame for this, but we can start with the authors of this proposal. http://www.unicode.org/L2/L2009/09025r2-emoji.pdf http://www.unicode.org/L2/L2009/09025r2-emoji.pdf
ScottBurson 11y ago
I'm far from an expert on Unicode, but I definitely have the feeling that they've had trouble sticking to their original mission. A few months ago we had a discussion here on HN about some people still not being able to write their names in their native script -- Burmese? maybe, I forget -- but now we have skin tone modifiers for emoji. That said, I guess I agree, if you're going to have emoji at all, you have to at least think about this issue. My inclination would have been to make them all green from the beginning, so there's no question of ethnic favoritism, but I guess it didn't happen that way.

by bane 11y ago

aw3c2 11y ago

Do emojis always look the same or are there different sets? I often wonder if the emotion that I feel some icon perfectly shows, will be displayed the same way at the recipient. I know that fonts and icon sets are usually different per app. Is it just as unpredictable here?

masklinn 11y ago
Emoji are like letters, they have a basic meaning but the exact representation depends on the font. So there are "different sets" in the sense that U+1F46E "POLICE OFFICER" may not look the same across all systems, in the same way "A" will not always look the same unless you specify the font to use.

masklinn 11y ago

> On the plus side, this is really great work. On the minus side I sometimes think people working on unicode, a system intended to allow for all the world's written languages to be represented by a single encoding, have lost their minds sometimes. Why? It's a useful and convenient pictographic addition to text, and it brings attention to astral characters and combining characters whose handling is routinely screwed up by western developers (let alone anglo-centric ones) who don't routinely deal with them and don't care to test for these cases. I mean it took until 2010 for MySQL to start not destroying astral characters at all (with the introduction of "utf8mb4" in 5.5(.3)). And oddly enough, one of the big world-wide changes to text content between 2006 (MySQL 5.1) and 2010 was emoji spreading outside of japan starting around 2008. I'd like to say it's a coincidence, but that's not really likely. > And then the argument is that this is just intended to encode Japanese carrier emoji. Great, let's spend brain cycles building into unicode icons used by phone carriers in one country. That's an argument nobody is making because emoji escaped japan back in 2008 when people outside japan started unlocking the emoji iOS keyboard and using emoji elsewhere. > I'm already bothered by the separate encodings I've seen from time to time for the exact same script, but in slightly different font variations. That's supposed to be handled by the font that's representing the script, not by the encoding. That doesn't really work, because now you can't mix the two languages anymore, this is actually a big issue with han unification, mixing Chinese and Japanese in the same text becomes a pain in the ass because they can't look right without a bunch of font-based hackery, which expects the client to have all the right fonts in the first place. > This really needs to be a separate encoding that's not unicode. And so you couldn't mix emoji and text, now that would be convenient and absolutely wouldn't lead to proprietary emoji implementations in private unicode fields at all. Well it would actually because that's how emoji were first integrated in unicode in the first place.

bane 11y ago
You talk about this issue like text and images have never shared space next to one another in the same gui control on a computer screen. More importantly, your history is all wrong. Emoticons have been used in the west since the telegraph and kaomojis (emojis) showed up in the 80s and have been known outside of Japan not long after. These images are simply an interpretation of common emojis into graphical form, they literally map something like ^_^ to a smiley face. Somewhere along the line, it was thought to be a good idea to let people just choose from common emoticons and have it map to the character implementation rather than doing the reverse (because remembering things is hard). On the receiving end, the ^_^ is simply replaced with whatever image the chat app has chosen to represent that. Later, DoCoMo, KDDI and Softbank decided to further formalize the emoji codec mapping as part of Shift-JIS (and ISO-2022-JP) and that's why we have encodings for snail, minidisc and chicken leg, but not mosque, pork chop and blue jeans. Here's the mapping table https://docs.google.com/viewer?url=http%3A%2F%2Fwww.unicode.org%2F~scherer%2Femoji4unicode%2Fsnapshot%2Femojidata.pdf https://docs.google.com/viewer?url=http%3A%2F%2Fwww.unicode.... What the logic was to do this is anybody's guess, but it was probably for bandwidth efficiency. It would be more appropriate for there to be an entirely separate "expressions" standard for encoding various emoticon systems (there's several) and providing a standard iconography. You can already mix and match fonts and languages with images in most rich-text controls. Unicode is not the correct place to do this. And amazingly, text-controls, as we have already established, can support fonts and images next to each other. So extending a text control to support unicode, images and expressions, would seem more logical than shoehorning shift-jis, which is not a human written language, into unicode. Here's the original proposal that started this madness, so you can read the rationale. There's lots of people to blame for this, but we can start with the authors of this proposal. http://www.unicode.org/L2/L2009/09025r2-emoji.pdf http://www.unicode.org/L2/L2009/09025r2-emoji.pdf

ScottBurson 11y ago

I'm far from an expert on Unicode, but I definitely have the feeling that they've had trouble sticking to their original mission. A few months ago we had a discussion here on HN about some people still not being able to write their names in their native script -- Burmese? maybe, I forget -- but now we have skin tone modifiers for emoji. That said, I guess I agree, if you're going to have emoji at all, you have to at least think about this issue. My inclination would have been to make them all green from the beginning, so there's no question of ethnic favoritism, but I guess it didn't happen that way.