5 ms·

The number of graphemes an emoji displays as depends on the platform and font. How many "characters" does Safari think Ninja Cat[1] is? It displays as a single

The number of graphemes an emoji displays as depends on the platform and font. How many "characters" does Safari think Ninja Cat[1] is? It displays as a single grapheme on Windows 10. 1. https://emojipedia.org/ninja-cat/ https://emojipedia.org/ninja-cat/

ASalazarMX 4y ago
I'm not saying Apple's implementation should be the gold standard, just that characters that the Unicode specification allows to combine into one character, should be counted as one character. I understand that technical limitations will sometimes cause a combined character to be displayed as more than one character, as it happens in Safari with that ninja cat, and in those cases, it's preferable to decompose the emoji, or use a tofu character, than changing it for another.
- jakelazaroff 4y ago
  They are decomposing the emoji, are they not? The resulting glyph (with three family members rather than four) is just the result of the ligatures between the remaining emojis.
  - tobr 4y ago
    From a user perspective, a family emoji is one character. Whether it’s implemented as a ligature seems irrelevant, as that’s not visible to the user.
    - junon 4y ago
      Emoji was such a mistake in my opinion. It's cluttered the Unicode spec for a long time now and it's a continual source of problems.
      - tobr 4y ago
        An incredibly popular and successful mistake, in that case. Is the purpose of Unicode to be an uncluttered spec, or to be a useful way to represent text?
        wwalexander 4y ago
        Emojis aren’t text, though. Nobody “writes” emoji outside of the digital world so I don’t think emoji could accurately be called “text” or part of a “writing system: > Unicode, formally The Unicode Standard,[note 1][note 2] is an information technology standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems People write smiley faces, but there was already a code point for basic smileys. I think standardized emoticons are a good thing, but I don’t think the Unicode spec is where they best belong.
        tobr 4y ago
        I don’t see anything in the quoted passage that limits it to non-digital writing systems.
        Nevermark 4y ago
        Given that emoji are most typically used in text, having it all together seems better than two bodies coming up with incompatible solutions to emoji and the glyph issues of various languages. In the end emoji is a language, a visual icon language.
        WorldMaker 4y ago
        Where would they belong if not Unicode? Emoticons are a "writing system" (whether or not you believe them to be more "text" or "meta-text") that needs encoding, and one that was encoded in standards that predated Unicode. (They were at first added for compatibility with CJK systems that had already been using them for some time. They've been greatly expanded since those original encodings that Unicode "inherited" from other text encodings, but they started as text encodings in other standards.) Personally, I think Unicode actually did the world a favor by taking ownership of emoji and standardizing them. There's nothing "weird" that emoji do that doesn't exist somewhere else in some other language's text encodings. Things like ZWJ sequences for complex graphemes existed in a number of languages that Unicode encoded well before emoji. Emoji being common and popular helps break certain Western-centric assumptions that language encodings are "simple" and 1-to-1 code point to grapheme, assumptions that had been needed to break for many years before emoji gave Unicode users a tool to test that even "Western developers" needed to respect if they wanted happy users. Better emoji support is better Unicode support across the lovely and incredible diversity of what Unicode encodes.
      - gpderetta 4y ago
        Emojis simply manifest existing bugs in programs. Those bugs would have existed anyway as emojis are not the only symbols beyond the BMP nor the only symbols composed of grapheme clusters. Being much more frequent than other more exotic characters they just reveal bugs more often.
    - rkeene2 4y ago
      How many characters are in "offer" ? 4 (because of the ligature for "ff") or 5 ?
      - tobr 4y ago
        5. As I said, whether it’s implemented as a ligature is irrelevant, as that’s not visible to the user.
    - 10240 4y ago
      It's ambiguous. At least on my system (X11), I can move the cursor through a 4-member family with one press of Left or Right, and can't put the cursor inside it, and I can delete it with one press of Delete. But it takes 5 presses of Backspace to delete it.
MereInterest 4y ago
And depends on the location. For example, the characters U+1F1FA U+1F1F8 are the regional indicators "U" and "S", and are rendered as . These are two separate codepoints that may together be displayed as a United States flag. Similarly, the regional indicators "T" and "W" are rendered as and "H" and "K" are rendered as . On my system, this is rendered as the flag of Taiwan and Hong Kong, respectively. Depending on where you live, these regional indicators might not be rendered as flags. Edit: Looks like HN stripped out the character codes. The effect can be reproduced by copy-pasting the symbols from https://en.wikipedia.org/wiki/Regional_indicator_symbol https://en.wikipedia.org/wiki/Regional_indicator_symbol
- WorldMaker 4y ago
  > Depending on where you live, these regional indicators might not be rendered as flags. This is also why Microsoft-vended emoji fonts don't include flags/"regional indicators" support at all. On a Windows machine you just see the letters US or TW in an ugly boxed type and nothing like a flag. The interesting background story goes: everyone remembers that Windows 95 had that fancy graphical Time Zone selector that looked real pretty in screenshots (but wasn't actually the best way to select time zones anyway). The reasons Microsoft removed that tool are less well known and almost entirely geopolitical: time zone borders often correspond to country borders and several countries got mad at their border looking wrong and sued Microsoft for "lying" about their borders. After a bunch of micro-changes to the graphics of that widget and then a lot of money spent on lawsuits and geopolitical fights, Microsoft eventually just entirely removed the widget because it was a "nice-to-have" and never actually necessary. It is said that from that fight Microsoft decided that it never wanted to be involved in that sort of geopolitics ever again, and flags are nothing if not geopolitical symbols, so Microsoft's emoji fonts don't encode hardly any flags at all. (In exchange they encode a lot of fun "cats with jobs" that other font vendors still don't. I find that a fun trade-off myself.)
crazygringo 4y ago
According to the Unicode standard there is a canonical notion of countable graphemes, with zero ambiguity. I don't actually know whether ninja cat would count as one or two though. The spec for calculating grapheme boundaries is actually several pages long. And the OS might not obey the Unicode standard. Ninja cat appears to be proprietary to Microsoft.
- WorldMaker 4y ago
  What I recall of the standard algorithm is that it does include font/user-agent/locale pieces in its calculations and "zero ambiguity" is a bit of a stretch, unfortunately. Even the overview includes this note: > Note: Font-based information may be required to determine the appropriate unit to use for UI purposes First of all, grapheme counting depends on normalization and there's like 6 Unicode normalization algorithms to consider, depending on locale and intended use/display style. Keep in mind that grapheme counting includes things like natural ligatures which English has always had a rough time counting. Things like `fi` and `ft` that sometimes form single visual graphemes from the font's perspective for some serif fonts but are always supposed to be counted as two graphemes from the user's perspective. Relatedly, one of the simpler parts of the Unicode standard algorithm is that the ZWJ always signals a grapheme cluster/merged grapheme, so per the standard algorithm the Ninja Cat is always a single grapheme even though the usual emoji rules for all the non-Microsoft fonts that don't include Cat+ZWJ+Job sequences ("cats with jobs emojis") fallback to two display characters to try to get the idea across, the spec does say that they should still count that as only a single grapheme even when not presented as such. (ETA Aside: I think the cats with jobs emojis are great and fun and should be wider adopted outside of just Microsoft. Why should only the people emojis have jobs?)
  - Dylan16807 4y ago
    > First of all, grapheme counting depends on normalization and there's like 6 Unicode normalization algorithms to consider, depending on locale and intended use/display style. It says "The boundary specifications are stated in terms of text normalized according to Normalization Form NFD (see Unicode Standard Annex #15, “Unicode Normalization Forms” [UAX15]). In practice, normalization of the input is not required." And there's also "Even in Normalization Form NFC, a syllable block may contain a precomposed Hangul syllable in the middle." So are you sure normalization matters? I'm excluding K normalizations since they're not for general use and alter a bunch of characters on purpose. But also that's only NFC/NFD/NFKC/NFKD, are there two others or was "six" just a misremembering?
thih9 4y ago
> This Emoji ZWJ Sequence has not been Recommended For General Interchange (RGI) by Unicode. Expect limited cross-platform support.

ASalazarMX 4y ago

I'm not saying Apple's implementation should be the gold standard, just that characters that the Unicode specification allows to combine into one character, should be counted as one character. I understand that technical limitations will sometimes cause a combined character to be displayed as more than one character, as it happens in Safari with that ninja cat, and in those cases, it's preferable to decompose the emoji, or use a tofu character, than changing it for another.

jakelazaroff 4y ago
They are decomposing the emoji, are they not? The resulting glyph (with three family members rather than four) is just the result of the ligatures between the remaining emojis.
- tobr 4y ago
  From a user perspective, a family emoji is one character. Whether it’s implemented as a ligature seems irrelevant, as that’s not visible to the user.
  - junon 4y ago
    Emoji was such a mistake in my opinion. It's cluttered the Unicode spec for a long time now and it's a continual source of problems.
    - tobr 4y ago
      An incredibly popular and successful mistake, in that case. Is the purpose of Unicode to be an uncluttered spec, or to be a useful way to represent text?
      - wwalexander 4y ago
        Emojis aren’t text, though. Nobody “writes” emoji outside of the digital world so I don’t think emoji could accurately be called “text” or part of a “writing system: > Unicode, formally The Unicode Standard,[note 1][note 2] is an information technology standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems People write smiley faces, but there was already a code point for basic smileys. I think standardized emoticons are a good thing, but I don’t think the Unicode spec is where they best belong.
        tobr 4y ago
        I don’t see anything in the quoted passage that limits it to non-digital writing systems.
        Nevermark 4y ago
        Given that emoji are most typically used in text, having it all together seems better than two bodies coming up with incompatible solutions to emoji and the glyph issues of various languages. In the end emoji is a language, a visual icon language.
        WorldMaker 4y ago
        Where would they belong if not Unicode? Emoticons are a "writing system" (whether or not you believe them to be more "text" or "meta-text") that needs encoding, and one that was encoded in standards that predated Unicode. (They were at first added for compatibility with CJK systems that had already been using them for some time. They've been greatly expanded since those original encodings that Unicode "inherited" from other text encodings, but they started as text encodings in other standards.) Personally, I think Unicode actually did the world a favor by taking ownership of emoji and standardizing them. There's nothing "weird" that emoji do that doesn't exist somewhere else in some other language's text encodings. Things like ZWJ sequences for complex graphemes existed in a number of languages that Unicode encoded well before emoji. Emoji being common and popular helps break certain Western-centric assumptions that language encodings are "simple" and 1-to-1 code point to grapheme, assumptions that had been needed to break for many years before emoji gave Unicode users a tool to test that even "Western developers" needed to respect if they wanted happy users. Better emoji support is better Unicode support across the lovely and incredible diversity of what Unicode encodes.
    - gpderetta 4y ago
      Emojis simply manifest existing bugs in programs. Those bugs would have existed anyway as emojis are not the only symbols beyond the BMP nor the only symbols composed of grapheme clusters. Being much more frequent than other more exotic characters they just reveal bugs more often.
  - rkeene2 4y ago
    How many characters are in "offer" ? 4 (because of the ligature for "ff") or 5 ?
    - tobr 4y ago
      5. As I said, whether it’s implemented as a ligature is irrelevant, as that’s not visible to the user.
  - 10240 4y ago
    It's ambiguous. At least on my system (X11), I can move the cursor through a 4-member family with one press of Left or Right, and can't put the cursor inside it, and I can delete it with one press of Delete. But it takes 5 presses of Backspace to delete it.

MereInterest 4y ago

And depends on the location. For example, the characters U+1F1FA U+1F1F8 are the regional indicators "U" and "S", and are rendered as . These are two separate codepoints that may together be displayed as a United States flag. Similarly, the regional indicators "T" and "W" are rendered as and "H" and "K" are rendered as . On my system, this is rendered as the flag of Taiwan and Hong Kong, respectively. Depending on where you live, these regional indicators might not be rendered as flags. Edit: Looks like HN stripped out the character codes. The effect can be reproduced by copy-pasting the symbols from https://en.wikipedia.org/wiki/Regional_indicator_symbol https://en.wikipedia.org/wiki/Regional_indicator_symbol

WorldMaker 4y ago
> Depending on where you live, these regional indicators might not be rendered as flags. This is also why Microsoft-vended emoji fonts don't include flags/"regional indicators" support at all. On a Windows machine you just see the letters US or TW in an ugly boxed type and nothing like a flag. The interesting background story goes: everyone remembers that Windows 95 had that fancy graphical Time Zone selector that looked real pretty in screenshots (but wasn't actually the best way to select time zones anyway). The reasons Microsoft removed that tool are less well known and almost entirely geopolitical: time zone borders often correspond to country borders and several countries got mad at their border looking wrong and sued Microsoft for "lying" about their borders. After a bunch of micro-changes to the graphics of that widget and then a lot of money spent on lawsuits and geopolitical fights, Microsoft eventually just entirely removed the widget because it was a "nice-to-have" and never actually necessary. It is said that from that fight Microsoft decided that it never wanted to be involved in that sort of geopolitics ever again, and flags are nothing if not geopolitical symbols, so Microsoft's emoji fonts don't encode hardly any flags at all. (In exchange they encode a lot of fun "cats with jobs" that other font vendors still don't. I find that a fun trade-off myself.)

crazygringo 4y ago

According to the Unicode standard there is a canonical notion of countable graphemes, with zero ambiguity. I don't actually know whether ninja cat would count as one or two though. The spec for calculating grapheme boundaries is actually several pages long. And the OS might not obey the Unicode standard. Ninja cat appears to be proprietary to Microsoft.

WorldMaker 4y ago
What I recall of the standard algorithm is that it does include font/user-agent/locale pieces in its calculations and "zero ambiguity" is a bit of a stretch, unfortunately. Even the overview includes this note: > Note: Font-based information may be required to determine the appropriate unit to use for UI purposes First of all, grapheme counting depends on normalization and there's like 6 Unicode normalization algorithms to consider, depending on locale and intended use/display style. Keep in mind that grapheme counting includes things like natural ligatures which English has always had a rough time counting. Things like `fi` and `ft` that sometimes form single visual graphemes from the font's perspective for some serif fonts but are always supposed to be counted as two graphemes from the user's perspective. Relatedly, one of the simpler parts of the Unicode standard algorithm is that the ZWJ always signals a grapheme cluster/merged grapheme, so per the standard algorithm the Ninja Cat is always a single grapheme even though the usual emoji rules for all the non-Microsoft fonts that don't include Cat+ZWJ+Job sequences ("cats with jobs emojis") fallback to two display characters to try to get the idea across, the spec does say that they should still count that as only a single grapheme even when not presented as such. (ETA Aside: I think the cats with jobs emojis are great and fun and should be wider adopted outside of just Microsoft. Why should only the people emojis have jobs?)
- Dylan16807 4y ago
  > First of all, grapheme counting depends on normalization and there's like 6 Unicode normalization algorithms to consider, depending on locale and intended use/display style. It says "The boundary specifications are stated in terms of text normalized according to Normalization Form NFD (see Unicode Standard Annex #15, “Unicode Normalization Forms” [UAX15]). In practice, normalization of the input is not required." And there's also "Even in Normalization Form NFC, a syllable block may contain a precomposed Hangul syllable in the middle." So are you sure normalization matters? I'm excluding K normalizations since they're not for general use and alter a bunch of characters on purpose. But also that's only NFC/NFD/NFKC/NFKD, are there two others or was "six" just a misremembering?

thih9 4y ago

> This Emoji ZWJ Sequence has not been Recommended For General Interchange (RGI) by Unicode. Expect limited cross-platform support.