8 ms·

Apple's take seems more reasonable. When a user uses an emoji, they think of it as a single symbol, they don't care of the Unicode implementation, or its length

Apple's take seems more reasonable. When a user uses an emoji, they think of it as a single symbol, they don't care of the Unicode implementation, or its length in bytes. IMO this should be the standard, and all other interpretations are a repeat of the transition from ASCII to Unicode.

Sharlin 4y ago
Indeed Swift is one of the very few languages that even have a stdlib API for working with this intuitive definition of a "character" (the technical Unicode term is "(extended) grapheme cluster" and the division of text into EGCs is called Unicode text segmentation, described by [uax29] – spoiler: definitely not trivial). [uax29]: https://unicode.org/reports/tr29/ https://unicode.org/reports/tr29/
- alwaysbeconsing 4y ago
  There's no Swift involved here; this is in WebKit.
alpaca128 4y ago
Agreed, I'm surprised the linked OP filed a bug for Webkit when I'd say it's the only correct implementation. HTML is for UIs and I don't think many users would expect an emoji to be counted as 11 symbols when it looks like just one. If this wasn't an emoji but a symbol from a more complex writing system splitting that up would clearly be unacceptable, so I don't see why one should make an exception here just because this one known example of a family emoji still mostly survives the truncation.
scatters 4y ago
The number of graphemes an emoji displays as depends on the platform and font. How many "characters" does Safari think Ninja Cat[1] is? It displays as a single grapheme on Windows 10. 1. https://emojipedia.org/ninja-cat/ https://emojipedia.org/ninja-cat/
- ASalazarMX 4y ago
  I'm not saying Apple's implementation should be the gold standard, just that characters that the Unicode specification allows to combine into one character, should be counted as one character. I understand that technical limitations will sometimes cause a combined character to be displayed as more than one character, as it happens in Safari with that ninja cat, and in those cases, it's preferable to decompose the emoji, or use a tofu character, than changing it for another.
  - jakelazaroff 4y ago
    They are decomposing the emoji, are they not? The resulting glyph (with three family members rather than four) is just the result of the ligatures between the remaining emojis.
    - tobr 4y ago
      From a user perspective, a family emoji is one character. Whether it’s implemented as a ligature seems irrelevant, as that’s not visible to the user.
      - junon 4y ago
        Emoji was such a mistake in my opinion. It's cluttered the Unicode spec for a long time now and it's a continual source of problems.
        tobr 4y ago
        An incredibly popular and successful mistake, in that case. Is the purpose of Unicode to be an uncluttered spec, or to be a useful way to represent text?
        wwalexander 4y ago
        Emojis aren’t text, though. Nobody “writes” emoji outside of the digital world so I don’t think emoji could accurately be called “text” or part of a “writing system: > Unicode, formally The Unicode Standard,[note 1][note 2] is an information technology standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems People write smiley faces, but there was already a code point for basic smileys. I think standardized emoticons are a good thing, but I don’t think the Unicode spec is where they best belong.
duskwuff 4y ago
It isn't just for emoji, either. More ordinary grapheme clusters like characters with combining accents are also perceived by users as a single character, and should be counted as one by software.
david422 4y ago
The max length isn't for the user though. It's to prevent the user from inputting more data than the system can handle.
- rurban 4y ago
  If it isn't for the user, it must be named maxsize. A string length is now the number grapheme clusters, or just the width.
- tobr 4y ago
  Max length is there to communicate to the user how much data the system can handle.
- withinboredom 4y ago
  The user can submit data of any length… browsers aren’t required to implement validation for you. Nor is your user required to use a browser or client you wrote. This is security and application development 102, welcome to my class.
kevin_thibedeau 4y ago
This breaks down in the terminal where you need to know which codepoints are double width for managing layout. Some are ambiguous width because they've been present since before the inclusion of emoji and had optional emoji presentation added to them. The final determination is font dependent so you can never be sure without full insight into the rendering chain.
- pie_flavor 4y ago
  This doesn't break down anything. If a grapheme cluster is double width and you treat it as 5x because there's 5 code points in it, then you've still gotten the layout wrong. You can enforce text storage length limitations to prevent infinite zalgo, but text display length limitations should always deal in extended grapheme clusters.

Sharlin 4y ago

Indeed Swift is one of the very few languages that even have a stdlib API for working with this intuitive definition of a "character" (the technical Unicode term is "(extended) grapheme cluster" and the division of text into EGCs is called Unicode text segmentation, described by [uax29] – spoiler: definitely not trivial). [uax29]: https://unicode.org/reports/tr29/ https://unicode.org/reports/tr29/

alwaysbeconsing 4y ago
There's no Swift involved here; this is in WebKit.

alpaca128 4y ago

Agreed, I'm surprised the linked OP filed a bug for Webkit when I'd say it's the only correct implementation. HTML is for UIs and I don't think many users would expect an emoji to be counted as 11 symbols when it looks like just one. If this wasn't an emoji but a symbol from a more complex writing system splitting that up would clearly be unacceptable, so I don't see why one should make an exception here just because this one known example of a family emoji still mostly survives the truncation.

scatters 4y ago

The number of graphemes an emoji displays as depends on the platform and font. How many "characters" does Safari think Ninja Cat[1] is? It displays as a single grapheme on Windows 10. 1. https://emojipedia.org/ninja-cat/ https://emojipedia.org/ninja-cat/

ASalazarMX 4y ago
I'm not saying Apple's implementation should be the gold standard, just that characters that the Unicode specification allows to combine into one character, should be counted as one character. I understand that technical limitations will sometimes cause a combined character to be displayed as more than one character, as it happens in Safari with that ninja cat, and in those cases, it's preferable to decompose the emoji, or use a tofu character, than changing it for another.
- jakelazaroff 4y ago
  They are decomposing the emoji, are they not? The resulting glyph (with three family members rather than four) is just the result of the ligatures between the remaining emojis.
  - tobr 4y ago
    From a user perspective, a family emoji is one character. Whether it’s implemented as a ligature seems irrelevant, as that’s not visible to the user.
    - junon 4y ago
      Emoji was such a mistake in my opinion. It's cluttered the Unicode spec for a long time now and it's a continual source of problems.
      - tobr 4y ago
        An incredibly popular and successful mistake, in that case. Is the purpose of Unicode to be an uncluttered spec, or to be a useful way to represent text?
        wwalexander 4y ago
        Emojis aren’t text, though. Nobody “writes” emoji outside of the digital world so I don’t think emoji could accurately be called “text” or part of a “writing system: > Unicode, formally The Unicode Standard,[note 1][note 2] is an information technology standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems People write smiley faces, but there was already a code point for basic smileys. I think standardized emoticons are a good thing, but I don’t think the Unicode spec is where they best belong.

duskwuff 4y ago

It isn't just for emoji, either. More ordinary grapheme clusters like characters with combining accents are also perceived by users as a single character, and should be counted as one by software.

david422 4y ago

The max length isn't for the user though. It's to prevent the user from inputting more data than the system can handle.

rurban 4y ago
If it isn't for the user, it must be named maxsize. A string length is now the number grapheme clusters, or just the width.
tobr 4y ago
Max length is there to communicate to the user how much data the system can handle.
withinboredom 4y ago
The user can submit data of any length… browsers aren’t required to implement validation for you. Nor is your user required to use a browser or client you wrote. This is security and application development 102, welcome to my class.

kevin_thibedeau 4y ago

This breaks down in the terminal where you need to know which codepoints are double width for managing layout. Some are ambiguous width because they've been present since before the inclusion of emoji and had optional emoji presentation added to them. The final determination is font dependent so you can never be sure without full insight into the rendering chain.

pie_flavor 4y ago
This doesn't break down anything. If a grapheme cluster is double width and you treat it as 5x because there's 5 code points in it, then you've still gotten the layout wrong. You can enforce text storage length limitations to prevent infinite zalgo, but text display length limitations should always deal in extended grapheme clusters.