3 ms·

As someone who greatly values the ability to (occasionally!) use emojis as/in identifiers in my personal-project code, I'll never understand why the Unicode Con

As someone who greatly values the ability to (occasionally!) use emojis as/in identifiers in my personal-project code, I'll never understand why the Unicode Consortium chose to exclude emojis from the ID_START and ID_CONTINUE character classes [1] (in terms of which so many programming languages define their concept of a valid identifer). I'm not even sure the exclusion of emojis is consistent with their own explanation [2] of what an identifier should consist of, given that IMO an emoji pretty clearly qualifies as a 'ideograph' (a graphic symbol that represents an idea or concept). It is reasonable to exclude non-representational characters like punctuations marks, dashes, parentheses, brackets, etc that intuitively don't 'stand for' anything and instead solely serve to delineate syntactic structure. But the vast majority of emojis are entirely unlike punctuations marks and do intuitively 'stand for' some idea or object. [1] https://unicode.org/reports/tr31/ https://unicode.org/reports/tr31/ [2] "The formal syntax provided here captures the general intent that an identifier consists of a string of characters beginning with a letter or an ideograph, and followed by any number of letters, ideographs, digits, or underscores" (https://unicode.org/reports/tr31/#Default_Identifier_Syntax https://unicode.org/reports/tr31/#Default_Identifier_Syntax)

lifthrasiir 3y ago
> I'll never understand why the Unicode Consortium chose to exclude emojis from the ID_START and ID_CONTINUE character classes [1] (in terms of which so many programming languages define their concept of a valid identifer). Emojis are essentially colored symbols in terms of Unicode general categories and many languages do not accept symbols in an identifier, so it's no surprise that emojis are handled equally. UTR #31 does have a profile for identifiers with emoji (see the section 7.2), but it also demonstrates why it can't be a default: there are cases where `AB` is not valid but `ABC` is valid for some characters A, B and C. So character classes can't fully describe identifiers with emoji after all! (And in fact, many Unicode character classes were defined only for describing standard algorithms.) You need a full emoji regular expression to define such identifiers. > [2] "The formal syntax provided here captures the general intent that an identifier consists of a string of characters beginning with a letter or an ideograph, and followed by any number of letters, ideographs, digits, or underscores" (Emphases original) Unicode has an unfortunate but particular definition of "ideograph", defined in the third sense of [1]. As a rule of thumb you should always mentally replace it with "Han character", which is not an ideogram (like numerals) but a logogram. This misnomer was too old to fix so Unicode decided to go with it. [1] https://unicode.org/glossary/#ideograph https://unicode.org/glossary/#ideograph
- aragonite 3y ago
  Thanks for the very instructive reply!

lifthrasiir 3y ago

> I'll never understand why the Unicode Consortium chose to exclude emojis from the ID_START and ID_CONTINUE character classes [1] (in terms of which so many programming languages define their concept of a valid identifer). Emojis are essentially colored symbols in terms of Unicode general categories and many languages do not accept symbols in an identifier, so it's no surprise that emojis are handled equally. UTR #31 does have a profile for identifiers with emoji (see the section 7.2), but it also demonstrates why it can't be a default: there are cases where `AB` is not valid but `ABC` is valid for some characters A, B and C. So character classes can't fully describe identifiers with emoji after all! (And in fact, many Unicode character classes were defined only for describing standard algorithms.) You need a full emoji regular expression to define such identifiers. > [2] "The formal syntax provided here captures the general intent that an identifier consists of a string of characters beginning with a letter or an ideograph, and followed by any number of letters, ideographs, digits, or underscores" (Emphases original) Unicode has an unfortunate but particular definition of "ideograph", defined in the third sense of [1]. As a rule of thumb you should always mentally replace it with "Han character", which is not an ideogram (like numerals) but a logogram. This misnomer was too old to fix so Unicode decided to go with it. [1] https://unicode.org/glossary/#ideograph https://unicode.org/glossary/#ideograph

aragonite 3y ago
Thanks for the very instructive reply!