6 ms·

They want every symbol used in a document to have a unique encoding, so that you can change fonts without losing meaning. Fonts like wingdings are a horrible ha

by wmil 10y ago

They want every symbol used in a document to have a unique encoding, so that you can change fonts without losing meaning. Fonts like wingdings are a horrible hack. The idea is one (complex) encoding that will represent the info until the end of time. It creates a lot of trouble, but it's still a good idea.

thaumasiotes 10y ago
Technically, glyphs are supposed to meet some standards, like being shown in use in running text, before they can be added to unicode. It's not supposed to be a repository of every picture anyone ever dreamed up. The standards are not applied consistently. Even leaving emoji out of it, the chinese "character" 囍 never occurs in running text, but there it is in unicode.
- ximeng 10y ago
  I don't think it's true that 囍 never occurs in running text - it's used in company names which would be used in text. It would be odd not to have an encoding for such a common character.
  - thaumasiotes 10y ago
    All right, I spent some time trying to find the requirement. I did not find it, but my tentative conclusion is that it does not apply to chinese characters. FROM MEMORY, a while back there was an article on HN complaining that emoji seemed to magically bypass the requirements other characters needed to meet for inclusion in unicode, and that in fact they were commonly in violation. The taco symbol was called out as an example. I can no longer find this article, but it mentioned the running text requirement, and -- I believe -- specifically indicated that use in names does not count as use in running text. (For an idea of why that might be the case, check out http://tvtropes.org/pmwiki/pmwiki.php/Main/LuckyCharmsTitle http://tvtropes.org/pmwiki/pmwiki.php/Main/LuckyCharmsTitle .) HOWEVER, I was not even able to find, on the unicode web site, any discussion of a running text requirement at all, for any kind of symbol. Some example proposals do refer to "running text" by name, but they don't indicate why. The example proposal given for adding characters to an existing block ( http://std.dkuug.dk/jtc1/sc2/wg2/docs/n2934.pdf http://std.dkuug.dk/jtc1/sc2/wg2/docs/n2934.pdf , suggested as a prototype in http://www.unicode.org/faq/char_proposal.html http://www.unicode.org/faq/char_proposal.html ) does not mention "running text" at all, and doesn't appear to go to much trouble to document it, although some such documentation is given. The rough guidelines for character proposals at http://unicode.org/pending/proposals.html http://unicode.org/pending/proposals.html do not refer to "running text" at all, but they do suggest that, late in the process (specifically, on a proposal summary form, which is different from, and subsequent to, an actual proposal), "references to dictionaries and descriptive texts establishing authoritative information" are required. I conclude that the Unicode standard's preferred criterion for chinese character inclusion is "would an authoritative chinese dictionary include this character", and while the answer to that question for 囍 is not unambiguous -- a lot of dictionaries don't include it -- it's easy to imagine that some do. I would appreciate a pointer to the actual running text requirements, as well as what they are supposed to apply to, if anyone can provide that. > It would be odd not to have an encoding for such a common character. Outside of its use as a wedding decoration, which is plainly nonlinguistic, how common is it?
    - barkingcat 10y ago
      That character is extremely common. It's not just a one off wedding celebration thing. Also used in new year celebrations. Basically every single year you'd see tons of these printed. https://en.wikipedia.org/wiki/Double_Happiness_(calligraphy) https://en.wikipedia.org/wiki/Double_Happiness_(calligraphy) It's a pun actually, double happiness.
      - thaumasiotes 10y ago
        Every year you see tons of 福倒 printed too (福, but upside down). That has yet to receive a code point.
        msbarnett 10y ago
        You're not wrong, but I don't know if it's a great example. CJK characters are, broadly, an example of the Unicode Consortium trying to be way too reductive about what they'd accept, leading to a lot of bad decisions like Han Unification, which caused a lot of damage and which the Consortium has generally now backed away from and recognized as a bad idea. So, yes, if you look closely at CJK character sets in Unicode, you can find a lot of decision making that appears to contradict decision making elsewhere in the standard. This is in large part because the decisions they made wrt CJK characters turned out to be largely wrong, and they've since changed their approach.
        thaumasiotes 10y ago
        Unification is a mistake. But 囍 has nothing to do with unification. Do you think that 福倒 should have a code point? Would it be considered one of the chinese characters (very iffy) or one of the holiday symbols? The 天书 ( https://en.wikipedia.org/wiki/A_Book_from_the_Sky https://en.wikipedia.org/wiki/A_Book_from_the_Sky ), by design, consists solely of chinese characters that don't exist. (Theoretically. A couple of them, by oversight, did exist.) They are still recognizably "chinese characters" by virtue of being composed of the same components. Should they have unicode points? 囍 plainly exists, but has no textual use. Is it more similar to 靑 or to ️U+2764 "heavy black heart"?
- barkingcat 10y ago
  That character is super common. Way more common than any emoji I'd expect (even teens using emojis won't outweigh the number of weddings and new years in Chinese speaking countries).
  - thaumasiotes 10y ago
    The question was whether it can meet the Unicode inclusion requirements, not whether it's common.
    - barkingcat 10y ago
      Well, even if it doesn't meet unicode inclusion requirements, it is necessary for printing in one of the largest markets in the world. Without that character in unicode, Chinese display systems and printers probably won't use unicode at all (and before unicode they used some standard of their own) - meaning the question is whether unicode wants to be relevant or not, not whether the inclusion requirements fit.
      - thaumasiotes 10y ago
        Again, it's not a character that appears in text. This: > Without that character in unicode, Chinese display systems and printers probably won't use unicode at all is baseless. In its current uses, it doesn't appear on display systems and when printed it is almost always designed as an image, not printed as part of a font. Compare: http://pic10.nipic.com/20100928/5211371_231333032314_2.jpg http://pic10.nipic.com/20100928/5211371_231333032314_2.jpg
        ximeng 10y ago
        This is used in discussions of the character, would you not consider that text? it does seem to have a more figurative than literal reference than most characters, in a way that I am not sure how to translate into English.
        10y ago
- kens 10y ago
  The character 囍 appears in other standards (big5, JIS, ISO2022_JP), so Unicode is basically obligated to include it for compatibility. The running text requirement is for symbols, so it doesn't apply to Chinese: http://www.unicode.org/pending/symbol-guidelines.html http://www.unicode.org/pending/symbol-guidelines.html
  - thaumasiotes 10y ago
    As to compatibility, excellent point. The word "running" doesn't appear on that page. (Actually, no requirements at all appear on that page; it speaks strictly in terms of strengthening or weakening the case for inclusion, not disqualifying.) Can you explain briefly why that page is evidence that the running text requirement does not apply to Chinese, and where it specifies what the running text requirement is? Alternatively, what requirements do apply to Chinese, and would they preclude an invented character like one with 女 on the left and 离 on the right?
    - kens 10y ago
      I thought the symbol guideline page discussed "running text", but I guess not. Apparently the "running text" requirement isn't part of the published criteria even though it is enforced in discussion.

by wmil 10y ago

thaumasiotes 10y ago

Technically, glyphs are supposed to meet some standards, like being shown in use in running text, before they can be added to unicode. It's not supposed to be a repository of every picture anyone ever dreamed up. The standards are not applied consistently. Even leaving emoji out of it, the chinese "character" 囍 never occurs in running text, but there it is in unicode.

ximeng 10y ago
I don't think it's true that 囍 never occurs in running text - it's used in company names which would be used in text. It would be odd not to have an encoding for such a common character.
- thaumasiotes 10y ago
  All right, I spent some time trying to find the requirement. I did not find it, but my tentative conclusion is that it does not apply to chinese characters. FROM MEMORY, a while back there was an article on HN complaining that emoji seemed to magically bypass the requirements other characters needed to meet for inclusion in unicode, and that in fact they were commonly in violation. The taco symbol was called out as an example. I can no longer find this article, but it mentioned the running text requirement, and -- I believe -- specifically indicated that use in names does not count as use in running text. (For an idea of why that might be the case, check out http://tvtropes.org/pmwiki/pmwiki.php/Main/LuckyCharmsTitle http://tvtropes.org/pmwiki/pmwiki.php/Main/LuckyCharmsTitle .) HOWEVER, I was not even able to find, on the unicode web site, any discussion of a running text requirement at all, for any kind of symbol. Some example proposals do refer to "running text" by name, but they don't indicate why. The example proposal given for adding characters to an existing block ( http://std.dkuug.dk/jtc1/sc2/wg2/docs/n2934.pdf http://std.dkuug.dk/jtc1/sc2/wg2/docs/n2934.pdf , suggested as a prototype in http://www.unicode.org/faq/char_proposal.html http://www.unicode.org/faq/char_proposal.html ) does not mention "running text" at all, and doesn't appear to go to much trouble to document it, although some such documentation is given. The rough guidelines for character proposals at http://unicode.org/pending/proposals.html http://unicode.org/pending/proposals.html do not refer to "running text" at all, but they do suggest that, late in the process (specifically, on a proposal summary form, which is different from, and subsequent to, an actual proposal), "references to dictionaries and descriptive texts establishing authoritative information" are required. I conclude that the Unicode standard's preferred criterion for chinese character inclusion is "would an authoritative chinese dictionary include this character", and while the answer to that question for 囍 is not unambiguous -- a lot of dictionaries don't include it -- it's easy to imagine that some do. I would appreciate a pointer to the actual running text requirements, as well as what they are supposed to apply to, if anyone can provide that. > It would be odd not to have an encoding for such a common character. Outside of its use as a wedding decoration, which is plainly nonlinguistic, how common is it?
  - barkingcat 10y ago
    That character is extremely common. It's not just a one off wedding celebration thing. Also used in new year celebrations. Basically every single year you'd see tons of these printed. https://en.wikipedia.org/wiki/Double_Happiness_(calligraphy) https://en.wikipedia.org/wiki/Double_Happiness_(calligraphy) It's a pun actually, double happiness.
    - thaumasiotes 10y ago
      Every year you see tons of 福倒 printed too (福, but upside down). That has yet to receive a code point.
      - msbarnett 10y ago
        You're not wrong, but I don't know if it's a great example. CJK characters are, broadly, an example of the Unicode Consortium trying to be way too reductive about what they'd accept, leading to a lot of bad decisions like Han Unification, which caused a lot of damage and which the Consortium has generally now backed away from and recognized as a bad idea. So, yes, if you look closely at CJK character sets in Unicode, you can find a lot of decision making that appears to contradict decision making elsewhere in the standard. This is in large part because the decisions they made wrt CJK characters turned out to be largely wrong, and they've since changed their approach.
        thaumasiotes 10y ago
        Unification is a mistake. But 囍 has nothing to do with unification. Do you think that 福倒 should have a code point? Would it be considered one of the chinese characters (very iffy) or one of the holiday symbols? The 天书 ( https://en.wikipedia.org/wiki/A_Book_from_the_Sky https://en.wikipedia.org/wiki/A_Book_from_the_Sky ), by design, consists solely of chinese characters that don't exist. (Theoretically. A couple of them, by oversight, did exist.) They are still recognizably "chinese characters" by virtue of being composed of the same components. Should they have unicode points? 囍 plainly exists, but has no textual use. Is it more similar to 靑 or to ️U+2764 "heavy black heart"?
barkingcat 10y ago
That character is super common. Way more common than any emoji I'd expect (even teens using emojis won't outweigh the number of weddings and new years in Chinese speaking countries).
- thaumasiotes 10y ago
  The question was whether it can meet the Unicode inclusion requirements, not whether it's common.
  - barkingcat 10y ago
    Well, even if it doesn't meet unicode inclusion requirements, it is necessary for printing in one of the largest markets in the world. Without that character in unicode, Chinese display systems and printers probably won't use unicode at all (and before unicode they used some standard of their own) - meaning the question is whether unicode wants to be relevant or not, not whether the inclusion requirements fit.
    - thaumasiotes 10y ago
      Again, it's not a character that appears in text. This: > Without that character in unicode, Chinese display systems and printers probably won't use unicode at all is baseless. In its current uses, it doesn't appear on display systems and when printed it is almost always designed as an image, not printed as part of a font. Compare: http://pic10.nipic.com/20100928/5211371_231333032314_2.jpg http://pic10.nipic.com/20100928/5211371_231333032314_2.jpg
      - ximeng 10y ago
        This is used in discussions of the character, would you not consider that text? it does seem to have a more figurative than literal reference than most characters, in a way that I am not sure how to translate into English.
        10y ago
kens 10y ago
The character 囍 appears in other standards (big5, JIS, ISO2022_JP), so Unicode is basically obligated to include it for compatibility. The running text requirement is for symbols, so it doesn't apply to Chinese: http://www.unicode.org/pending/symbol-guidelines.html http://www.unicode.org/pending/symbol-guidelines.html
- thaumasiotes 10y ago
  As to compatibility, excellent point. The word "running" doesn't appear on that page. (Actually, no requirements at all appear on that page; it speaks strictly in terms of strengthening or weakening the case for inclusion, not disqualifying.) Can you explain briefly why that page is evidence that the running text requirement does not apply to Chinese, and where it specifies what the running text requirement is? Alternatively, what requirements do apply to Chinese, and would they preclude an invented character like one with 女 on the left and 离 on the right?
  - kens 10y ago
    I thought the symbol guideline page discussed "running text", but I guess not. Apparently the "running text" requirement isn't part of the published criteria even though it is enforced in discussion.