5 ms·

I wonder if the author has submitted a proposal to get the missing glyph for their name added. You don't need to be a member of the consortium to propose addin

by nemo 12y ago

I wonder if the author has submitted a proposal to get the missing glyph for their name added. You don't need to be a member of the consortium to propose adding a missing glyph/updating the standard. The point of the committee as I understand it isn't to be an expert in all forms of writing, but to take the recommendations from scholars/experts and get a working implementation, though more diverse representation of language groups would definitely be a positive change. http://unicode.org/pending/proposals.html http://unicode.org/pending/proposals.html Also, the CJK unification project sounds horrible.

vilhelm_s 12y ago
It sounds like the glyph is in unicode already, but expressed using combining characters?
- breadbox 12y ago
  Imagine if the letter Q had been left out of Unicode's Latin alphabet. The argument against it is that it can be written with a capital O combined with a comma. (That's going to play hell with naive sorting algorithms, of course, but oh well.) Oh, and also imagine your name is Quentin.
  - vilhelm_s 12y ago
    Combining characters are not just used for Bengali though. E.g. umlauted letters in European languages can also be expressed using combining characters, and implementations need to deal with those when sorting.
  - icebraining 12y ago
    But the letter wasn't left out of Unicode; it's actually typed in the article. It's just internally represented as multiple codepoints, much like one of parts of my name (é) may be. Frankly, this is irrelevant to the actual problem, which is the input system, and which has nothing to do with Unicode. Nothing prevents a single key from typing multiple codepoints at once.
    - cplease 12y ago
      > It's just internally represented as multiple codepoints And in fact it is not, and even in the article it is U+09CE. One codepoint. If his input method irks him, he's as free to tweak it as I am to switch to Dvorak. Also folks, there's no "CJK unification" project. It's Han unification. Han characters are Han characters, just like Latin characters are Latin characters. Just because German has ß and Danish has Ø doesn't mean A isn't a Latin character and not, say, a French one. Not to get all Ayn Rand-y, but A is A is U+0041 in all Western European/Latin alphabets. It makes sense for 中国 and 日本to have the same encoding in Chinese and Japanese.
      - mikekchar 12y ago
        I hate to say it, but I think the author's objections seem to stem from his lack of understanding of character encoding issues. I don't know Bengali at all and so I will try to refrain from commenting on it, but I do speak and read Japanese fluently and Han Unification is a very, very good thing. Can you imagine the absolute hell you would have to go through trying to determine if place names were the same if they used different code points for identical characters -- just because of geopolitical origins? Yes, there are some frustrating issues -- it has been historically difficult to set priorities for fonts in X and since Chinese fonts tend to have more glyphs, you often wind up with Chinese glyphs when you wanted Japanese glyphs. But this is not an encoding issue. Localization/Internationalization is really difficult. Making a separate code point for every glyph is not going to change that for the better.
        Manishearth 12y ago
        I feel that way too. The distinction between codepoint, glyph, grapheme, character, (...) is not an easy one, and that's what he seems to be stumbling over. Unicode worries itself about only some of these issues, many of the other issues are about rendering (the job of the font) or input.
  - jsmthrowaway 12y ago
    My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" Maybe that's just me. I look forward to the future where any mistake not involving a straight white Anglo-Saxon man or his customs can be built up as antiprogressive agenda, and the best advocacy is taking the people who made them down rather than fixing the problem that is the, you know, problem. (As an aside, imagine my surprise to see a Model View Culture link on HN given how much MVC absolutely hates and criticizes HN, including a weekly "worst of HN" comment dissection.)
    - ohjesusthatguy 12y ago
      "My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" " I could tell you, but I'll need $18,000 first.
      - ohjesusthatguy 12y ago
        I absolutely agree with the down-vote; why didn't you white people pull together and work something out for the white guy that has a reasonable pull request for an encoding of a language in which he probably doesn't ever write? Don't y'all un-niggers pull together?
        ohjesusthatguy 12y ago
        Just because I'm harsh doesn't mean I don't love you guys :)
        ohjesusthatguy 12y ago
  - estebank 12y ago
    > Imagine if the letter Q had been left out of Unicode's Latin alphabet. To properly write my european last name I have to press between 2 and 4 different simultaneous keys, depending on the system. Han unification is beyond misguided, but combining characters is not the problem.
    - frivoal 12y ago
      Han unification as a hole is misguided? I'll grant you that some characters which were unified probably shouldn't have been, and maybe some that some that should have been weren't, but what's the argument for the whole thing to be misguided? Should Norwegian A and English A be different Unicode code points just because Norwegian also has Ø, proving that it is a different writing system? You may want to debate whether i and ı should the same letter (they aren't), but most letters in the Turkish alphabet are the same as the letters in the English alphabet.
      - estebank 12y ago
        We'll the Turkish i/ı/I/I is I think exactly the example I would have come up with of characters that looks the same as i/I, but should have it's own code point, just like cyrillic characters have their own code points despite looking like latin characters.
        frivoal 12y ago
        Absolutely. So i/ı/I/I do have their own codepoints. But the rest of the letters, which are the same, don't. Just like han unification. Letters which are the same are the same, and those which are not are not, even if they look pretty close.
        estebank 12y ago
        The thing is that the turkish "i" and "I" don't have their own codepoints, it is the same one as latin "i" and "I", when they should have been their own codepoints representing the same glyphs. That way going from I to ı and from i to İ wouldn't be a locale dependant problem.
- deleted 12y ago
  [deleted]
- nemo 12y ago
  His writing there is pretty confusing. He started by complaining about a glyph that was missing until 2005, but was either fixed in 2005 or approximated by combining some characters 'ত + ্ + ‍ = ‍ৎ'. He doesn't really make it very clear whether ৎ is a substitute for a glyph, or whether it's the correct glyph and a case of an input system not making that easy to enter, but it seems like the glyph was added in 2005 and he's complaining about the input method. Assuming it is a case of a clunky input system, then pointing the finger at the Unicode consortium seems pretty weak, since so far as I understand it, various OS vendors/app platforms handle that implementation.
bbreier 12y ago
The author's explanation of what characters Chinese, Japanese, and Korean share is very limited. All three languages use Chinese characters in written language to varying extents, and in some cases the differences begin significantly less than a century ago. Though there are cases where the same Chinese character represented in Japanese writing is different from how it is represented in Traditional Chinese writing (i.e. 国, Japanese version only because I don't have Chinese installed on this PC), which could be different still from how it is represented in Simplified Chinese, there are also many instances where the character is identical across all three languages (i.e. 中). Although I am not privy to the specifics of the CJK unification project, identifying these cases and using the same character for them doesn't sound unreasonable. Edit- To be clear, Korean primarily uses Hangul, which basically derives jack and shit from the Chinese alphabet, and Japanese uses a mixture of the Chinese alphabet, and two alphabets that sort-of kind-of owe some of their heritage to the Chinese alphabet, but look nothing like it. If they are talking about unifying these alphabets, then they are out of their minds.
- yummyfajitas 12y ago
  Nor is it unreasonable to "unify" Latin, Greek and Cyrilic: Cyrillic ПФ vs Greek ΠΦ Cyrillic АВ vs Latin AB Obviously using ω for w (as he does) is stupid, but his reducto-ad-absurdum is not particularly absurd.
  - microcolonel 12y ago
    I think their distinct calligraphic representations mean that these would be destructive; whereas with regard to CJK, the characters are clearly represented the same way between the considered languages.
  - saalweachter 12y ago
    (For those who are rusty on their Greek, ω is a lower-case omega, and unrelated to the English/German letter w.)
  - jeorgun 12y ago
    Unicode kind of does this already with dotless 'i'; capital 'ı' and lowercase 'İ' are represented as regular latin 'I' and 'i' respectively, despite being semantically different letters.
psychometry 12y ago
I found it extremely annoying that he doesn't even say specifically what the problem with his name is. So there's a letter that's unavailable? Which letter?
- theandrewbailey 12y ago
  > Even today, I am forced to do this when writing my own name. My name is not only a common Indian name, but one of the top 1,000 names in the United States as well. But the final letter has still not been given its own Unicode character, so I have to use a substitute. Not as descriptive as it could be, but this article isn't about him.
  - BillinghamJ 12y ago
    Yet the title is solely about him.
gerbal 12y ago
Holy cow CJK unification is a terrible idea. Maybe if it originated from the CJK governments, it might be an OK idea, but the idea of a Western multinationals trying to save Unicode space by disregarding the distinctness of a whole language group is idiotic. The fundamental roll of an institution like the Unicode Consortium is to be descriptive, not prescriptive. If there is a human script, passing certain, low, low thresholds, it should be eligible to be included in its distinct whole in Unicode.
- fixermark 12y ago
  I don't disagree, while observing that it plays hell with security if (forgive my use of Latin; I don't have a codepoint translator handy) 'bat.com' and 'bat.com' are two different websites because the 'a' in the first is a Chinese-a and the 'a' in the second is a Korean-a. (Of course, this calls into question the wisdom of expanding DNS into the Unicode space in the first place---a space that does nothing like guarantee 1-to-1 association between visual glyph and code for an application that has been built on the assumption that different codes are visually distinguishable. But that ship has sailed).
- mrgriscom 12y ago
  To oppose Han unification is to say that an 'a' in English and an 'a' in French should be different code points, because they're different languages. Alternatively, if Unicode directly encoded words, rather than letters, of Western languages akin to ideographs in East Asian languages, it's like arguing that 'color' and 'colour' should be separate code points.
- astrange 12y ago
  The Han unification working group (IRG) members were in fact from Asia and appointed by their governments. Why would you think otherwise? Apparently this even includes North Korea.
discardorama 12y ago
> I wonder if the author has submitted a proposal to get the missing glyph for their name added. Probably not. If he did, he wouldn't be able to rant about the injustices of the White Man, now, would he?

by nemo 12y ago

vilhelm_s 12y ago

It sounds like the glyph is in unicode already, but expressed using combining characters?

breadbox 12y ago
Imagine if the letter Q had been left out of Unicode's Latin alphabet. The argument against it is that it can be written with a capital O combined with a comma. (That's going to play hell with naive sorting algorithms, of course, but oh well.) Oh, and also imagine your name is Quentin.
- vilhelm_s 12y ago
  Combining characters are not just used for Bengali though. E.g. umlauted letters in European languages can also be expressed using combining characters, and implementations need to deal with those when sorting.
- icebraining 12y ago
  But the letter wasn't left out of Unicode; it's actually typed in the article. It's just internally represented as multiple codepoints, much like one of parts of my name (é) may be. Frankly, this is irrelevant to the actual problem, which is the input system, and which has nothing to do with Unicode. Nothing prevents a single key from typing multiple codepoints at once.
  - cplease 12y ago
    > It's just internally represented as multiple codepoints And in fact it is not, and even in the article it is U+09CE. One codepoint. If his input method irks him, he's as free to tweak it as I am to switch to Dvorak. Also folks, there's no "CJK unification" project. It's Han unification. Han characters are Han characters, just like Latin characters are Latin characters. Just because German has ß and Danish has Ø doesn't mean A isn't a Latin character and not, say, a French one. Not to get all Ayn Rand-y, but A is A is U+0041 in all Western European/Latin alphabets. It makes sense for 中国 and 日本to have the same encoding in Chinese and Japanese.
    - mikekchar 12y ago
      I hate to say it, but I think the author's objections seem to stem from his lack of understanding of character encoding issues. I don't know Bengali at all and so I will try to refrain from commenting on it, but I do speak and read Japanese fluently and Han Unification is a very, very good thing. Can you imagine the absolute hell you would have to go through trying to determine if place names were the same if they used different code points for identical characters -- just because of geopolitical origins? Yes, there are some frustrating issues -- it has been historically difficult to set priorities for fonts in X and since Chinese fonts tend to have more glyphs, you often wind up with Chinese glyphs when you wanted Japanese glyphs. But this is not an encoding issue. Localization/Internationalization is really difficult. Making a separate code point for every glyph is not going to change that for the better.
      - Manishearth 12y ago
        I feel that way too. The distinction between codepoint, glyph, grapheme, character, (...) is not an easy one, and that's what he seems to be stumbling over. Unicode worries itself about only some of these issues, many of the other issues are about rendering (the job of the font) or input.
- jsmthrowaway 12y ago
  My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" Maybe that's just me. I look forward to the future where any mistake not involving a straight white Anglo-Saxon man or his customs can be built up as antiprogressive agenda, and the best advocacy is taking the people who made them down rather than fixing the problem that is the, you know, problem. (As an aside, imagine my surprise to see a Model View Culture link on HN given how much MVC absolutely hates and criticizes HN, including a weekly "worst of HN" comment dissection.)
  - ohjesusthatguy 12y ago
    "My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" " I could tell you, but I'll need $18,000 first.
    - ohjesusthatguy 12y ago
      I absolutely agree with the down-vote; why didn't you white people pull together and work something out for the white guy that has a reasonable pull request for an encoding of a language in which he probably doesn't ever write? Don't y'all un-niggers pull together?
      - ohjesusthatguy 12y ago
        Just because I'm harsh doesn't mean I don't love you guys :)
        ohjesusthatguy 12y ago
- estebank 12y ago
  > Imagine if the letter Q had been left out of Unicode's Latin alphabet. To properly write my european last name I have to press between 2 and 4 different simultaneous keys, depending on the system. Han unification is beyond misguided, but combining characters is not the problem.
  - frivoal 12y ago
    Han unification as a hole is misguided? I'll grant you that some characters which were unified probably shouldn't have been, and maybe some that some that should have been weren't, but what's the argument for the whole thing to be misguided? Should Norwegian A and English A be different Unicode code points just because Norwegian also has Ø, proving that it is a different writing system? You may want to debate whether i and ı should the same letter (they aren't), but most letters in the Turkish alphabet are the same as the letters in the English alphabet.
    - estebank 12y ago
      We'll the Turkish i/ı/I/I is I think exactly the example I would have come up with of characters that looks the same as i/I, but should have it's own code point, just like cyrillic characters have their own code points despite looking like latin characters.
      - frivoal 12y ago
        Absolutely. So i/ı/I/I do have their own codepoints. But the rest of the letters, which are the same, don't. Just like han unification. Letters which are the same are the same, and those which are not are not, even if they look pretty close.
        estebank 12y ago
        The thing is that the turkish "i" and "I" don't have their own codepoints, it is the same one as latin "i" and "I", when they should have been their own codepoints representing the same glyphs. That way going from I to ı and from i to İ wouldn't be a locale dependant problem.
deleted 12y ago
[deleted]
nemo 12y ago
His writing there is pretty confusing. He started by complaining about a glyph that was missing until 2005, but was either fixed in 2005 or approximated by combining some characters 'ত + ্ + ‍ = ‍ৎ'. He doesn't really make it very clear whether ৎ is a substitute for a glyph, or whether it's the correct glyph and a case of an input system not making that easy to enter, but it seems like the glyph was added in 2005 and he's complaining about the input method. Assuming it is a case of a clunky input system, then pointing the finger at the Unicode consortium seems pretty weak, since so far as I understand it, various OS vendors/app platforms handle that implementation.

bbreier 12y ago

The author's explanation of what characters Chinese, Japanese, and Korean share is very limited. All three languages use Chinese characters in written language to varying extents, and in some cases the differences begin significantly less than a century ago. Though there are cases where the same Chinese character represented in Japanese writing is different from how it is represented in Traditional Chinese writing (i.e. 国, Japanese version only because I don't have Chinese installed on this PC), which could be different still from how it is represented in Simplified Chinese, there are also many instances where the character is identical across all three languages (i.e. 中). Although I am not privy to the specifics of the CJK unification project, identifying these cases and using the same character for them doesn't sound unreasonable. Edit- To be clear, Korean primarily uses Hangul, which basically derives jack and shit from the Chinese alphabet, and Japanese uses a mixture of the Chinese alphabet, and two alphabets that sort-of kind-of owe some of their heritage to the Chinese alphabet, but look nothing like it. If they are talking about unifying these alphabets, then they are out of their minds.

yummyfajitas 12y ago
Nor is it unreasonable to "unify" Latin, Greek and Cyrilic: Cyrillic ПФ vs Greek ΠΦ Cyrillic АВ vs Latin AB Obviously using ω for w (as he does) is stupid, but his reducto-ad-absurdum is not particularly absurd.
- microcolonel 12y ago
  I think their distinct calligraphic representations mean that these would be destructive; whereas with regard to CJK, the characters are clearly represented the same way between the considered languages.
- saalweachter 12y ago
  (For those who are rusty on their Greek, ω is a lower-case omega, and unrelated to the English/German letter w.)
- jeorgun 12y ago
  Unicode kind of does this already with dotless 'i'; capital 'ı' and lowercase 'İ' are represented as regular latin 'I' and 'i' respectively, despite being semantically different letters.

psychometry 12y ago

I found it extremely annoying that he doesn't even say specifically what the problem with his name is. So there's a letter that's unavailable? Which letter?

theandrewbailey 12y ago
> Even today, I am forced to do this when writing my own name. My name is not only a common Indian name, but one of the top 1,000 names in the United States as well. But the final letter has still not been given its own Unicode character, so I have to use a substitute. Not as descriptive as it could be, but this article isn't about him.
- BillinghamJ 12y ago
  Yet the title is solely about him.

gerbal 12y ago

Holy cow CJK unification is a terrible idea. Maybe if it originated from the CJK governments, it might be an OK idea, but the idea of a Western multinationals trying to save Unicode space by disregarding the distinctness of a whole language group is idiotic. The fundamental roll of an institution like the Unicode Consortium is to be descriptive, not prescriptive. If there is a human script, passing certain, low, low thresholds, it should be eligible to be included in its distinct whole in Unicode.

fixermark 12y ago
I don't disagree, while observing that it plays hell with security if (forgive my use of Latin; I don't have a codepoint translator handy) 'bat.com' and 'bat.com' are two different websites because the 'a' in the first is a Chinese-a and the 'a' in the second is a Korean-a. (Of course, this calls into question the wisdom of expanding DNS into the Unicode space in the first place---a space that does nothing like guarantee 1-to-1 association between visual glyph and code for an application that has been built on the assumption that different codes are visually distinguishable. But that ship has sailed).
mrgriscom 12y ago
To oppose Han unification is to say that an 'a' in English and an 'a' in French should be different code points, because they're different languages. Alternatively, if Unicode directly encoded words, rather than letters, of Western languages akin to ideographs in East Asian languages, it's like arguing that 'color' and 'colour' should be separate code points.
astrange 12y ago
The Han unification working group (IRG) members were in fact from Asia and appointed by their governments. Why would you think otherwise? Apparently this even includes North Korea.

discardorama 12y ago

> I wonder if the author has submitted a proposal to get the missing glyph for their name added. Probably not. If he did, he wouldn't be able to rant about the injustices of the White Man, now, would he?