10 ms·

> > It seems like you're trying to single out combining pairs as "less legitimate" when they're extensively used in the standard. > I'm saying that Unicode on

> > It seems like you're trying to single out combining pairs as "less legitimate" when they're extensively used in the standard. > I'm saying that Unicode only does it in English where it makes semantic sense to a native English speaker. Well, combining characters almost never come up in English. The best I can think of would be the use of cedillas, diaereses, and acute accents in words like façade, coördinate and renownèd (I've been reading Tolkien's translation of Beowulf, and he used renownèd a lot). Thinking about the Spanish I learned in high school, ch, ll, ñ, and rr are all considered separate letters (i.e., the Spanish alphabet has 30 letters; ch is between c and d, ll is between l and m, ñ is between n and o, and rr is between r and s; interestingly, accented vowels aren't separate letters). Unicode does not provide code points for ch, ll, or rr; and ñ has a code point more from historical accident than anything (the decision to start with Latin1). Then again, I don't think Spanish keyboards have separate keys for ch, ll, or rr. Portuguese, on the other hand, doesn't officially include k or y in the alphabet. But it uses far more accents than Spanish. So, a, ã and á are all the same letter. In a perfect world, how would Unicode handle this? Either they accept the Spanish view of the world, or the Portuguese view. Or, perhaps, they make a big deal about not worrying about languages and instead worrying about alphabets ( http://www.unicode.org/faq/basic_q.html#4 http://www.unicode.org/faq/basic_q.html#4 ). They haven't been perfect. And they've certainly changed their approach over time. And I suspect they're including emoji to appear more welcoming to Japanese teenagers than they were in the past. But (1) combining characters aren't second-class citizens, and (2) the standard is still open to revisions ( http://www.unicode.org/alloc/Pipeline.html http://www.unicode.org/alloc/Pipeline.html ).

darklajid 12y ago
I'm coming from a German background and I sympathize with the author. German has 4 (7 if you consider cases) non-ASCII characters: äüöß(and upper-case umlauts). All of these are unique, well-defined codepoints. That's not related to composing on a keyboard. In fact, although I'm German I'm using the US keyboard layout and HAD to compose these characters now. But I wouldn't need to and the result is a single codepoint again..
- masklinn 12y ago
  > But I wouldn't need to and the result is a single codepoint again.. Doesn't have to be though, it'd be perfectly correct for an IME to generate multiple codepoints. IIRC, that's what you'd get if you typed those in a filename on OSX then asked for the native file path, as HFS+ stores filenames in NFD. Meanwhile Safari does (used to do?) the opposite, text is automatically NFC'd before sending. Things get interesting when you don't expect it and don't do unicode-equivalent comparisons.
- vilhelm_s 12y ago
  Well, they have codepoints, but not unique ones (since they can be written both using combining characters or using the compatibility pre-combined form). Software libraries dealing with unicode strings needs to handle both versions, by applying unicode normalization before doing comparisons. The reason they have two representations is for backwards compatibility with previous character encoding standards, but the unicode standard is more complex because of this (it needs to specify more equivalences for normalization). I guess for languages which were not previously covered by any standards, the unicode consortium tries to represent things "as uniquely as possible".
- the_mitsuhiko 12y ago
  > German has 4 (7 if you consider cases) non-ASCII characters: äüöß(and upper-case umlauts). All of these are unique, well-defined codepoints. German does not consider "ä", "ö" and "ü" letters. Our alphabet has 26 letters none of which are the ones you mentioned. In fact, if you go back in History it becomes even clearer that those letters used to be ligatures in writing. They still are collated as the basic letters the represent, even if they sound different. That we use the uncomposed representation in Unicode usually, is merely a historical artifact because of iso-8859-1 and others, not because it logically makes sense. When you used an old typewriter you usually did not have those keys either, you composed them.
  - darklajid 12y ago
    One by one: I'm confused by your use of 'our' and 'we'. It seems you're trying to write from the general point of view of a German, answering .. a German? Are umlauts letters? Yes. [1] [2] Maybe not the best source, but please provide a better one if you disagree so that I can actually understand where you're coming from. I understand - I hope? - composition. And I tend to agree that it shouldn't matter much if the input just works. If I press a key labeled ü and that letter shows up on the screen, I shouldn't really care if that is one codepoint or a composition of two (or more). I do think that the history you mention is an indicator that supports the author's argument. There IS a codepoint for ü (painful to type..). For 'legacy reasons' perhaps. And it feels to me that non-ASCII characters - for legacy reasons or whatever - have better support than the ones he is complaining about, if they originate in western Europe/in my home country. Typewriters and umlauts: http://i.ebayimg.com/00/s/Mzk2WDQwMA==/$T2eC16N,!)sE9swmYlFPBP8DdJ8zFQ~~48_20.JPG http://i.ebayimg.com/00/s/Mzk2WDQwMA==/$T2eC16N,!)sE9swmYlFP... (basically I searched for old typewriter models, 'Adler Schreibmaschinen' results in lots of hits like that). Note the separate umlaut keys. And these are typewriters from .. the 60s? Maybe?) 1: https://de.wikipedia.org/wiki/Alphabet https://de.wikipedia.org/wiki/Alphabet 2: https://de.wikipedia.org/wiki/Deutsches_Alphabet https://de.wikipedia.org/wiki/Deutsches_Alphabet
nova 12y ago
> ch, ll, ñ, and rr are all considered separate letters In Spanish "rr" has never been considered as a single letter. "Ch" and "ll" used to be, but not anymore. Ñ is, of course.
- maxlybbert 12y ago
  That's funny; I spent several hours of class time trilling r's to make sure we pronounced "carro" correctly, and repeating a 30 character alphabet.
  - jimktrains2 12y ago
    rr not being its own letter has no bearing on if you can pronounce carro correctly, just like saying church right has no bearing on if c and h are two letters or ch is a single letter.
    - maxlybbert 12y ago
      What can I say? Apparently the textbook was wrong.
- k0ga 12y ago
  I think i'm older than you. I learnt in the school they were different letters, and also I remember when they were removed at the beginning of 90's
- elatahualpa 12y ago
  I'm afraid that I have to add my voice to the list of people raised in Spanish speaking countries prior to the 90s who was VERY clearly taught that rr was a separate letter. This is what I recall from my childhood: http://img.docstoccdn.com/thumb/orig/113964108.png http://img.docstoccdn.com/thumb/orig/113964108.png
Maken 12y ago
Spanish speaker here. "ch" and "ll" being separate letters has been discussed for a long time and finally the decision was that they weren't separate letters but a combination of two [1]. Meanwhile, "ñ" stands as a letter of its own. Accented vowels aren't considered different letters in Spanish because they affect the word they are in rather than the letter, as they serve to indicate which one is the "strong" syllable in a word. From a Spanish view of point, "a" and "á" are exactly the same letter. [1] http://www.rae.es/consultas/exclusion-de-ch-y-ll-del-abecedario http://www.rae.es/consultas/exclusion-de-ch-y-ll-del-abeceda...
- maxlybbert 12y ago
  That's news to me. Perhaps I'll have better luck finding words like "chancho" in a dictionary; I'll be right to look in the c's!
elros 12y ago
> Portuguese, on the other hand, doesn't officially include k > or y in the alphabet. With no judgement towards your broader point, I'd like to point out that this is no longer the case as of the orthographic agreement of 1990[0]. As far as I know it's been added back in order to better suit African speakers. [0] https://pt.wikipedia.org/wiki/Acordo_Ortogr%C3%A1fico_de_1990 https://pt.wikipedia.org/wiki/Acordo_Ortogr%C3%A1fico_de_199...
- maxlybbert 12y ago
  That's good to know. I learned Portuguese in '97-'99, so the information I had was incorrect at the time. We Anericans always recited the alphabet with k and y, but our teacher said they weren't official (although he also said that Brazilians would recognize them).

darklajid 12y ago

I'm coming from a German background and I sympathize with the author. German has 4 (7 if you consider cases) non-ASCII characters: äüöß(and upper-case umlauts). All of these are unique, well-defined codepoints. That's not related to composing on a keyboard. In fact, although I'm German I'm using the US keyboard layout and HAD to compose these characters now. But I wouldn't need to and the result is a single codepoint again..

masklinn 12y ago
> But I wouldn't need to and the result is a single codepoint again.. Doesn't have to be though, it'd be perfectly correct for an IME to generate multiple codepoints. IIRC, that's what you'd get if you typed those in a filename on OSX then asked for the native file path, as HFS+ stores filenames in NFD. Meanwhile Safari does (used to do?) the opposite, text is automatically NFC'd before sending. Things get interesting when you don't expect it and don't do unicode-equivalent comparisons.
vilhelm_s 12y ago
Well, they have codepoints, but not unique ones (since they can be written both using combining characters or using the compatibility pre-combined form). Software libraries dealing with unicode strings needs to handle both versions, by applying unicode normalization before doing comparisons. The reason they have two representations is for backwards compatibility with previous character encoding standards, but the unicode standard is more complex because of this (it needs to specify more equivalences for normalization). I guess for languages which were not previously covered by any standards, the unicode consortium tries to represent things "as uniquely as possible".
the_mitsuhiko 12y ago
> German has 4 (7 if you consider cases) non-ASCII characters: äüöß(and upper-case umlauts). All of these are unique, well-defined codepoints. German does not consider "ä", "ö" and "ü" letters. Our alphabet has 26 letters none of which are the ones you mentioned. In fact, if you go back in History it becomes even clearer that those letters used to be ligatures in writing. They still are collated as the basic letters the represent, even if they sound different. That we use the uncomposed representation in Unicode usually, is merely a historical artifact because of iso-8859-1 and others, not because it logically makes sense. When you used an old typewriter you usually did not have those keys either, you composed them.
- darklajid 12y ago
  One by one: I'm confused by your use of 'our' and 'we'. It seems you're trying to write from the general point of view of a German, answering .. a German? Are umlauts letters? Yes. [1] [2] Maybe not the best source, but please provide a better one if you disagree so that I can actually understand where you're coming from. I understand - I hope? - composition. And I tend to agree that it shouldn't matter much if the input just works. If I press a key labeled ü and that letter shows up on the screen, I shouldn't really care if that is one codepoint or a composition of two (or more). I do think that the history you mention is an indicator that supports the author's argument. There IS a codepoint for ü (painful to type..). For 'legacy reasons' perhaps. And it feels to me that non-ASCII characters - for legacy reasons or whatever - have better support than the ones he is complaining about, if they originate in western Europe/in my home country. Typewriters and umlauts: http://i.ebayimg.com/00/s/Mzk2WDQwMA==/$T2eC16N,!)sE9swmYlFPBP8DdJ8zFQ~~48_20.JPG http://i.ebayimg.com/00/s/Mzk2WDQwMA==/$T2eC16N,!)sE9swmYlFP... (basically I searched for old typewriter models, 'Adler Schreibmaschinen' results in lots of hits like that). Note the separate umlaut keys. And these are typewriters from .. the 60s? Maybe?) 1: https://de.wikipedia.org/wiki/Alphabet https://de.wikipedia.org/wiki/Alphabet 2: https://de.wikipedia.org/wiki/Deutsches_Alphabet https://de.wikipedia.org/wiki/Deutsches_Alphabet

nova 12y ago

> ch, ll, ñ, and rr are all considered separate letters In Spanish "rr" has never been considered as a single letter. "Ch" and "ll" used to be, but not anymore. Ñ is, of course.

maxlybbert 12y ago
That's funny; I spent several hours of class time trilling r's to make sure we pronounced "carro" correctly, and repeating a 30 character alphabet.
- jimktrains2 12y ago
  rr not being its own letter has no bearing on if you can pronounce carro correctly, just like saying church right has no bearing on if c and h are two letters or ch is a single letter.
  - maxlybbert 12y ago
    What can I say? Apparently the textbook was wrong.
k0ga 12y ago
I think i'm older than you. I learnt in the school they were different letters, and also I remember when they were removed at the beginning of 90's
elatahualpa 12y ago
I'm afraid that I have to add my voice to the list of people raised in Spanish speaking countries prior to the 90s who was VERY clearly taught that rr was a separate letter. This is what I recall from my childhood: http://img.docstoccdn.com/thumb/orig/113964108.png http://img.docstoccdn.com/thumb/orig/113964108.png

Maken 12y ago

Spanish speaker here. "ch" and "ll" being separate letters has been discussed for a long time and finally the decision was that they weren't separate letters but a combination of two [1]. Meanwhile, "ñ" stands as a letter of its own. Accented vowels aren't considered different letters in Spanish because they affect the word they are in rather than the letter, as they serve to indicate which one is the "strong" syllable in a word. From a Spanish view of point, "a" and "á" are exactly the same letter. [1] http://www.rae.es/consultas/exclusion-de-ch-y-ll-del-abecedario http://www.rae.es/consultas/exclusion-de-ch-y-ll-del-abeceda...

maxlybbert 12y ago
That's news to me. Perhaps I'll have better luck finding words like "chancho" in a dictionary; I'll be right to look in the c's!

elros 12y ago

> Portuguese, on the other hand, doesn't officially include k > or y in the alphabet. With no judgement towards your broader point, I'd like to point out that this is no longer the case as of the orthographic agreement of 1990[0]. As far as I know it's been added back in order to better suit African speakers. [0] https://pt.wikipedia.org/wiki/Acordo_Ortogr%C3%A1fico_de_1990 https://pt.wikipedia.org/wiki/Acordo_Ortogr%C3%A1fico_de_199...

maxlybbert 12y ago
That's good to know. I learned Portuguese in '97-'99, so the information I had was incorrect at the time. We Anericans always recited the alphabet with k and y, but our teacher said they weren't official (although he also said that Brazilians would recognize them).