4 ms·

That is not the compromise struck, though; there are even many Cyrillic glyphs that are visually identical to those in Latin, but assigned differing codepoints.

jhanschoo 10y ago
There are multiple reasons for that, one of which is compatibility with previous encodings and standards. If a previous encoding Unicode wanted to be compatible with encoded these as different characters, Unicode needs these to have separate code points for them too.
- hvidgaard 10y ago
  That is the surefire way to incorporate complexities from 2 different systems into 1.
  - xyproto 10y ago
    Being able to easily check if a letter is between 'a' and 'z' in code is an advantage. This is only possible if the codepoints are sequential.
    - hvidgaard 10y ago
      I didn't dispute that. I just state that trying to remain compatible for the sake of being compatible is a great way to design a convuluted and difficult to understand standard.
      - gpderetta 10y ago
        Of course, but lack backward compatibility is a great way to make sure a standard is not adopted. For example he reason that UTF-8 'won' is that it has a great backward compatibility story with other ASCII based encodings and systems.
    - bambax 10y ago
      Is it? Couldn't Unicode have pointers or links, where a codepoint "exists" with no content and only links to another? (I don't know anything about Unicode, so maybe it already has that.)
      - edent 10y ago
        Semantically, yes. In the code tables you'll see that that "opposite" symbols have links to each other. Programatically, it is much easier to say "does a character lie between 0x12 and 0xBC" than to create a function like `isSymbolForTrafficInEurope()`
        Natanael_L 10y ago
        Perhaps Unicode could just have tables listing all relevant sequences of symbols, instead. So "latin letters lowercase" would list the codepoints for a-z in order, for example. Would no longer matter if the codepoints themselves are sequential or not. (And relevant to my country, "Swedish characters lowercase" would map to latin letters lowercase + åäö.)
        ygra 10y ago
        Characters have a script associated with them (e.g. Latin), and caseness is also part of a character's properties. Now, language-specific subsets¹ of those are a bit iffy to deal with. Especially when text can contain loan words from other languages, so in my experience it's rarely a useful thing to ask for. ¹ Yes, subsets. Latin letters lowercase is not the set abcdefghijklmnopqrstuvwxyz. It is the set abcdefghijklmnopqrstuvwxyzªºßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ āăąćĉċčďđēĕėęěĝğġģĥħĩīĭįıĳĵķĸĺļľŀłńņňŉŋōŏőœŕŗřśŝşšţťŧũūŭůűųŵ ŷźżžſƀƃƅƈƌƍƒƕƙƚƛƞơƣƥƨƪƫƭưƴƶƹƺƽƾƿǆǉǌǎǐǒǔǖǘǚǜǝǟǡǣǥǧǩǫǭǯǰǳǵǹǻǽǿ ȁȃȅȇȉȋȍȏȑȓȕȗșțȝȟȡȣȥȧȩȫȭȯȱȳȴȵȶȷȸȹȼȿɀɂɇɉɋɍɏɐɑɒɓɔɕɖɗɘəɚɛɜɝɞɟɠɡɢ ɣɤɥɦɧɨɩɪɫɬɭɮɯɰɱɲɳɴɵɶɷɸɹɺɻɼɽɾɿʀʁʂʃʄʅʆʇʈʉʊʋʌʍʎʏʐʑʒʓʕʖʗʘʙʚʛʜʝʞʟ ʠʡʢʣʤʥʦʧʨʩʪʫʬʭʮʯʰʱʲʳʴʵʶʷʸˠˡˢˣˤᴀᴁᴂᴃᴄᴅᴆᴇᴈᴉᴊᴋᴌᴍᴎᴏᴐᴑᴒᴓᴔᴕᴖᴗᴘᴙᴚᴛᴜᴝ ᴞᴟᴠᴡᴢᴣᴤᴥᴬᴭᴮᴯᴰᴱᴲᴳᴴᴵᴶᴷᴸᴹᴺᴻᴼᴽᴾᴿᵀᵁᵂᵃᵄᵅᵆᵇᵈᵉᵊᵋᵌᵍᵎᵏᵐᵑᵒᵓᵔᵕᵖᵗᵘᵙᵚᵛᵜᵢᵣᵤ ᵥᵫᵬᵭᵮᵯᵰᵱᵲᵳᵴᵵᵶᵷᵹᵺᵻᵼᵽᵾᵿᶀᶁᶂᶃᶄᶅᶆᶇᶈᶉᶊᶋᶌᶍᶎᶏᶐᶑᶒᶓᶔᶕᶖᶗᶘᶙᶚᶛᶜᶝᶞᶟᶠᶡᶢᶣᶤᶥᶦ ᶧᶨᶩᶪᶫᶬᶭᶮᶯᶰᶱᶲᶳᶴᶵᶶᶷᶸᶹᶺᶻᶼᶽᶾḁḃḅḇḉḋḍḏḑḓḕḗḙḛḝḟḡḣḥḧḩḫḭḯḱḳḵḷḹḻḽḿṁṃṅṇ ṉṋṍṏṑṓṕṗṙṛṝṟṡṣṥṧṩṫṭṯṱṳṵṷṹṻṽṿẁẃẅẇẉẋẍẏẑẓẕẖẗẘẙẚẛẜẝẟạảấầẩẫậắằẳẵặẹ ẻẽếềểễệỉịọỏốồổỗộớờởỡợụủứừửữựỳỵỷỹỻỽỿⁱⁿₐₑₒₓₔₕₖₗₘₙₚₛₜⅎↄⱡⱥⱦⱨⱪ ⱬⱱⱳⱴⱶⱷⱸⱹⱺⱻⱼⱽꜣꜥꜧꜩꜫꜭꜯꜰꜱꜳꜵꜷꜹꜻꜽꜿꝁꝃꝅꝇꝉꝋꝍꝏꝑꝓꝕꝗꝙꝛꝝꝟꝡꝣꝥꝧꝩꝫꝭꝯꝰꝱꝲꝳꝴꝵ ꝶꝷꝸꝺꝼꝿꞁꞃꞅꞇꞌꞎꞑꞓꞡꞣꞥꞧꞩꟸꟹꟺﬀﬁﬂﬃﬄﬅﬆａｂｃｄｅｆｇｈｉｊｋｌｍｎｏｐｑｒｓｔｕｖｗｘｙｚ How do you condense that again into language-specific subsets? Every letter that appears in a word in a dictionary? Then at least é belongs to German as well, even though it's usually not considered part of the German Latin subset. Unicode stays clear of that issue by simply not defining what script subsets a character belongs to (rightfully so, IMHO).
    - dalke 10y ago
      It's not a big advantage. EBCDIC, for example, didn't do that, and programmers managed just fine without it. Also, why are you doing that check? Is it to see if something is lowercase? If so, your check will get the wrong answer for lowercase letters like å. Unicode does have a way to check if something is uppercase/lowercase, when that distinction exists. This is in UnicodeData.txt.
      - msbarnett 10y ago
        > It's not a big advantage. EBCDIC, for example, didn't do that, and programmers managed just fine without it. You might try asking an old IBM programmer just how "fine" they felt dealing with EBCDIC...
        dalke 10y ago
        EBCDIC had many problems, agreed. One is the non-contiguous range. Another is the many variants (https://en.wikipedia.org/wiki/EBCDIC_code_pages https://en.wikipedia.org/wiki/EBCDIC_code_pages lists 11, x2 for the Euro Update). But how serious is the problem? How many times do you need to test if a given character is one of the 26 allowed letters of the English alphabet, and where you implement it by testing it against the range? Typically you write it as "islower_english(c)" once, and be done with it. Is that really hard? If you do think that's a serious problem, then what of those programmers who need to test for lowercase letters in "España", "München", "Diyarbakır", and "façade"?
        msbarnett 10y ago
        I mean, if you really want to get into it, it was a huge pain in the ass at a time when paying the cost of a call to islower_english was much more expensive than a hardware less-than instruction. We've broadly moved beyond that, but there's still value in grouping sets together in a way that makes certain kinds of frequent tests less computationally expensive than they would be if codepoints were randomly distributed.
        dalke 10y ago
- deleted 10y ago
  [deleted]
iopq 10y ago
Yes, but they have alternate italic forms, for example. Sure, some one of the glyphs like с doesn't have an alternate italic form. Since the other ones do, it would be weird to only assign a separate codepoint to some of them and overlap the others. It would be a workable solution, but still weird.
- Asooka 10y ago
  You mean like Han unification?
  - iopq 10y ago
    Which is a bad idea because the characters don't look right unless you use a Japanese font. But if you want to write an article comparing Japanese and Chinese characters, you have to use two different fonts.

jhanschoo 10y ago

There are multiple reasons for that, one of which is compatibility with previous encodings and standards. If a previous encoding Unicode wanted to be compatible with encoded these as different characters, Unicode needs these to have separate code points for them too.

hvidgaard 10y ago
That is the surefire way to incorporate complexities from 2 different systems into 1.
- xyproto 10y ago
  Being able to easily check if a letter is between 'a' and 'z' in code is an advantage. This is only possible if the codepoints are sequential.
  - hvidgaard 10y ago
    I didn't dispute that. I just state that trying to remain compatible for the sake of being compatible is a great way to design a convuluted and difficult to understand standard.
    - gpderetta 10y ago
      Of course, but lack backward compatibility is a great way to make sure a standard is not adopted. For example he reason that UTF-8 'won' is that it has a great backward compatibility story with other ASCII based encodings and systems.
  - bambax 10y ago
    Is it? Couldn't Unicode have pointers or links, where a codepoint "exists" with no content and only links to another? (I don't know anything about Unicode, so maybe it already has that.)
    - edent 10y ago
      Semantically, yes. In the code tables you'll see that that "opposite" symbols have links to each other. Programatically, it is much easier to say "does a character lie between 0x12 and 0xBC" than to create a function like `isSymbolForTrafficInEurope()`
      - Natanael_L 10y ago
        Perhaps Unicode could just have tables listing all relevant sequences of symbols, instead. So "latin letters lowercase" would list the codepoints for a-z in order, for example. Would no longer matter if the codepoints themselves are sequential or not. (And relevant to my country, "Swedish characters lowercase" would map to latin letters lowercase + åäö.)
        ygra 10y ago
        Characters have a script associated with them (e.g. Latin), and caseness is also part of a character's properties. Now, language-specific subsets¹ of those are a bit iffy to deal with. Especially when text can contain loan words from other languages, so in my experience it's rarely a useful thing to ask for. ¹ Yes, subsets. Latin letters lowercase is not the set abcdefghijklmnopqrstuvwxyz. It is the set abcdefghijklmnopqrstuvwxyzªºßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ āăąćĉċčďđēĕėęěĝğġģĥħĩīĭįıĳĵķĸĺļľŀłńņňŉŋōŏőœŕŗřśŝşšţťŧũūŭůűųŵ ŷźżžſƀƃƅƈƌƍƒƕƙƚƛƞơƣƥƨƪƫƭưƴƶƹƺƽƾƿǆǉǌǎǐǒǔǖǘǚǜǝǟǡǣǥǧǩǫǭǯǰǳǵǹǻǽǿ ȁȃȅȇȉȋȍȏȑȓȕȗșțȝȟȡȣȥȧȩȫȭȯȱȳȴȵȶȷȸȹȼȿɀɂɇɉɋɍɏɐɑɒɓɔɕɖɗɘəɚɛɜɝɞɟɠɡɢ ɣɤɥɦɧɨɩɪɫɬɭɮɯɰɱɲɳɴɵɶɷɸɹɺɻɼɽɾɿʀʁʂʃʄʅʆʇʈʉʊʋʌʍʎʏʐʑʒʓʕʖʗʘʙʚʛʜʝʞʟ ʠʡʢʣʤʥʦʧʨʩʪʫʬʭʮʯʰʱʲʳʴʵʶʷʸˠˡˢˣˤᴀᴁᴂᴃᴄᴅᴆᴇᴈᴉᴊᴋᴌᴍᴎᴏᴐᴑᴒᴓᴔᴕᴖᴗᴘᴙᴚᴛᴜᴝ ᴞᴟᴠᴡᴢᴣᴤᴥᴬᴭᴮᴯᴰᴱᴲᴳᴴᴵᴶᴷᴸᴹᴺᴻᴼᴽᴾᴿᵀᵁᵂᵃᵄᵅᵆᵇᵈᵉᵊᵋᵌᵍᵎᵏᵐᵑᵒᵓᵔᵕᵖᵗᵘᵙᵚᵛᵜᵢᵣᵤ ᵥᵫᵬᵭᵮᵯᵰᵱᵲᵳᵴᵵᵶᵷᵹᵺᵻᵼᵽᵾᵿᶀᶁᶂᶃᶄᶅᶆᶇᶈᶉᶊᶋᶌᶍᶎᶏᶐᶑᶒᶓᶔᶕᶖᶗᶘᶙᶚᶛᶜᶝᶞᶟᶠᶡᶢᶣᶤᶥᶦ ᶧᶨᶩᶪᶫᶬᶭᶮᶯᶰᶱᶲᶳᶴᶵᶶᶷᶸᶹᶺᶻᶼᶽᶾḁḃḅḇḉḋḍḏḑḓḕḗḙḛḝḟḡḣḥḧḩḫḭḯḱḳḵḷḹḻḽḿṁṃṅṇ ṉṋṍṏṑṓṕṗṙṛṝṟṡṣṥṧṩṫṭṯṱṳṵṷṹṻṽṿẁẃẅẇẉẋẍẏẑẓẕẖẗẘẙẚẛẜẝẟạảấầẩẫậắằẳẵặẹ ẻẽếềểễệỉịọỏốồổỗộớờởỡợụủứừửữựỳỵỷỹỻỽỿⁱⁿₐₑₒₓₔₕₖₗₘₙₚₛₜⅎↄⱡⱥⱦⱨⱪ ⱬⱱⱳⱴⱶⱷⱸⱹⱺⱻⱼⱽꜣꜥꜧꜩꜫꜭꜯꜰꜱꜳꜵꜷꜹꜻꜽꜿꝁꝃꝅꝇꝉꝋꝍꝏꝑꝓꝕꝗꝙꝛꝝꝟꝡꝣꝥꝧꝩꝫꝭꝯꝰꝱꝲꝳꝴꝵ ꝶꝷꝸꝺꝼꝿꞁꞃꞅꞇꞌꞎꞑꞓꞡꞣꞥꞧꞩꟸꟹꟺﬀﬁﬂﬃﬄﬅﬆａｂｃｄｅｆｇｈｉｊｋｌｍｎｏｐｑｒｓｔｕｖｗｘｙｚ How do you condense that again into language-specific subsets? Every letter that appears in a word in a dictionary? Then at least é belongs to German as well, even though it's usually not considered part of the German Latin subset. Unicode stays clear of that issue by simply not defining what script subsets a character belongs to (rightfully so, IMHO).
  - dalke 10y ago
    It's not a big advantage. EBCDIC, for example, didn't do that, and programmers managed just fine without it. Also, why are you doing that check? Is it to see if something is lowercase? If so, your check will get the wrong answer for lowercase letters like å. Unicode does have a way to check if something is uppercase/lowercase, when that distinction exists. This is in UnicodeData.txt.
    - msbarnett 10y ago
      > It's not a big advantage. EBCDIC, for example, didn't do that, and programmers managed just fine without it. You might try asking an old IBM programmer just how "fine" they felt dealing with EBCDIC...
      - dalke 10y ago
        EBCDIC had many problems, agreed. One is the non-contiguous range. Another is the many variants (https://en.wikipedia.org/wiki/EBCDIC_code_pages https://en.wikipedia.org/wiki/EBCDIC_code_pages lists 11, x2 for the Euro Update). But how serious is the problem? How many times do you need to test if a given character is one of the 26 allowed letters of the English alphabet, and where you implement it by testing it against the range? Typically you write it as "islower_english(c)" once, and be done with it. Is that really hard? If you do think that's a serious problem, then what of those programmers who need to test for lowercase letters in "España", "München", "Diyarbakır", and "façade"?
        msbarnett 10y ago
        I mean, if you really want to get into it, it was a huge pain in the ass at a time when paying the cost of a call to islower_english was much more expensive than a hardware less-than instruction. We've broadly moved beyond that, but there's still value in grouping sets together in a way that makes certain kinds of frequent tests less computationally expensive than they would be if codepoints were randomly distributed.
        dalke 10y ago
deleted 10y ago
[deleted]

iopq 10y ago

Yes, but they have alternate italic forms, for example. Sure, some one of the glyphs like с doesn't have an alternate italic form. Since the other ones do, it would be weird to only assign a separate codepoint to some of them and overlap the others. It would be a workable solution, but still weird.

Asooka 10y ago
You mean like Han unification?
- iopq 10y ago
  Which is a bad idea because the characters don't look right unless you use a Japanese font. But if you want to write an article comparing Japanese and Chinese characters, you have to use two different fonts.