15 ms·
Known anomalies in Unicode character names (2017)
- billme 6y ago(2017) should be append, since both the header and footer state that document is from 2017. — Also, from the parent page for that link: “ Unicode Technical Notes provide information on a variety of topics related to Unicode and internationalization technologies. These technical notes are independent publications, not approved by any of the Unicode Technical Committees, nor are they part of the Unicode Standard or any other Unicode specification. Publication does not imply endorsement by the Unicode Consortium in any way. These documents are not subject to the Unicode Patent Policy.” SOURCE: https://unicode.org/notes/ https://unicode.org/notes/
- dang 6y agoAdded, thanks.
- cooper12 6y agoConsidering how huge the Unicode standard is, it's more surprising that there are so little (known) errors. CJK characters alone account for thousands upon thousands, and these can sometimes vary by just a single stroke. I suspect the majority of redundancies or suboptimal choices were the result of subsuming so many existing standards though. There might also be plain errors in research but are probably in the more obscure blocks. See also, ghost kanji: https://www.japantimes.co.jp/life/2018/10/29/language/ghost-kanji-lurk-japanese-lexicon/ https://www.japantimes.co.jp/life/2018/10/29/language/ghost-...
- duskwuff 6y ago> CJK characters alone account for thousands upon thousands The vast majority of CJK characters have systematic names like "CJK UNIFIED IDEOGRAPH-72AC" which aren't really subject to errors in the same way as the more verbose names used for other scripts.
- claudiawerner 6y agoI have to wonder what the systematic names refer to; is it a Chinese/Japanese dictionary ordering in which they were assigned, stroke count (but then, in what order are characters of the same stroke count organized), or some other or arbitrary ordering? As in, why is the 72AC character at 72AC, not at 72AD?
- duskwuff 6y agoThat's a good question. Unfortunately, a lot of old Unicode process documentation isn't available online -- you can see a couple of relevant-sounding documents at [1], but very little of it is available until 1999 or so. There are some pretty clear patterns to the character ordering, though -- if you look closely, you can see big runs of characters which share radicals. For example, characters 5000 through 500F are: 倀 倁 倂 倃 倄 倅 倆 倇 倈 倉 倊 個 倌 倍 倎 倏 all of which have the 亻radical on the left side. It's clearly not arbitrary. [1]: https://www.unicode.org/L2/L1990/Register-1990.html https://www.unicode.org/L2/L1990/Register-1990.html
- cynix 6y ago> all of which have the 亻radical on the left side. Except for 倉 it seems.
- deleted 6y ago[deleted]
- jschwartzi 6y agoIt looks like it's merged under the roof in that character.
- thaumasiotes 6y agoNo, a1369209993 is correct - 亻is the combining form of 人 (the "单人旁"); 倉 uses the more basic 人 form.
- 6y ago
- Muromec 6y ago>There might also be plain errors in research but are probably in the more obscure blocks. The one with Lao letters "LO LING" and "LO LOOT" is actually very blunt error -- names for them are swapped. It's pretty safe to assume that whoever was making the standard does not know this writing system.
- TheRealPomax 6y agoIt really isn't: it's safe to assume they did, because full language mappings are not the job of a single person, and don't get accepted willy-nilly (unlike emoji). However, clerical errors are SUPER EASY, so it's safe to assume someone accidentally pasted something in the wrong database-presented-as-a-spreadsheet close enough to the time of official revision publication, and when that isn't caught, now it's official.
- jschwartzi 6y agoHaving done a Japanese localization by communicating entirely in spreadsheets, this is extremely likely. In fact we had to fix the translations a couple of times because we'd send it off for feedback and get a different translation for the same English.
- gfaure 6y agoDoes anyone know the background behind: > U+262B FARSI SYMBOL > This symbol is so named because as symbol of Iran it cannot be encoded in ISO standards. It would seem strange that ISO standards cannot refer to countries, so I'm guessing this is something specific to Iran?
- duskwuff 6y agoISO standards refer to countries all the time -- ISO 3166 is literally all about countries, for instance. I suspect this is actually shorthand for "ISO wanted to avoid referring to countries in Unicode character names". But that ship has clearly sailed nowadays with characters like U+1F5FE ("SILHOUETTE OF JAPAN").
- _vbdg 6y ago> SILHOUETTE OF JAPAN This may sound pedantic, but Japan the island or Japan the nation? It seems to refer to the island(s). On the other hand, the "farsi symbol" is of a particular nation rather than a given set of boundaries. Nations can be renamed, but the island of Japan refers to the chunk of earth. Add to that that nations get redrawn every so often in the middle east and I can understand the position.
- dmurray 6y agoJapan is not an island. Having a distinct symbol for the 4 large islands comprising most of Japan is definitely a political statement, if not an especially controversial one.
- andyjohnson0 6y agoA quick search brought me to [1] which says: <quote> As noted by Roozbeh Pournader: Neither Farsi, nor a symbol. In real life, it is the official emblem of the goverment of the Islamic Republic of Iran. Technically that would make it a logo and thus not a suitable candidate for encoding. But Roozbeh also noted: Exactly. The funny fact is that it has been in Unicode since 1.0... </quote> How reliable this is I don't know, but it sounds plausible. [1] http://archives.miloush.net/michkap/archive/2005/01/29/363208.html http://archives.miloush.net/michkap/archive/2005/01/29/36320...
- raldi 6y agoI don't understand why some of these errors were fixed via formal aliases but not others. Why not give them all proper aliases?
- polm23 6y agoI think that for some the correct alias was already in use for another character, like with the digrams.
- adontz 6y agoI am not sure if native speakers are involved in Unicode standardization at all. Personal example. In Georgian (a few millions native speakers) we had historically three different independent alphabets: mtavruli, nuskhuri and mkhedruli. All represent same letters, but different ways to write them. Alphabets are represented in Unicode under different code points. Only mkhedruli is used currently (last few centuries) and therefore most fonts have glyphs for mkhedruli only. Also there is no concept of capitalization in Georgian, there is no "a" and "A", we have only "ა", one does not start a sentence, a word or a personal name with some variant of a letter. Anna is ანნა in Georgian, the first and the last letters are written the same way. Now, because of some reasons, letters of mtavruli alphabet are registered as uppercase versions of letters of mkhedruli alphabet. Mind blown. And since well written software uses rules defined by Unicode standard, via ICU library or some platform specific routines, many desktop and mobile applications as well as major web-sites are hard to use. First letters of sentences are capitalized, but since mtavruli glyphs are missing from fonts, they are rendered as ⍰. So instead of მაგათი დედაც ვატირე. we see ⍰აგათი დედაც ვატირე. and have to guess what was the first letter. But who cares, if it is not used in U.S. right?
- CharlotteBuff 6y agoThe uppercase mapping of Mkhedruli is Mtavruli in Unicode, but the titlecase mapping is not. If you use the uppercase mapping to automatically capitalise the first letter of a sentence, you are not following the recommendations of the Unicode standard.
- Mindless2112 6y agoTIL about ToTitleCase. An easy-to-visualize example of the difference is ToUpperCase("dz") => "DZ" ToTitleCase("dz") => "Dz"
- techdragon 6y agoBut how many programmers are even aware of the fact Unicode has both an Uppercase and Titlecase mapping? Let alone that they can do different things! I routinely see code that treats titlecase as “set individual character in this Unicode string to its upper case equivalent”... It wouldn’t surprise me at all to find out this kind of titlecase vs uppercase mistake is a very widespread issue.
- dwheeler 6y agoThe use of the obscure word "solidus" instead of "slash" for the character "/" is absurd. The term "slash" should at least be a formal alias, if it isn't already. This text seems to imply it's not a formal alias, but maybe it is. After all, the POSIX standard includes slash as a name for the character, and that an ISO standard: https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ What's more, Unicode refers to "\" as "backslash". Very weird.
- Sniffnoy 6y ago"Slash" and "virgule" are both formal aliases for U+002F SOLIDUS.
- reaperducer 6y agoWhat's more, Unicode refers to "\" as "backslash". I'd be happy if nobody ever referred to the word "backslash." I heard it read in a URL in a TV commercial this week. I can't believe it. People have been getting that wrong since AOL days.
- rplst8 6y agoWhat would you prefer then?
- cool_dude85 6y agoAntivirgule, of course.
- reaperducer 6y agoThey simply don't belong in web addresses.
- deleted 6y ago[deleted]
- dhosek 6y agoPeople say backslash in web addresses when they mean a plain old slash.
- js8 6y ago> The "caron" should have been called hacek and combining hacek. The term "caron" is suspected by some to be an invention of some early standards body That's funny. Hacek is a Czech invention (it was actually a dot originally), but I always assumed that "caron" is the official "correct" (english) name.
- smcl 6y agoKnown anomalies in “Known anomalies in Unicode character names”: they refer to “háček” as “hacek”. I wouldn’t normally pick something up like this, but isn’t the whole point of ASCII to have a way to represent non-English characters?