7 ms·

> in alphabetical order Well, it's in an order, but I don't know about alphabetical. I clicked on today's English featured article and looked at the languages:

> in alphabetical order Well, it's in an order, but I don't know about alphabetical. I clicked on today's English featured article and looked at the languages: "中文", "Italiano" are "suggested", then the remainder are grouped by geographic region, and aren't particularly alphabetical. They appear to be in groups which are still not alphabetical. Europe seems to have a Cyrillic group but "Қазақша" is shown after "Українська" which isn't accurate in Kazakh and probably also unexpected for anybody who isn't familiar with the letter Қ (Қ isn't a letter in Russian, this is probably why this happens). The Chinese languages don't seem to be in stroke order (no expert here), although Korean is below them (because of course, K for Korean alphabetizes after C for Chinese). Anyways, no hate for Wikipedia; they do a great job of localizing. Just a bit of nuance/pedantry about how you can't "alphabetize" language names in their own language.

euazOn 1y ago
Yes, that’s confusing and probably hard to find a good balance. Someone speaking Greek or Czech may expect to find their language around E (Ελληνικά) or C (Čeština), but nope, on Wiki it’s all the way after Z.
- af78 1y ago
  I guess the default (when no language is specified) is Unicode order: U+005A LATIN CAPITAL LETTER Z U+010C LATIN CAPITAL LETTER C WITH CARON U+0395 GREEK CAPITAL LETTER EPSILON
- nis251413 1y ago
  Don’t special characters always go after the Latin alphabet? I think this is pretty common, and fairly expected behaviour. Of course nothing is perfect but I feel like the way Wikipedia handles it is consistent.
  - makapuf 1y ago
    Well in my language "é" is absolutely not special, and should definitively be placed near "e" (to the point that uppercase é is often written E instead of É)
  - e-topy 1y ago
    Not in the Czech alphabet: a, á, b, c, č, d, ď, e, é, ě, ... Also, we regard 'Ch' as its own letter. So yeah, try sorting alphabetically. I'll wait.
    - bmn__ 1y ago
      perl -E'Unicode::Collate::Locale->new(locale => 'cs')->sort … … …' works. Test cases at https://prirucka.ujc.cas.cz/?action=view&id=900 https://prirucka.ujc.cas.cz/?action=view&id=900
      - mulmen 1y ago
        I love some perl on Monday morning but how does this work when you don’t know the locale?
        bmn__ 1y ago
        Then a system should fall back to DUCET which produces more or less sensible results across all locales.
    - bawolff 1y ago
      Digraphs like Ch are common in a lot of languages. Wikipedia supports that fine on category pages. E.g. https://cs.wikipedia.org/wiki/Kategorie:CHKO_%C5%A0umava https://cs.wikipedia.org/wiki/Kategorie:CHKO_%C5%A0umava If you want to see bizarre sort rules, look up how french sorts accent characters.
      - thaumasiotes 1y ago
        > If you want to see bizarre sort rules, look up how french sorts accent characters. I tried to do this, but there do not appear to be any sources addressing this question. I did find a French Stack Exchange question asking for this exact information, and complaining that there are no sources (other than an uncited wikipedia page) that address it. There is no answer posted, but there is a comment from a French guy suggesting that there are no official rules. https://french.stackexchange.com/questions/54217/french-dictionary-sort-order-what-are-the-rules https://french.stackexchange.com/questions/54217/french-dict... How were you imagining I would look this up?
  - jurip 1y ago
    It depends on the language. Unicode defines rules for it: https://www.unicode.org/reports/tr10/ https://www.unicode.org/reports/tr10/
  - paulddraper 1y ago
    Sorting by character codes, yes. But in the language native locale, no.
  - philistine 1y ago
    That’s English hegemony. Languages have their own sorting that they expect. You can’t impose rules to other languages. Of course at some point Unicode needs to be ordered, but you don’t get to impose technical details to people around the world because it matches with how English does it. That’s where geo-ip guessing becomes relevant. Show a list with the most likely languages at the top.
    - swiftcoder 1y ago
      Or use the Accept-Language. Since we already know the User understands that one, it's probably a reasonable choice for which sort order they expect too.
    - adastra22 1y ago
      That’s not English sort order either.
  - deleted 1y ago
    [deleted]
  - psychoslave 1y ago
    If I recall correctly, the default propose a first list that push items which are guessed most likely what the user expect, then a list more complete, and in any case let you filter by typing. I think it also can change the way it behave if you are connected and tweaked your preferences in the matter for your account.
  - bawolff 1y ago
    Wikipedia uses UCA sort order in categories (depending on which lang wikipedia you are reading). Most other lists just sort using unicode codepoint order (in NFC). So it depends, but yes, for generated lists other than categories ascii characters usually come first.
- soulofmischief 1y ago
  When serving that many languages, a search bar is paramount.
- kazinator 1y ago
  The problem may be is that you need to set the locale in order to get certain alphabetization, but setting the locale won't happen until after the language is chosen. A reasonable approach might be to sort the list of names by using, as the sort keys, the strings projected through a Unicode normalization function, followed by folding to upper case. Then Čestina gets mapped to CESTINA and at least appears among the C's.
bmn__ 1y ago
> how you can't "alphabetize" language names Not so, this sort order has been standardised as part of Unicode for at least 28 years. To see it in action, pipe the list of languages as a text file through a conforming tool like `ucsort`. When Қ is falsely sorted after Ч, then the wrong algorithm or no algorithm at all has been used. > because of course, K for Korean alphabetizes after C for Chinese That's not how it works.
- mananaysiempre 1y ago
  Yes, the DUCET is bound to disappoint everybody (especially users of the Latin script with diacritics, as none of them agree on the sort order and everyone’s preferences are tied to the specific subset of diacritics they need), but at least it disappoints everybody more or less equally. (Do yourself a favor, though, and use the CLDR root collation instead of the raw DUCET—they are basically the same, except, and I’m quoting the standard here[1], “the DUCET is not entirely well-formed”.) [1] https://www.unicode.org/reports/tr10/#Well_Formed_DUCET https://www.unicode.org/reports/tr10/#Well_Formed_DUCET
- adastra22 1y ago
  Sort rules are different in different locales.
  - vikingerik 1y ago
    It's a circular dependency: how do you sort and list the locales or languages for someone to pick one, when by definition you don't know their locale yet? You have to either make some best-guess approximation (IP geo, browser headers, etc) or use a locale-invariant sort, both of which will be wrong in some cases.
    - notpushkin 1y ago
      We can find a sorting order with the minimal total distance between where we place a language entry and where this entry would be in that language. If there’s no pair of languages A and Ä such that A > Ä in one and Ä > A in the other, then (I guess???) this total distance will be zero.
bawolff 1y ago
> Well, it's in an order, but I don't know about alphabetical. I clicked on today's English featured article and looked at the languages This depends on whether you are viewing desktop site or mobile site. It also depends on if you have a non-default skin set in your preferences. Seems like desktop (vector-2023) does the region thing. Mobile does alphabetical by language name (i imagine codepoint order but i didnt check) Some other skins are alphabetical by bcp47 code.
thaumasiotes 1y ago
> The Chinese languages don't seem to be in stroke order (no expert here) They are for me. In the Asia section, 中文 ["Chinese"] is listed first, followed by 吴语 ["Wu"] and then 粤语 ["Cantonese"]. Stroke order is first by stroke count and then by an obscure criterion that I don't know (and that, in my experience, Chinese people living in China also don't know), but stroke count is unambiguous and these are in order: 中 4, 吴 7, 粤 12. Note that they aren't in alphabetical order: 中 Z, 吴 W, 粤 Y. Japanese appears between Wu and Cantonese for unclear reasons.
- matvore 1y ago
  It is sorted FIRST by radical and SECOND by stroke order. This is roughly equivalent to the Unicode codepoint sort if you stay in the basic multilingual plane. The order also puts literary chinese afer wu Chinese, which breaks with a pure stroke-count sort: 中文 - 中 = 丨 + 3 strokes 吴语 - 吴 = 口 + 4 strokes 文言 - 文 = 文 + 0 strokes 日本語 - 日 = 日 + 0 strokes 粵語 - 粵 = 米 + 7 strokes
  - thaumasiotes 1y ago
    Dictionary lookup is done first by radical and second by stroke count. Collation is not. Stroke count is first. For example, I have a book of 成语 stories that gives its table of contents in non-alphabetical order. (Since nobody understands the traditional ordering, I also have several such books that put their table of contents in alphabetical order.) Here is the collation order in the book: 一七八入九人口千小三亡大不专天井见毛月文风为心水四 ... Note that 三's radical is 一, the first Kangxi radical, and that 一 is listed first. Your theory is wrong. 三 isn't even first among the 3-stroke characters, which start (among these) with 口. Why did you make up a false answer to this question?
    - matvore 1y ago
      The Wikipedia sort for the languages is as I stated above, with Literary Chinese and Japanese between Wu Chinese and Yue Chinese. I explained why it was sorted that way, because radical is considered first. You could not explain why Japanese appeared between Wu and Yue because you insisted and continue to insist that radicals are not used. I didn't say sorting is never done by stroke count alone. But I have seen radical+residual stroke count much more often than stroke count alone. Probably a result of the content I'm accessing. It's mostly Japanese and not intended for children. The dictionary and non-dictionary sorting distinction that you make doesn't sound like a real thing. The audience, the country, and the number of items sorted are bigger factors. But you're not wrong in that stroke count is sometimes used alone.

euazOn 1y ago

Yes, that’s confusing and probably hard to find a good balance. Someone speaking Greek or Czech may expect to find their language around E (Ελληνικά) or C (Čeština), but nope, on Wiki it’s all the way after Z.

af78 1y ago
I guess the default (when no language is specified) is Unicode order: U+005A LATIN CAPITAL LETTER Z U+010C LATIN CAPITAL LETTER C WITH CARON U+0395 GREEK CAPITAL LETTER EPSILON
nis251413 1y ago
Don’t special characters always go after the Latin alphabet? I think this is pretty common, and fairly expected behaviour. Of course nothing is perfect but I feel like the way Wikipedia handles it is consistent.
- makapuf 1y ago
  Well in my language "é" is absolutely not special, and should definitively be placed near "e" (to the point that uppercase é is often written E instead of É)
- e-topy 1y ago
  Not in the Czech alphabet: a, á, b, c, č, d, ď, e, é, ě, ... Also, we regard 'Ch' as its own letter. So yeah, try sorting alphabetically. I'll wait.
  - bmn__ 1y ago
    perl -E'Unicode::Collate::Locale->new(locale => 'cs')->sort … … …' works. Test cases at https://prirucka.ujc.cas.cz/?action=view&id=900 https://prirucka.ujc.cas.cz/?action=view&id=900
    - mulmen 1y ago
      I love some perl on Monday morning but how does this work when you don’t know the locale?
      - bmn__ 1y ago
        Then a system should fall back to DUCET which produces more or less sensible results across all locales.
  - bawolff 1y ago
    Digraphs like Ch are common in a lot of languages. Wikipedia supports that fine on category pages. E.g. https://cs.wikipedia.org/wiki/Kategorie:CHKO_%C5%A0umava https://cs.wikipedia.org/wiki/Kategorie:CHKO_%C5%A0umava If you want to see bizarre sort rules, look up how french sorts accent characters.
    - thaumasiotes 1y ago
      > If you want to see bizarre sort rules, look up how french sorts accent characters. I tried to do this, but there do not appear to be any sources addressing this question. I did find a French Stack Exchange question asking for this exact information, and complaining that there are no sources (other than an uncited wikipedia page) that address it. There is no answer posted, but there is a comment from a French guy suggesting that there are no official rules. https://french.stackexchange.com/questions/54217/french-dictionary-sort-order-what-are-the-rules https://french.stackexchange.com/questions/54217/french-dict... How were you imagining I would look this up?
- jurip 1y ago
  It depends on the language. Unicode defines rules for it: https://www.unicode.org/reports/tr10/ https://www.unicode.org/reports/tr10/
- paulddraper 1y ago
  Sorting by character codes, yes. But in the language native locale, no.
- philistine 1y ago
  That’s English hegemony. Languages have their own sorting that they expect. You can’t impose rules to other languages. Of course at some point Unicode needs to be ordered, but you don’t get to impose technical details to people around the world because it matches with how English does it. That’s where geo-ip guessing becomes relevant. Show a list with the most likely languages at the top.
  - swiftcoder 1y ago
    Or use the Accept-Language. Since we already know the User understands that one, it's probably a reasonable choice for which sort order they expect too.
  - adastra22 1y ago
    That’s not English sort order either.
- deleted 1y ago
  [deleted]
- psychoslave 1y ago
  If I recall correctly, the default propose a first list that push items which are guessed most likely what the user expect, then a list more complete, and in any case let you filter by typing. I think it also can change the way it behave if you are connected and tweaked your preferences in the matter for your account.
- bawolff 1y ago
  Wikipedia uses UCA sort order in categories (depending on which lang wikipedia you are reading). Most other lists just sort using unicode codepoint order (in NFC). So it depends, but yes, for generated lists other than categories ascii characters usually come first.
soulofmischief 1y ago
When serving that many languages, a search bar is paramount.
kazinator 1y ago
The problem may be is that you need to set the locale in order to get certain alphabetization, but setting the locale won't happen until after the language is chosen. A reasonable approach might be to sort the list of names by using, as the sort keys, the strings projected through a Unicode normalization function, followed by folding to upper case. Then Čestina gets mapped to CESTINA and at least appears among the C's.

bmn__ 1y ago

> how you can't "alphabetize" language names Not so, this sort order has been standardised as part of Unicode for at least 28 years. To see it in action, pipe the list of languages as a text file through a conforming tool like `ucsort`. When Қ is falsely sorted after Ч, then the wrong algorithm or no algorithm at all has been used. > because of course, K for Korean alphabetizes after C for Chinese That's not how it works.

mananaysiempre 1y ago
Yes, the DUCET is bound to disappoint everybody (especially users of the Latin script with diacritics, as none of them agree on the sort order and everyone’s preferences are tied to the specific subset of diacritics they need), but at least it disappoints everybody more or less equally. (Do yourself a favor, though, and use the CLDR root collation instead of the raw DUCET—they are basically the same, except, and I’m quoting the standard here[1], “the DUCET is not entirely well-formed”.) [1] https://www.unicode.org/reports/tr10/#Well_Formed_DUCET https://www.unicode.org/reports/tr10/#Well_Formed_DUCET
adastra22 1y ago
Sort rules are different in different locales.
- vikingerik 1y ago
  It's a circular dependency: how do you sort and list the locales or languages for someone to pick one, when by definition you don't know their locale yet? You have to either make some best-guess approximation (IP geo, browser headers, etc) or use a locale-invariant sort, both of which will be wrong in some cases.
  - notpushkin 1y ago
    We can find a sorting order with the minimal total distance between where we place a language entry and where this entry would be in that language. If there’s no pair of languages A and Ä such that A > Ä in one and Ä > A in the other, then (I guess???) this total distance will be zero.

bawolff 1y ago

> Well, it's in an order, but I don't know about alphabetical. I clicked on today's English featured article and looked at the languages This depends on whether you are viewing desktop site or mobile site. It also depends on if you have a non-default skin set in your preferences. Seems like desktop (vector-2023) does the region thing. Mobile does alphabetical by language name (i imagine codepoint order but i didnt check) Some other skins are alphabetical by bcp47 code.

thaumasiotes 1y ago

> The Chinese languages don't seem to be in stroke order (no expert here) They are for me. In the Asia section, 中文 ["Chinese"] is listed first, followed by 吴语 ["Wu"] and then 粤语 ["Cantonese"]. Stroke order is first by stroke count and then by an obscure criterion that I don't know (and that, in my experience, Chinese people living in China also don't know), but stroke count is unambiguous and these are in order: 中 4, 吴 7, 粤 12. Note that they aren't in alphabetical order: 中 Z, 吴 W, 粤 Y. Japanese appears between Wu and Cantonese for unclear reasons.

matvore 1y ago
It is sorted FIRST by radical and SECOND by stroke order. This is roughly equivalent to the Unicode codepoint sort if you stay in the basic multilingual plane. The order also puts literary chinese afer wu Chinese, which breaks with a pure stroke-count sort: 中文 - 中 = 丨 + 3 strokes 吴语 - 吴 = 口 + 4 strokes 文言 - 文 = 文 + 0 strokes 日本語 - 日 = 日 + 0 strokes 粵語 - 粵 = 米 + 7 strokes
- thaumasiotes 1y ago
  Dictionary lookup is done first by radical and second by stroke count. Collation is not. Stroke count is first. For example, I have a book of 成语 stories that gives its table of contents in non-alphabetical order. (Since nobody understands the traditional ordering, I also have several such books that put their table of contents in alphabetical order.) Here is the collation order in the book: 一七八入九人口千小三亡大不专天井见毛月文风为心水四 ... Note that 三's radical is 一, the first Kangxi radical, and that 一 is listed first. Your theory is wrong. 三 isn't even first among the 3-stroke characters, which start (among these) with 口. Why did you make up a false answer to this question?
  - matvore 1y ago
    The Wikipedia sort for the languages is as I stated above, with Literary Chinese and Japanese between Wu Chinese and Yue Chinese. I explained why it was sorted that way, because radical is considered first. You could not explain why Japanese appeared between Wu and Yue because you insisted and continue to insist that radicals are not used. I didn't say sorting is never done by stroke count alone. But I have seen radical+residual stroke count much more often than stroke count alone. Probably a result of the content I'm accessing. It's mostly Japanese and not intended for children. The dictionary and non-dictionary sorting distinction that you make doesn't sound like a real thing. The audience, the country, and the number of items sorted are bigger factors. But you're not wrong in that stroke count is sometimes used alone.