4 ms·

Um, the first character of each key is different -- they are not the same.

by nocman 10y ago

kbenson 10y ago
Ah, I see. That wasn't obvious to me when I first looked. Isn't sorting of Unicode characters defined under Unicode? I think the correct answer for "how do I sort my unicode strings" is "defer to unicode". That may be quite expensive in the end, depending on object size and content.
- cwyers 10y ago
  Alphabetical order changes from language to language. If you don't specify a collation, you can run into edge cases.
  - wrs 10y ago
    And you can't specify "a collation" in a way that will be guaranteed to mean exactly the same thing next year. I don't believe there's a way to do a permanently stable sort if you also want it to reflect the current state of Unicode. (An obvious example is that an entire language might have been added to Unicode since you serialized something with those code points in it.) NTFS, which has to build a b-tree of filenames in a permanently stable sort order, solved this by writing the collation table to the disk when it's formatted, and never changing it again. Which means the Windows shell still has to re-sort the filenames because the currently "correct" collation may be different.
    - kbenson 10y ago
      Hmm, then maybe the easiest solution is to sort numerically by unicode code point, and accept that combining characters will look weird. Or are there problems with that as well that I'm not considering (beyond it being less human readable)?
      - seagreen 10y ago
        I think the way to go is definitely by code point. We don't want to pull any more of the Unicode spec into Son than we have to. Happily sorting is only used to order members within Son objects. Individual strings are never sorted, so there isn't a worry about sorted strings coming out messy.
    - seagreen 10y ago
      Great point about new languages meaning language-aware sorting systems will definitely have breaking changes.
- pwdisswordfish 10y ago
  Of those two? That depends on whether your language is German or Swedish.

Um, the first character of each key is different -- they are not the same.

by nocman 10y ago

Um, the first character of each key is different -- they are not the same.

Ah, I see. That wasn't obvious to me when I first looked. Isn't sorting of Unicode characters defined under Unicode? I think the correct answer for "how do I sort my unicode strings" is "defer to unicode". That may be quite expensive in the end, depending on object size and content.

cwyers 10y ago
Alphabetical order changes from language to language. If you don't specify a collation, you can run into edge cases.
- wrs 10y ago
  And you can't specify "a collation" in a way that will be guaranteed to mean exactly the same thing next year. I don't believe there's a way to do a permanently stable sort if you also want it to reflect the current state of Unicode. (An obvious example is that an entire language might have been added to Unicode since you serialized something with those code points in it.) NTFS, which has to build a b-tree of filenames in a permanently stable sort order, solved this by writing the collation table to the disk when it's formatted, and never changing it again. Which means the Windows shell still has to re-sort the filenames because the currently "correct" collation may be different.
  - kbenson 10y ago
    Hmm, then maybe the easiest solution is to sort numerically by unicode code point, and accept that combining characters will look weird. Or are there problems with that as well that I'm not considering (beyond it being less human readable)?
    - seagreen 10y ago
      I think the way to go is definitely by code point. We don't want to pull any more of the Unicode spec into Son than we have to. Happily sorting is only used to order members within Son objects. Individual strings are never sorted, so there isn't a worry about sorted strings coming out messy.
  - seagreen 10y ago
    Great point about new languages meaning language-aware sorting systems will definitely have breaking changes.
pwdisswordfish 10y ago
Of those two? That depends on whether your language is German or Swedish.