4 ms·

That said, the Unicode standard does a great job of tackling that question and providing a framework that can cover all the various language standards (and even

by dhosek 5y ago

That said, the Unicode standard does a great job of tackling that question and providing a framework that can cover all the various language standards (and even with a fully normalized Unicode string you still need to do things like handle ch as a single character for sorting in Czech or Spanish).

pezezin 5y ago
In Spanish, "ch" and "ll" stopped being considered a single character for sorting purposes in 1994. In have some dictionaries printed before that date, and I find them quite confusing...
- dhosek 5y ago
  As someone who learned his Spanish before 1994 (and whose dictionary also predates that time), I was unaware of this until now.
Toutouxc 5y ago
As a Czech I would love "ch" to go fuck itself for that exact reason.
oldsecondhand 5y ago
In Hungarian "dzs", "zs" and "sz" are so called compund letters, however most software treats them as separate characters, because it would be a pain in the ass do sorting the correct way. (It would either require changing the input method, or software would need to be aware of etymology.) It's only sorted the right way in paper dictionaries.
- dhosek 5y ago
  Actually, it just needs to be context sensitive. If I have an English document and I have an index which has an entry for Zsigmond, it would be placed between Zebra and Zygote. But, if I have a Hungarian document, then the correct ordering would be to have Zsigmond after Zebra and Zygote. Any correctly-implemented program that deals with sorting should not be assuming it can just sort on code point¹ and Hungarian is one of the standard collations that is part of the Unicode reference implementations so it shouldn't be a problem. The fact that a lot of software is badly-implemented is not an excuse for software to continue to be so. I did a quick check and saw that I can specify Hungarian in MySQL as a collation for a table and I also know that it's available in Java and even JavaScript. ⸻ 1. This sorting is wrong not just for Hungarian but for every language including English which would expect, e.g., naïve to be sorted between nag and nanny and not after nay.

by dhosek 5y ago

pezezin 5y ago

In Spanish, "ch" and "ll" stopped being considered a single character for sorting purposes in 1994. In have some dictionaries printed before that date, and I find them quite confusing...

dhosek 5y ago
As someone who learned his Spanish before 1994 (and whose dictionary also predates that time), I was unaware of this until now.

Toutouxc 5y ago

As a Czech I would love "ch" to go fuck itself for that exact reason.

oldsecondhand 5y ago

In Hungarian "dzs", "zs" and "sz" are so called compund letters, however most software treats them as separate characters, because it would be a pain in the ass do sorting the correct way. (It would either require changing the input method, or software would need to be aware of etymology.) It's only sorted the right way in paper dictionaries.

dhosek 5y ago
Actually, it just needs to be context sensitive. If I have an English document and I have an index which has an entry for Zsigmond, it would be placed between Zebra and Zygote. But, if I have a Hungarian document, then the correct ordering would be to have Zsigmond after Zebra and Zygote. Any correctly-implemented program that deals with sorting should not be assuming it can just sort on code point¹ and Hungarian is one of the standard collations that is part of the Unicode reference implementations so it shouldn't be a problem. The fact that a lot of software is badly-implemented is not an excuse for software to continue to be so. I did a quick check and saw that I can specify Hungarian in MySQL as a collation for a table and I also know that it's available in Java and even JavaScript. ⸻ 1. This sorting is wrong not just for Hungarian but for every language including English which would expect, e.g., naïve to be sorted between nag and nanny and not after nay.