4 ms·
> for sorting strings of arbitrary lengths here Ahh, I see. It only loads CACHED_BYTES at a time. This could be used to delay the tie-breaking loads until need
by eesmith 4y ago
> for sorting strings of arbitrary lengths here
Ahh, I see. It only loads CACHED_BYTES at a time. This could be used to delay the tie-breaking loads until needed, with some sort of on-the-fly encoding, and potentially skip those loads completely.
I did not consider that. Thanks!
You are right - my experience is with LSB radix sorts, most specifically the one in NumPy. I see it references https://github.com/eloj/radix-sorting#-key-derivation https://github.com/eloj/radix-sorting#-key-derivation which says "an LSB implemention is not well suited to sort character strings of any significant length. For those, a MSB radix sort is better" and in turn links to an implementation which includes benchmarks https://github.com/eloj/radix-sorting#cpp-benchmark https://github.com/eloj/radix-sorting#cpp-benchmark . It in turn cites Bingmann.
Another thanks for pointing out a missing part of my understanding.
The radix approach still requires a mapping to a byte representation which is order-preserving.
> but I think these can be solved by unicode normalization + tolower + codepoint-wise comparison, which is probably what you'd do in your comparator for a comparison-based sort as well
tolower() doesn't work for all collations. Quoting https://unicode.org/reports/tr10/ https://unicode.org/reports/tr10/ "Some dictionaries and authors collate uppercase before lowercase while others use the reverse ... Sometimes the case ordering is mandated by the government, as in Denmark."
codepoint-wise comparison doesn't work for all collations. Quoting the same source, "There are additional complications in certain languages, where the comparison is context sensitive and depends on more than just single characters compared directly against one another" as well as "In some French dictionary ordering traditions, however, it is the last accent difference that determines the order".