4 ms·
It says it sorts one byte at a time. I think this would break for anything not utf-8.
by nanidin 2y ago
It says it sorts one byte at a time. I think this would break for anything not utf-8.
- ts4z 2y agoSeems like it should work for arbitrary byte strings (any charset, any encoding)but obviously the performance characteristics will differ because of non-uniform distribution. But that happens even in ASCII.
- nanidin 2y agoYes, you’ll get something sorted based on the bytes in the string but it won’t be lexicographically correct - for example, à will be sorted after b.
- brirec 2y agoThis would even break UTF-8, since multi-byte characters are a thing!
- dumbo-octopus 2y agoHow would that break anything? The strings aren't being split.
- EmilyHughes 2y agoUTF-8 is downward compatible to ASCII, so anything that is just a standard character (like every character in this comment) is just a byte.
- deleted 2y ago[deleted]
- ursusmaritimus 2y agoLexicographic encoding of UTF-8 byte sequences matches lexicographic order of the sequence of Unicode code-points. So you can sort UTF-8 strings as byte strings. Not that sorting by code-points has much meaning, but you can use the Unicode collation algorithm first.
- remram 2y agoTIL, thanks. That makes sense given how the length prefixes look like but I never thought about it. I wonder if this was by chance or if the UTF-8 creator thought about it.
- mzs 2y ago% printf "%s\n" A B | sort A B % printf "%s" A B | xxd -b -c4 00000000: 11110000 10011101 10010000 10110100 .... 00000004: 11110000 10011101 10010000 10110101 .... % printf "%s" A B | xxd -c1 -ps | sort | xxd -r -ps | xxd -b -c4 00000000: 10010000 10010000 10011101 10011101 .... 00000004: 10110100 10110101 11110000 11110000 .... % printf "%s" A B | xxd -c1 -ps | sort | xxd -r -ps ????????
- nanidin 2y agoIndeed, and is à less than b? Not in Unicode!
- deleted 2y ago[deleted]