2 ms·
One aspect the article is omitting is that UTF-16 and UTF-32 aren’t strictly speaking encodings, but UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE are. Furthermore, w
by layer8 3y ago
One aspect the article is omitting is that UTF-16 and UTF-32 aren’t strictly speaking encodings, but UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE are. Furthermore, with a 21-bit packed encoding like UTF-21, not only byte-level endianness is relevant, but inner-byte bit-level endianness becomes relevant as well. While the encoding examples given imply a particular endianness, this is something that would have to be specified more explicitly.
- chrismorgan 3y agoEndianness is such a pain. I wish everything had always stayed big-endian, the generally superior option (though I will acknowledge it’s not quite uniformly superior). Implement any bit-level algorithms and endianness tends to rear its ugly head. I implemented the Speck cipher last year, and it basically mixed little- and big-endianness (byte/bit expression), but the paper never mentioned this at all, so that trial and error on the sample vectors, or emailing the authors for clarification, was the only way you’d find that out! The article gives examples that assume big-endianness, but in practice, UTF-16 is almost always UTF-16LE (no idea about UTF-32 since it’s not used so much), so the bytes within each code unit are, for practical purposes, back to front.