3 ms·
> I have an old saw about UTF-16 not being an irredeemable format and UTF-8 eating the world being bad, and I'm happy to dig it out again. > UTF-16 is great fo
by thaumasiotes 5y ago
> I have an old saw about UTF-16 not being an irredeemable format and UTF-8 eating the world being bad, and I'm happy to dig it out again.
> UTF-16 is great for lots of East Asian languages, which billions of people use. In UTF-8, most of those languages require 3 bytes to encode a 32-bit codepoint, in UTF-16 they only ever need 2. This ends up being a huge savings.
That doesn't make UTF-16 not a terrible format, though. East Asian countries use better formats instead. For example, in China they use GB18030.
When your case for "UTF-16 is not an irredeemable format" is "it's better than UTF-8 at a task where nobody would use either format anyway", you're not making a strong case.
- camgunz 5y ago> When your case for "UTF-16 is not an irredeemable format" is "it's better than UTF-8 at a task where nobody would use either format anyway", you're not making a strong case. Haha this is pretty fair; I know I'm being pedantic. But I wouldn't say any GB* encoding is better than UTF-16. Another commenter pointed out that you really want the ability to--say--paste arbitrary text into an editor and for that editor to be using an encoding that can handle it. Or even for the web, if we actually took the Accept-Language header [0] seriously, that could also be a big savings. [0]: https://www.rfc-editor.org/rfc/rfc7231#section-5.3.5 https://www.rfc-editor.org/rfc/rfc7231#section-5.3.5