3 ms·

UTF-16 is a variable width encoding that is also more convenient to process compared to UTF-8. For example you don't have to deal with invalid code units, non-s

UTF-16 is a variable width encoding that is also more convenient to process compared to UTF-8. For example you don't have to deal with invalid code units, non-shortest forms, etc. I think you're right that older languages use UTF-16 and newer ones use UTF-8. But it also seems empirically true that UTF-16 languages do better at grappling with Unicode's subtleties, compared to UTF-8. The temptation of UTF8-is-bsically-C-strings is hard to ignore.

duskwuff 9y ago
> For example you don't have to deal with invalid code units, non-shortest forms, etc. You have to deal with unpaired surrogates, though. And just because that wasn't annoying enough, any UTF-8 sequence which contains an encoded surrogate (which is technically invalid, but not prohibited by most implementations) is impossible to encode as UTF-16.