3 ms·

To be fair, maybe it would have been a bit of a shit show if they standardised on a string format before the popularisation (or even invention) of UTF-16 (or UT

• vardump 5y ago
  UTF-16 makes only sense for backwards compatibility with something that was originally designed to use OCS-2. Annoyingly, it can be both big and little endian encoding. UTF-8 saves space and is just as easy (and fast!) to process as UTF-16, as both are variable length encodings. UTF-8 is always unambiguous and self-synchronizing. The byte order stays always the same. The only complication is potentially multiple ways to encode a character, but that can be solved by just declaring all but the shortest encoding illegal. Hopefully we'll only have UTF-8 in the long run, both externally and (by default) internally. C string functions are mostly compatible with UTF-8, as long as individual non-7-bit characters are not processed or the string indexed, etc.