3 ms·

Also, UTF-16 is not fixed width in modern usage. As soon as someone uses an emoji, boom!, surrogate pairs. So unless UTF-32 is used at a glorious four bytes per

• atoav 7y ago
  > Interesting. Isn't that only returned when the input bytes contain a non-UTF-8 byte sequence? As a Rust user: this is what your function returns if the user inputs non-UTF-8 bytes into something that expects UTF-8 bytes and the programmer explicitly choses not to handle that error. I don’t see anything wrong with that. Sometimes you might be interested in receiving, processing, storing valid UTF-8 strings rather than arbitrary byte sequences, that may or may not be able to be translated back into something valid that you can display. I always hated to do encoding related work with a passion before I started using Rust. Rust forced me to do it the right way and actually understand why I am doing it a certain way. When it comes to encoding I feel safer in Rust then e.g. in Python, despite having used it for four times as long.
• camgunz 7y ago
  Using any UTF encoding as fixed-width is almost always a mistake. There's no concept of "characters" in Unicode or the UTF encodings, they use codepoints, and those are only very rarely useful--you cannot treat them as "characters". This is a common misconception with UTF encodings.