3 ms·
UTF-8 should never contain surrogates as they serve no purpose when all code points can be encoded directly. That doesn't prevent programmers doing stupid thing
by kevin_thibedeau 3y ago
UTF-8 should never contain surrogates as they serve no purpose when all code points can be encoded directly. That doesn't prevent programmers doing stupid things with them but that's not a Unicode problem.
- comex 3y agoCorrect – but that’s exactly why you can’t just say “all strings are UTF-8, we’re done here”. Because Windows filenames can contain unpaired surrogates, which don’t have a sensible UTF-8 translation. (Unix has a similar problem with filenames being arbitrary bytes that may not be valid UTF-8.) Note that Rust has OsString to handle both of these cases, but it’s somewhat hard to use.
- masfuerte 3y agoWith networking code you sometimes need an OsString for the platform you're not on. Rust doesn't seem to support this, other than by writing your own.