4 ms·
Yes, that's an ill-formed string. As I wrote, there's a long ways to go from a "sequence of valid code units" to "a valid string." But what I wrote is neverthe
by millstone 13y ago
Yes, that's an ill-formed string. As I wrote, there's a long ways to go from a "sequence of valid code units" to "a valid string."
But what I wrote is nevertheless correct, and matters. See for example the special requirements on UTF-8 processing in D93:
If the converter encounters an ill-formed UTF-8
code unit sequence...it must not consume the
successor bytes as part of the ill-formed
subsequence whenever those successor bytes
themselves constitute part of a well-formed UTF-8
code unit subsequence....For a UTF-8 conversion
process to consume valid successor bytes is not
only non-conformant, but also leaves the converter
open to security exploits."
"Ill-formed code unit sequence" is not possible with UTF-16. That's an advantage. Also note the last line: the existence of ill-formed code units has security implications that are not present with UTF-16.
To be fair, UTF-16 has its own unique issues too, like endianness.