2 ms·
Overlong forms would break a lot of useful properties of UTF-8 that rely on the encoding being unique, most important of all perhaps being compatibility with AS
by account42 4y ago
Overlong forms would break a lot of useful properties of UTF-8 that rely on the encoding being unique, most important of all perhaps being compatibility with ASCII tools.
Surrogates however have only one encoding (except for overlong forms) so this is not the same problem at all. You could say there is a disambiguity with decoding matching surrogate pairs in WTF-16 but this is a WTF-16 issue and would not exist if "UTF-16" would not have been built by requiring to carve out unicode code points.
> The choices are: encode that U+DEAD code point into UTF-8, thereby punting the problem to whoever/whatever decodes it, or else flag it here.
And encoding it in WTF-8 is the better choice because it retains more information, allowing you to perfectly round trip back to original WTF-16 string. But either way, handling unmatched surrogate pairs is not really a concern of the UTF-8/WTF-8 encoding (they are just code points as far as it is concerned) but of the UTF-16/WTF-16 decoding.