3 ms·
I looked it up. The one place where UTF-16 validation takes place is when using WideCharToMultiByte to convert to UTF-8 text. Before Vista, that was not valid
by Dwedit 13d ago
I looked it up. The one place where UTF-16 validation takes place is when using WideCharToMultiByte to convert to UTF-8 text. Before Vista, that was not validated.
- account42 13d agoWhat do you mean by not being validated. UTF-16 to UTF-8 conversion necessitates special treatment for surrogates to correctly convert matched pairs of them to their proper UTF-8 encoding of the code point they represent. The question is what you do when you encounter unmatched pairs: - Abort with an error (not useful) - Replace the unmatched surrogate with a replacement character (afaik that's what WideCharToMultiByte has always done_ - Treat unmatched surrogates like any other non-surrogate code unit and encode the code point they represent (which are reserved for surrogates) as UTF-8 like you would any other code unit. This gets you the WTF-8 encoding which is what you want if you need to lossless represent Windows almost-UTF-16 strings de-facto-but-no-de-jure-UTF-8.