2 ms·
Missing support for characters beyond U+FFFF is the main problem caused by surrogates (their existence, even if indirect)—it normally comes of some kind of UCS-
by chrismorgan 5y ago
Missing support for characters beyond U+FFFF is the main problem caused by surrogates (their existence, even if indirect)—it normally comes of some kind of UCS-2/UTF-16 confusion. It’s not fair to disqualify them. The only (class of) case that I’m aware of for a long time where it’s not linked to that is with MySQL’s idiotic utf8 → utf8mb3 type.
You may not have encountered such bugs, but I’m very familiar with surrogate-related bugs, because I use a Compose key extensively. I haven’t been using Windows for the last year, but from time to time I would definitely encounter bugs that are certainly due to surrogates. On the web, I found bugs a few times, all but once in Rust WebAssembly things, such as https://github.com/Pauan/rust-dominator/issues/10 https://github.com/Pauan/rust-dominator/issues/10. And even now I’m back on Linux, I know of one almost certainly surrogate-related bug: I can’t type astral plane characters in Zoom at all; pretty sure I had this problem back on Windows, too. Copy and paste, sure, but type, no, they become REPLACEMENT CHARACTER.
The history is unfortunate but I strongly refute that they had not much choice. UCS-2 should have been abandoned as a failed experiment. Certainly there had been significant investment into it in the last few years, but with the benefit of hindsight, switching to UTF-8 (which was invented before they decided on surrogates) would have made everyone’s life much easier, especially given its ASCII-compatibility, which would have allowed the Windows API to retire the misbegotten A/W split and return to sanity a few decades early.
Ah, BOMs. Haven’t seen one in years. Good riddance.