4 ms·
Surrogate pairs are such a useless thing to exist. A unicode character is just a number (of up to around 21 bits). The encoding of this number, and the unicode
by Aardwolf 3y ago
Surrogate pairs are such a useless thing to exist.
A unicode character is just a number (of up to around 21 bits). The encoding of this number, and the unicode value itself, can (and should) be made completely independent. Note how UTF-8 did not introduce any useless values in unicode itself. That's because this code is sane (if a bit inefficient due to over-focus on error correction) and correctly makes its encoded byte values independent of the numeric value being encoded.
For UTF-16, initially they thought 16 bits would be enough, and then they had to hack something on top of it to make it variable length. Fair enough, but even that hack, they could have done without introducing useless unicode values: they could have defined the variable length 16-bit code to use the unused codes to indicate variable length, and then add/subtract values while decoding it, that is only use the unused UTF-16 values to tell how to use the remaining UTF-16 bits to take existing consecutive values from unicode. This would have been not a bigger mess than surrogates are, be transparent to users (since the decoder deals with the math), and left the mess inside of UTF-16 only (which at this point they should have known was already game over since the planned 16 bits didn't fit) instead of infecting the rest with its mess.
So not only is UTF-16 the ugliest encoding ever by being the worst of both worlds (variable length and yet bigger than a byte and not ascii compatible) and damages both windows (ruined filename loading in multiplatform C libraries, horrible "wstrings" in windows code) and java/javascript (UTF-16 in the language while the internet they interface with are UTF-8, exposing programmers to "surrogates"), but in addition in the making of this ugly code they managed to also uglify unicode itself, and UTF-8 (which has to deal with the fact it can refer to surrogates), and any other future encoding of unicode too!