3 ms·
Yes, surrogate pairs is stupid. We should do away with UTF-16 and liberate the remaining 10 bits of codespace in UTF-8.
by cryptonector 3y ago
Yes, surrogate pairs is stupid. We should do away with UTF-16 and liberate the remaining 10 bits of codespace in UTF-8.
- kazinator 3y agoEasier said than done. Programs that don't care about surrogate pairs already use blocks like U+DCxx for their purpose. Liberating the code space for character use would be a poor idea; blessing it for local use would make more sense, but it would just be approving existing practices. I have an implementation in which codes U+DC00 through U+DCFF indicate "this was a bad byte in the original UTF-8". When UTF-8 is generated, these values reproduce the original invalid bytes. Additionally one of those values, U+DC00, is particularly useful since it encodes a null byte that occurred in the UTF-8 stream. While that is not strictly necessary as far as UTF-8 goes, the encoding provides a way to embed the null character in null-terminated strings.
- deleted 3y ago[deleted]