3 ms·
"it's much less likely to run into issues like String.length returning 32 yet only containing 31 characters" This is exactly the problem with UTF-16. Most APIs
by mjg59 15y ago
"it's much less likely to run into issues like String.length returning 32 yet only containing 31 characters"
This is exactly the problem with UTF-16. Most APIs that use it will have support for string operations that return the number of codepoints rather than the number of bytes, and as a result people think that it's a solved problem. But in fact you've only solved half the problem, because the number of codepoints is almost certainly not what you want - you want the number of characters, and the only way your library functions can know that is to know which Unicode codepoints are combining characters. And that set potentially gets larger with every new Unicode release.
In other words, if you're relying on languages that have native UTF-16 support to tell you the number of printable characters, your application is inevitably going to be broken the first time someone uses a newly-defined combining character. UTF-16 buys you absolutely nothing useful in this respect.
(Example: How many characters is "é"? "é"? "é"? Does UTF-16 give you a more useful answer to that question?)
- pjscott 15y ago> (Example: How many characters is "é"? "é"? "é"? Does UTF-16 give you a more useful answer to that question?) In case anybody was wondering: 1. The first "é" is an "e" followed by a combining acute accent. 2. The second "é" is a single code point for a lowercase-e-with-acute. 3. The third "é" is the same as the second "é", but with a zero-width non-breaking space in front of it. All of these are, of course, the same letter.