3 ms·
> But the 16 bit string character representation doesn't work well with modern Unicode. What do you mean? UTF-16 works well with all unicode characters. Some c
by vbezhenar 6y ago
> But the 16 bit string character representation doesn't work well with modern Unicode.
What do you mean? UTF-16 works well with all unicode characters. Some characters are encoded by two code units, but that's true for UTF-8 as well. ASCII strings are stored using 1-byte encoding on modern JVMs. I would prefer UTF-8, but hybrid ASCII/UTF-16 string works fine too.
- nradov 6y agoThe problem is that the internal character encoding is fixed to UTF-16 (although some JVMs use something else internally and then present a UTF-16 facade). A lot of Java code fails to correctly handle Unicode code points that don't fit into a single char. In retrospect string representation should have been more flexible and decoupled at the API layer from the underlying encoding.
- LoSboccacc 6y agoI've never had any issue for as long as every I/O had their charset properly declared, do you have done samples to share?
- maple3142 6y agoMaybe reversing or splitting a Unicode string (with surrogate pairs) may be a problem in Java. I am not familiar with Java, so I am not sure.
- nradov 6y agoIO isn't the problem. The issue is that a lot of code assumes that each string character is represented by a single char value, which isn't correct.
- LoSboccacc 6y agoyou can't do that in Java, string length is computed correctly for you and character loops give you normalised code points you have to go at greath length to get a broken character out of a valid string, including casting and cutting it intentionally