4 ms·
I'm curious if the internal representation will stay UTF-16? Someone smarter than I probably could say there's an advantage to using a fixed-length charset for
by exabrial 5y ago
I'm curious if the internal representation will stay UTF-16? Someone smarter than I probably could say there's an advantage to using a fixed-length charset for internal purposes.
- golergka 5y agoUTF16 is not fixed length, it has surrogate pairs.
- pdpi 5y agoUTF-16 isn't fixed length. It's sort of an extension of UCS-16, which was fixed length, but couldn't give you more than 64k codepoints. Internal representation is... complicated. java.lang.String uses UTF-16, but .class files use UTF-8.
- pjmlp 5y agoAnd since Java 9 there is some compression going on.
- colejohnson66 5y agoNitpick: UCS counts the bytes, not the bits. So UTF-16, but limited to the BMP, is UCS-2, and UTF-32 is UCS-4.
- pdpi 5y agoOooops. You're absolutely right.
- pjmlp 5y agoThe internal representation isn't pure UTF-16 for quite some time now. https://openjdk.java.net/jeps/254 https://openjdk.java.net/jeps/254
- bjoli 5y agoThe java internal representation previously had a look up table for each Nth char (let's say 128, which is a nice round number), and if the indexes increased by 128 between every 128 chars it simply did a linear lookup. For most text this means constant time access, instead of a linear search through the 128 chars. I don't think the benefits against a utf-8 system with a LUT would be very big, but as a retrofitted system (which it was) it is not bad. Pretty compact with mostly constant time access.