3 ms·
What is a “character” in Java? I’m guessing it’s not a unicode code point? Is it basically a byte? Go also has immutable strings which are basically just byte s
by throwaway894345 6y ago
What is a “character” in Java? I’m guessing it’s not a unicode code point? Is it basically a byte? Go also has immutable strings which are basically just byte slices (slice being a window into an array).
- hn_throwaway_99 6y agoJava basically uses UTF-16, thus, a char is 2 bytes. All characters in the Unicode Basic Multilingual Plane are encoded with just a single char, while characters in the Supplementary Plane (i.e those with code points above 64K) require 2 chars (4 bytes) in what is called "Surrogate Pairs". Interestingly, code points beyond the BMP used to be quite uncommon (Unicode was originally designed for just 64k code points), but now emoji are in there, which of course are used a lot.
- astrange 6y agoUsing UTF-16 is a mistake because it doesn't represent anything well. Even if you use UTF-32 you still can't fit all characters in a single codepoint because some of them (like flag emoji) are composed. So you should expect things to be variable length. But once you do, UTF-8 is more efficient than UTF-16 even if you're processing CJK text because it so often has ASCII characters mixed in. Cocoa/ObjC has the same problem - actually Java is based on Cocoa so that's probably where it came from.
- nearbuy 6y agoEven with the surrogate pairs, Java's choice strikes a compromise between the memory efficiency of UTF-8 and the performance of fixed size code points. Most of the string methods work as if all characters are two bytes. So if, for example, you have a string composed entirely of 4 byte emojis, charAt(10) would return the first part of the 6th emoji, not the 11th emoji. For many cases, strings won't use any characters outside of UCS-2. In these cases, programmers gain the performance advantages of fixed sized characters without worrying about the downsides of surrogate pairs. It doesn't have the small memory footprint of UTF-8, but one could argue that if memory size is really important and fixed size characters are not, you're better off using in-memory compression for your strings, which will beat UTF-8. If you are using characters outside of UCS-2, then Java's string methods are a potential source of confusion and bugs. There's always a trade-off, right?
- discreteevent 6y agoFrom java9 onward if all the characters in a string are ASCII then java will store them in a byte array with one byte per character. This ends up covering a lot of strings.
- alisonatwork 6y agoJava characters are 16 bit. Unfortunately many methods in the String API expect strings to be UCS-2 encoded, even though in practice nowadays they're UTF-16. But it's more complicated than that, especially since Java 9 which introduced compressed storage for strings that could fit into ISO-8859-1[0]. [0] https://openjdk.java.net/jeps/254 https://openjdk.java.net/jeps/254
- hn_throwaway_99 6y agoThanks for this! I stand corrected. The actual underlying storage switched from a char array to a byte array in Java 9: https://dzone.com/articles/going-beyond-java-8-compact-strings https://dzone.com/articles/going-beyond-java-8-compact-strin...