2 ms·
Java strings are sequences of 16-bit units and don't guarantee UTF-16 validity. There are newer standard library APIs that perform astral-aware operations, thou
by hsivonen 10y ago
Java strings are sequences of 16-bit units and don't guarantee UTF-16 validity. There are newer standard library APIs that perform astral-aware operations, though.
- Manishearth 10y agoSo does regex matching break up paired surrogates? It seems like Java Characters can be from the higher planes. JS treats surrogates as their own kind of character, but I think Java only does this for unpaired surrogates, so the encoding is still multibyte. ICBW.
- throwawayish 10y agoJava, like NT, was originally UCS-2 and moved to UTF-16 when it became clear that UCS-2 won't suffice. APIs that operate on units (anything involving "char"s [the type], CharSequences and so on) are legacy and, when used, are almost certainly used incorrectly. Note that newer APIs on String also use the word "char", but the type "int", these are actually operating on code points and not CUs, ie. these APIs are correct. (Note that java.lang.Character is a char, so wrong. I think there is no CodePoint type or similar that wraps int.)
- hsivonen 10y agoIn Java regexps, the dot matches a code point or an unpaired surrogate, so if you have a proper surrogate pair, the dot matches the whole pair without splitting it. It does split grapheme clusters. It's unclear to me what you mean by "Java Characters". A Java char is a UTF-16 code unit, and the java.lang.Character boxes a char (16 bits unsigned). However, since Java 5, java.lang.Character has static methods for astral-aware UTF-16 operations where a code point is a Java int (32 bits signed).
- Manishearth 10y agoYeah, by Character I meant java.lang.Character. > the dot matches the whole pair without splitting it Yeah, so UTF16 in Java is being used as a variable-width encoding. Nice to know.