5 ms·
Java Unicode escapes are kind of a mess, and the raw string literals proposal only makes it worse. Unicode escapes are processed in Java not just inside string
by moefh 8y ago
Java Unicode escapes are kind of a mess, and the raw string literals proposal only makes it worse.
Unicode escapes are processed in Java not just inside string literals, but everywhere in the source code. So, for example, the following program prints "Hello, world!" even though that line of code seems to be commented (\u000a is new line, so it ends the comment):
public class Test {
public static void main(String[] args) {
// \u000a System.out.println("Hello, world!");
}
}
Moreover, a \u000a inside a string literal is the same as an actual newline, so the compiler doesn't accept it:
Test2.java:3: error: unclosed string literal
String s = "\u000a";
^
But now with the raw string literal proposal, JEP 326[1], Unicode escape processing is disabled inside raw string literals, and \u0060 escapes (backticks) aren't considered backticks for the purposes of starting raw string literals.
So, with this proposal, Unicode escapes are in a worst-of-two-worlds middle way:
1) They can't be handled uniformly at a low level anymore, so a Java parser can't naively convert escapes while reading the source file, but
2) They must still be naively interpreted in unexpected places like comments and normal string literals, as shown in my examples.
What a mess.
[1] https://openjdk.java.net/jeps/326 https://openjdk.java.net/jeps/326
- yrro 8y agoDear God Why did they do it like this Just why
- elygre 8y agoWhy did they do what?
- jackewiehose 8y agoprobably this very big WTF: // \u000a no comment
- sebazzz 8y agoWhat is the reason for Unicode escapes? It is something legacy from when Unicode was not universally accepted in all systems? Do other programming languages have this?
- chrisseaton 8y agoSeveral programming languages have arcane ways to write what today seem like very normal characters. https://en.wikipedia.org/wiki/Digraphs_and_trigraphs https://en.wikipedia.org/wiki/Digraphs_and_trigraphs For example C has escape sequences for characters as basic as # and [.
- zapzupnz 8y agoDoesn't it have escape sequences for those because they're used in the language's syntax? Stands to reason, or is this even in string literals?
- klodolph 8y agoThey are alternative ways of writing the same character, unlike escape sequences, which have a different meaning from the unescaped version of the same character. For example, in C, I can write: int a<:10:>; Or I can write: int a[10]; They are the same, because <: is exactly the same as [, and :> is exactly the same as ] (except… mercifully, digraphs are not interpreted in strings; trigraphs are another story). However, in a string, the escape sequence \" means something different from ". One is a character in the string, the other ends the string. Digraphs and trigraphs exist because not all keyboards have the corresponding characters, like [ and ]. For example, look at a Swedish keyboard. (You can type [ and ], but not as easily, and maybe not at all on some older keyboards.)
- Aardwolf 8y agoWow, that is similar to the C++ trigraphs, where using ?? followed by various characters inside a string literal (and any other place) does not work as expected But those in C++ are deprecated, yay!
- userbinator 8y agonot just inside string literals This is the most unusual part and just leaves me asking why!?!? Was it an error of omission, or a deliberate design decision, to essentially preprocess the whole file blindly with Unicode escape parsing instead of putting it only inside the string literal (and character constant) parser like just about every other language that has a similar escape system? The fact that \n behaves differently (and in the sane manner that those are accustomed to from C/C++) is the most bizarre part --- Unicode escaped could've been handled by the same parser, but they didn't. Sibling comments mention trigraphs. Those are different, starting with ?? instead of \, so their "applicable to the whole file" behaviour is more reasonable. But \-escapes are, outside of preprocessor line continuations (where it's literally \ followed by a newline), customarily only interpreted inside character and string constants.
- teraflop 8y agoIt was a deliberate decision. The Java language specification [1] says: > The Java programming language specifies a standard way of transforming a program written in Unicode into ASCII that changes a program into a form that can be processed by ASCII-based tools. The transformation involves converting any Unicode escapes in the source text of the program to ASCII by adding an extra u - for example, \uxxxx becomes \uuxxxx - while simultaneously converting non-ASCII characters in the source text to Unicode escapes containing a single u each. The Java language grammar is defined in terms of Unicode, but it was designed in the early days before Unicode was ubiquitous. In particular, this was long before UTF-8 took its current place as the de facto standard character encoding. The designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well. But they also wanted to preserve interoperability between systems using different character encodings. It's certainly confusing that \u behaves differently from other backslash escape sequences, but I guess they figured that was less bad than introducing an entirely new escape character. [1]: https://docs.oracle.com/javase/specs/jls/se11/html/jls-3.html#jls-3.3 https://docs.oracle.com/javase/specs/jls/se11/html/jls-3.htm...
- userbinator 8y agoThe designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well. In my experience (although not with Java; maybe its users would make more use of the feature, but I doubt it) even developers whose native language is not English but something very different like Chinese or Japanese will continue to use ASCII-only identifiers, and only write things like string constants or comments in their native language (which may mean using high bytes and a non-UTF8 multibyte encoding.) What the identifiers mean may be Romanised foreign words, but they're still written with [A-Za-z0-9_].