3 ms·
not just inside string literals This is the most unusual part and just leaves me asking why!?!? Was it an error of omission, or a deliberate design decision, t
by userbinator 8y ago
not just inside string literals
This is the most unusual part and just leaves me asking why!?!? Was it an error of omission, or a deliberate design decision, to essentially preprocess the whole file blindly with Unicode escape parsing instead of putting it only inside the string literal (and character constant) parser like just about every other language that has a similar escape system? The fact that \n behaves differently (and in the sane manner that those are accustomed to from C/C++) is the most bizarre part --- Unicode escaped could've been handled by the same parser, but they didn't.
Sibling comments mention trigraphs. Those are different, starting with ?? instead of \, so their "applicable to the whole file" behaviour is more reasonable. But \-escapes are, outside of preprocessor line continuations (where it's literally \ followed by a newline), customarily only interpreted inside character and string constants.
- teraflop 8y agoIt was a deliberate decision. The Java language specification [1] says: > The Java programming language specifies a standard way of transforming a program written in Unicode into ASCII that changes a program into a form that can be processed by ASCII-based tools. The transformation involves converting any Unicode escapes in the source text of the program to ASCII by adding an extra u - for example, \uxxxx becomes \uuxxxx - while simultaneously converting non-ASCII characters in the source text to Unicode escapes containing a single u each. The Java language grammar is defined in terms of Unicode, but it was designed in the early days before Unicode was ubiquitous. In particular, this was long before UTF-8 took its current place as the de facto standard character encoding. The designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well. But they also wanted to preserve interoperability between systems using different character encodings. It's certainly confusing that \u behaves differently from other backslash escape sequences, but I guess they figured that was less bad than introducing an entirely new escape character. [1]: https://docs.oracle.com/javase/specs/jls/se11/html/jls-3.html#jls-3.3 https://docs.oracle.com/javase/specs/jls/se11/html/jls-3.htm...
- userbinator 8y agoThe designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well. In my experience (although not with Java; maybe its users would make more use of the feature, but I doubt it) even developers whose native language is not English but something very different like Chinese or Japanese will continue to use ASCII-only identifiers, and only write things like string constants or comments in their native language (which may mean using high bytes and a non-UTF8 multibyte encoding.) What the identifiers mean may be Romanised foreign words, but they're still written with [A-Za-z0-9_].
- teraflop 8y agoSure, but this was in the 1990s, when "internationalization" was The Next Big Thing. At the time, the Java team really didn't have any way of knowing for sure whether or not non-ASCII source code would catch on, because most existing languages didn't support it.
- aardvark179 8y agoI have seen quite a lot of source in Japanese where the language allows it easily, much of it was written by a Japanese partner and their customers so it made sense for them.
- jackewiehose 8y ago> The designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well That's bad! I recently noticed "my" C++ compiler allows you to write something like int mäin(){}. I didn't try it out but I guess it breaks with different file encodings. Please stay with ASCII!
- userbinator 8y agoDue to how they were written, a lot of compilers don't really care about character encoding in the sense that any bytes >127 were simply considered identifier characters because their tokenisers picked out the special characters and "anything else" would be classified with the usual identifier range. This meant you could use whatever single or multibyte encodings you wanted, as long as any trailing bytes of a single character didn't conflict with the special characters; with this, UTF-8 naturally works too.