6 ms·

I don't care how they get there, as long as they get there. In this case they're realizing that NULL makes optimization difficult and are swerving into solving

I don't care how they get there, as long as they get there. In this case they're realizing that NULL makes optimization difficult and are swerving into solving this 27 year old instance of the "billion-dollar mistake." NPEs shouldn't exist in work-a-day code.

sam_lowry_ 4y ago
What syntactic sugar they offer is not clear from the mail. There are already lots of techniques, from annotations to Optional.ofNullable but none matches the simplicity of != null. They can also go the route of `effectively final` and make the compiler deduce nullness instead of marking it explicitly.
- yarg 4y ago
  There's not really a nice way of implementing the syntax without changing the language. Although perhaps they could resolve this with magic imports? import java.lang.magic.optional; public class Foo{ public static void main(String...args){ Foo? f = ...; if(f){ f.doThings(); } f.doThings(); //compile time error } } The idea being that importing the magic class enables the cleaner syntax.
  - usrusr 4y ago
    I doubt that it's possible to open the door for "language version by imports" without turning that decision into a forever-floodgate. Note that I am not entirely sure that it would be a bad decision (I do lean in that direction however), but it would definitely be a severe one. It would be like a variant of inviting Dracula across the doorposts, only that this one may or may not actually be a cool guy and not a vampire, but in any case he will sure bring all his friends, and their friends and so on forever.
    - yarg 4y ago
      With great power comes great responsibility.
- kaba0 4y ago
  Discussions on the mailing list usually tries to avoid the exact syntax for as long as possible to avoid bike-shedding. But according to this mail there would be a default nullability (hopefully not-nullable as the default, but it may depend on whether it is a value class or not), and ! and ? as modifiers (and * for type variables). So `Point p = null` won’t type check, you would have to mark it as `Point?` where Point is a value (primitive) class. There were also points that it should work with the usual type inference, e.g. `final var p = methodThatReturnsNullablePoint()` would be inferred to `Point?`.
  - aardvark179 4y ago
    Reference types probably can’t be made non-billable by default as that would break too much. But String! myString = “Definitely not null”; doesn’t look too bad. Edit: it’s worth saying that a lot of careful thought around compatibility goes into features like this. That means the core libraries should be able to transition to non-billable types where this is already a run time requirement, and existing compiled jars must continue to work.
    - vips7L 4y ago
      I don’t think anyone would really use String! everywhere.
      - aardvark179 4y ago
        I think you’ll see it creep into parameter types in the core JDK and it will work its way out from there.
    - kaba0 4y ago
      I think it can work in a binary-additive way — the class file gets a new attribute for nullness information, while missing it (older class files) default to “unspecified”. That can be safely consumed by new code, which will get the updated syntax with a new option (specified on a module/package/class basis) where ‘T’ means ‘T!’ and that should be a proper subset of previously compiling code, so I don’t think anything would break that way. Similarly to how generics and unchecked warnings were rolled out.
- vbezhenar 4y ago
  I think the general idea is pretty clear. `MyClass! myvar` is not supposed to accept nulls. Assigning null to `myvar` would throw exception, comparing `myvar` with null is pointless and probably should emit a warning. Or maybe even error. `MyClass? myvar` is supposed to accept nulls. Dereferencing this value without checking for null is either warning or error. `MyClass myvar` is supposed to be unspecified with regards to nulls. So basically current code. You can do whatever you want but you should refactor it to `MyClass!` or `MyClass?` some day.
- cogman10 4y ago
  It'll allow the JVM to more aggressively optimize the code (the main benefit). Annotations are, unfortunately, all over the board. There's a dozen different `NotNull` annotations out there. Having it in the language will make it the last solution to this problem. You can't do deduction at compile time because you'll end up with different method signatures if someone calls a method with a null vs not. (not to mention the problem with someone calling these methods with reflection) You can do deduction at runtime (and they likely will), but then you have to deal with the problem of unwinding an optimization when the hypothesis is proven wrong. That can be really costly when the optimization is dealing with memory layout. (Imagine an array of value types where you deduced they should be non-null, yet some unexpected path inserts a null. In that case you have to reallocate the entire array and it's elements in order to handle that). It's more than just syntactic sugar.
yarg 4y ago
I've alway felt "billion dollar mistake" is unreasonably hyperbolic. In retrospect it would be very nice to have a language level ability to define something as optional, and require existence checking for such values; But it wasn't so obvious at the time.
- highwind 4y ago
  I believe ML family of languages do not have null. And it was developed well before Java. So I don't think timing was the issue.
  - pjmlp 4y ago
    Eiffel also didn't, and additionally it supports value types (expanded classes).
- pkulak 4y ago
  Can you expand on that a bit? Why would having it built in to the language (Kotlin-style, for example) be better than not having the concept of null in the language at all and using something like the Rust Option? Personally, I like the Rust approach. It seems a bit more flexible and can evolve easier over time.
  - cogman10 4y ago
    I think the main issue with the Rust option is they ended up with language syntactic sugar anyways. So in some ways, it's worse than the kotlin-style because reading `let baz = foo?.bar` could have all sorts of weird type implications. What is baz? Is it Option<Bar>? Is it Result<Bar>? Is it some user defined enum<Bar>? Who knows! You have to find that by looking up the foo definition. For kotlin, the answer is simple. "baz is a nullable Bar" Rust did this because interacting with the enum directly was cumbersome. The concept of "no value" is so integral to day to day programming that elevating it into the type system with "nullable types" makes more sense to me vs using generics trickery. Even if it's slightly less "pure".
    - 4y ago
deleted 4y ago
[deleted]
saagarjha 4y ago
> In this case they're realizing that NULL makes optimization difficult How so? Null checks are mostly “free” on the JVM, they trap on the zero page and get converted to an exception via a signal handler.
- Someone 4y ago
  It’s not only the checks, but allotted storage. An Integer that’s guaranteed not to be null can be stored in 4 bytes, a “Point” class with two “int” fields that is guaranteed to be not null can be loaded in a 64-bit register, etc. That makes it easier for the compiler to generate good code.
  - pjmlp 4y ago
    Eiffel was already doing this with expanded classes and non void references, but yeah, who would have thought such details are relevant. :)
marginalia_nu 4y ago
Do NPEs exist in work-a-day code? I remember working on a codebase where everything had null check pyramids like if (foo != null) { if (foo.getBar() != null) { Bar bar = foo.getBar(); if (bar.getBaz() != null) { for (Quux q : bar.getBaz()) { if (q.getFoobar() != null) { ... } } } } but that was a long time a go in a code base that operated by passing humongous mega-objects with hundreds of fields (running into the limit of how many parameters could be in the constructor was a constant headache), it solved a lot of its problems by printing a stacktrace and shrugging, and objects would be half-constructed and methods would return null in all sorts of scenarios. In comparison, the entire codebase for marginalia search has about 65 nullchecks in total. A decent chunk of them are in dealing with older APIs that sometimes idiomatically use nulls to communicate things. java.util.Map, BufferedReader and the Servlet api in particular, the rest in dealing with the inherently noisy nature of crawl data. So while there are a few examples where there are nulls in the data, but they are almost always contained to the module that produced them and don't cross interface boundaries. It's very far removed from some pinnacle of code quality, but just gets a few engineering principles right that that other code base didn't.
- dangets 4y ago
  Maybe this is from my epoch of education, but with early Java null checking everything was promoted as a good practice. Even today, depending on what your teacher said was correct, your coding experiences (e.g. if you came from C++, if you read 'Effective Java'), you might still null check everything. Personally, I lean towards the "use nulls sparingly and be very explicit where null values are possible", use immutable objects etc. This reduces the unnecessary null-check noise and makes things more robust. In java, I even prefer to add @Nullable to fields to try to switch the default mindset to "assume non-null unless @Nullable annotation exists" - though another developer made a note on a code review that "that is not how Java is designed and @Nullable is just noise". Having nullability as part of the type makes everything more explicit, so I'm a huge fan. Beyond just the perf and safety benefits, it's a win for domain modeling.
pjmlp 4y ago
ML solved it in the 70's, Eiffel in the mid-90's, however just like NPE there were quite a few things that Java didn't adopt, and here we are.

sam_lowry_ 4y ago

What syntactic sugar they offer is not clear from the mail. There are already lots of techniques, from annotations to Optional.ofNullable but none matches the simplicity of != null. They can also go the route of `effectively final` and make the compiler deduce nullness instead of marking it explicitly.

yarg 4y ago
There's not really a nice way of implementing the syntax without changing the language. Although perhaps they could resolve this with magic imports? import java.lang.magic.optional; public class Foo{ public static void main(String...args){ Foo? f = ...; if(f){ f.doThings(); } f.doThings(); //compile time error } } The idea being that importing the magic class enables the cleaner syntax.
- usrusr 4y ago
  I doubt that it's possible to open the door for "language version by imports" without turning that decision into a forever-floodgate. Note that I am not entirely sure that it would be a bad decision (I do lean in that direction however), but it would definitely be a severe one. It would be like a variant of inviting Dracula across the doorposts, only that this one may or may not actually be a cool guy and not a vampire, but in any case he will sure bring all his friends, and their friends and so on forever.
  - yarg 4y ago
    With great power comes great responsibility.
kaba0 4y ago
Discussions on the mailing list usually tries to avoid the exact syntax for as long as possible to avoid bike-shedding. But according to this mail there would be a default nullability (hopefully not-nullable as the default, but it may depend on whether it is a value class or not), and ! and ? as modifiers (and * for type variables). So `Point p = null` won’t type check, you would have to mark it as `Point?` where Point is a value (primitive) class. There were also points that it should work with the usual type inference, e.g. `final var p = methodThatReturnsNullablePoint()` would be inferred to `Point?`.
- aardvark179 4y ago
  Reference types probably can’t be made non-billable by default as that would break too much. But String! myString = “Definitely not null”; doesn’t look too bad. Edit: it’s worth saying that a lot of careful thought around compatibility goes into features like this. That means the core libraries should be able to transition to non-billable types where this is already a run time requirement, and existing compiled jars must continue to work.
  - vips7L 4y ago
    I don’t think anyone would really use String! everywhere.
    - aardvark179 4y ago
      I think you’ll see it creep into parameter types in the core JDK and it will work its way out from there.
  - kaba0 4y ago
    I think it can work in a binary-additive way — the class file gets a new attribute for nullness information, while missing it (older class files) default to “unspecified”. That can be safely consumed by new code, which will get the updated syntax with a new option (specified on a module/package/class basis) where ‘T’ means ‘T!’ and that should be a proper subset of previously compiling code, so I don’t think anything would break that way. Similarly to how generics and unchecked warnings were rolled out.
vbezhenar 4y ago
I think the general idea is pretty clear. `MyClass! myvar` is not supposed to accept nulls. Assigning null to `myvar` would throw exception, comparing `myvar` with null is pointless and probably should emit a warning. Or maybe even error. `MyClass? myvar` is supposed to accept nulls. Dereferencing this value without checking for null is either warning or error. `MyClass myvar` is supposed to be unspecified with regards to nulls. So basically current code. You can do whatever you want but you should refactor it to `MyClass!` or `MyClass?` some day.
cogman10 4y ago
It'll allow the JVM to more aggressively optimize the code (the main benefit). Annotations are, unfortunately, all over the board. There's a dozen different `NotNull` annotations out there. Having it in the language will make it the last solution to this problem. You can't do deduction at compile time because you'll end up with different method signatures if someone calls a method with a null vs not. (not to mention the problem with someone calling these methods with reflection) You can do deduction at runtime (and they likely will), but then you have to deal with the problem of unwinding an optimization when the hypothesis is proven wrong. That can be really costly when the optimization is dealing with memory layout. (Imagine an array of value types where you deduced they should be non-null, yet some unexpected path inserts a null. In that case you have to reallocate the entire array and it's elements in order to handle that). It's more than just syntactic sugar.

yarg 4y ago

I've alway felt "billion dollar mistake" is unreasonably hyperbolic. In retrospect it would be very nice to have a language level ability to define something as optional, and require existence checking for such values; But it wasn't so obvious at the time.

highwind 4y ago
I believe ML family of languages do not have null. And it was developed well before Java. So I don't think timing was the issue.
- pjmlp 4y ago
  Eiffel also didn't, and additionally it supports value types (expanded classes).
pkulak 4y ago
Can you expand on that a bit? Why would having it built in to the language (Kotlin-style, for example) be better than not having the concept of null in the language at all and using something like the Rust Option? Personally, I like the Rust approach. It seems a bit more flexible and can evolve easier over time.
- cogman10 4y ago
  I think the main issue with the Rust option is they ended up with language syntactic sugar anyways. So in some ways, it's worse than the kotlin-style because reading `let baz = foo?.bar` could have all sorts of weird type implications. What is baz? Is it Option<Bar>? Is it Result<Bar>? Is it some user defined enum<Bar>? Who knows! You have to find that by looking up the foo definition. For kotlin, the answer is simple. "baz is a nullable Bar" Rust did this because interacting with the enum directly was cumbersome. The concept of "no value" is so integral to day to day programming that elevating it into the type system with "nullable types" makes more sense to me vs using generics trickery. Even if it's slightly less "pure".
  - 4y ago

deleted 4y ago

[deleted]

saagarjha 4y ago

> In this case they're realizing that NULL makes optimization difficult How so? Null checks are mostly “free” on the JVM, they trap on the zero page and get converted to an exception via a signal handler.

Someone 4y ago
It’s not only the checks, but allotted storage. An Integer that’s guaranteed not to be null can be stored in 4 bytes, a “Point” class with two “int” fields that is guaranteed to be not null can be loaded in a 64-bit register, etc. That makes it easier for the compiler to generate good code.
- pjmlp 4y ago
  Eiffel was already doing this with expanded classes and non void references, but yeah, who would have thought such details are relevant. :)

marginalia_nu 4y ago

Do NPEs exist in work-a-day code? I remember working on a codebase where everything had null check pyramids like if (foo != null) { if (foo.getBar() != null) { Bar bar = foo.getBar(); if (bar.getBaz() != null) { for (Quux q : bar.getBaz()) { if (q.getFoobar() != null) { ... } } } } but that was a long time a go in a code base that operated by passing humongous mega-objects with hundreds of fields (running into the limit of how many parameters could be in the constructor was a constant headache), it solved a lot of its problems by printing a stacktrace and shrugging, and objects would be half-constructed and methods would return null in all sorts of scenarios. In comparison, the entire codebase for marginalia search has about 65 nullchecks in total. A decent chunk of them are in dealing with older APIs that sometimes idiomatically use nulls to communicate things. java.util.Map, BufferedReader and the Servlet api in particular, the rest in dealing with the inherently noisy nature of crawl data. So while there are a few examples where there are nulls in the data, but they are almost always contained to the module that produced them and don't cross interface boundaries. It's very far removed from some pinnacle of code quality, but just gets a few engineering principles right that that other code base didn't.

dangets 4y ago
Maybe this is from my epoch of education, but with early Java null checking everything was promoted as a good practice. Even today, depending on what your teacher said was correct, your coding experiences (e.g. if you came from C++, if you read 'Effective Java'), you might still null check everything. Personally, I lean towards the "use nulls sparingly and be very explicit where null values are possible", use immutable objects etc. This reduces the unnecessary null-check noise and makes things more robust. In java, I even prefer to add @Nullable to fields to try to switch the default mindset to "assume non-null unless @Nullable annotation exists" - though another developer made a note on a code review that "that is not how Java is designed and @Nullable is just noise". Having nullability as part of the type makes everything more explicit, so I'm a huge fan. Beyond just the perf and safety benefits, it's a win for domain modeling.

pjmlp 4y ago

ML solved it in the 70's, Eiffel in the mid-90's, however just like NPE there were quite a few things that Java didn't adopt, and here we are.