12 ms·
Kotlin: The Problem with null
- smitherfield 9y ago>Turns out Swift does not have null pointers. At all! Mostly true under the hood; optional value types are tagged unions (I didn't know this before reading this article and checking for myself), but optional reference types are still, as you'd expect, nullable pointers.[1] [1] https://godbolt.org/g/QZk6wL https://godbolt.org/g/QZk6wL
- taeric 9y agoThe java compiler might not know if a variable is safe to pass as null, but it is not unknowable. I think on every thread similar to this that I am on, I have plugged Coverity as a tool that can and will flag spots like this. That is, it won't just say "it is possible this will be a null pointer exception," instead it will say "setting this value to null will lead to this dereference of a null." It can look magical sometimes, because they will trace pretty deeply into your code. Which is just my way of saying over and over again that tooling can improve that does not require a complete rewrite of your software.
- drfuchs 9y agoIt sounds like you’re claiming that Coverity has a solution to the Halting Problem? That would indeed be magical.
- phpnode 9y agoIt doesn't sound like the OP is claiming that at all, Coverity and tools like it use a technique called Symbolic Execution [0] [0]. https://en.wikipedia.org/wiki/Symbolic_execution https://en.wikipedia.org/wiki/Symbolic_execution
- taeric 9y agoUh, no? Null pointers are far from the halting problem, though. And to be clear, there are still limitations. It can't say if any value you pass into a function that is not part of the current compilation is safe. For somewhat obvious reasons. But for a large large class of bugs, using advanced tooling can go a long way. As evidenced by the power of the advanced tooling new languages bring into the compilers. :)
- drfuchs 9y agoThe claim seems to be that the tool will always tell you absolutely whether a null-dereference will happen, and never say “maybe.” The Halting Problem is trivially reducible to this (just put an intentional null-deref in front of each halt).
- taeric 9y agoNot necessarily the same point being made. At least, they are subtly different, to my understanding. It can tell you that, "if this line is reached, and it was called from this path (with evidence on how this could happen), it will dereference a null." It is not saying that "running this program will guarantee give you a null dereference." The halting problem is more in line with "this program will terminate on any possible inputs", which is more expansive. It is trivial to say that you can prove some programs won't terminate on a particular input. Question is if you can do it for all inputs, no?
- zenhack 9y agoRice's theorem is a bit more direct here: C is Turing complete, some C programs derefernce null pointers, and some don't. Therefore, the question of whether a an arbitrary C program derefernces a null pointer is in general undecidable. But I don't think the Op was claiming it was possible to do this perfectly, just possible to write very useful tools. You will always have either flase positives or false negatives (or I suppose inputs where you just hang). Turing completeness is the bane of static analysis, but that doesn't make it a fruitless endeavor.
- UncleMeat 9y agoI wouldn't say that Rice's thm is the bane of static analysis. It is what makes the field interesting. If the problems were not undecidable then we wouldn't have do so many interesting and challenging things to make working tools.
- Gibbon1 9y agoReminds me c compilers will bitch if a variable is unused. That's basically the same problem. And sometimes the compiler will flag something that won't happen because the code path is a phantom[1]. [1] Recent headline 'Phantom Code Paths Found Harmful'
- bitL 9y agoOnly the experimental Coverity Hypercomputing branch ;-)
- slezakattack 9y agoI think you may be conflating the Halting Problem. The problem simply states whether it can be determined that a program, given a set of inputs, will eventually halt. Attempting to use a null pointer can be deterministic at compile time but it's not telling you that your program would halt or not.
- drfuchs 9y agoA tool that can reliably tell whether a given program will hit a null deference or not can also trivially be repurposed to solve the Halting Problem. Thus, no such perfect deref tool can exist. The best you can claim is that there’s a tool with few false positives and false negatives. But the original post seems to claim that there’s a perfect one.
- orf 9y agoSure, for an arbitrary program that is. One that is annotated to specify if a function returns/accepts null is a subset of this that is solveable.
- inlined 9y agoThe halting problem is used as a means to stick one's head in the sand. The halting problem imo is fundamentally uninteresting. A question that's almost as useful and is answerable is whether a program might not halt. Using your mapping to nullability, this tries to answer whether a pointer might be null. That's all the tools are trying to do and the halting problem doesn't get in the way.
- brabel 9y agoNo, it can't. You are just playing around with a problem you don't understand (null checks have 0 to do with the halting problem, it's always possible to know that a value will never be null in Java at compile time as long as you have all the source code - no dynamic libraries loaded at runtime)... unless you can actually show the proof of that, that would be really interesting to see.
- librexpr 9y agoWhile the halting problem can be pretty important in some cases, it's not very relevant in most day to day code. If you're not sure if your code will ever terminate, that usually means that you're doing something wrong. So sure, sometimes Coverity will fail because of the halting problem, and in those cases it can give an appropriate error message. Most of the time, though, it'll work just fine and be a very useful tool.
- RandomBK 9y agoThe issue on the Java side is more one of compatibility than technical feasibility. IDEA and Eclipse have been able to infer potential null issues for a while now, but it's difficult to retrofit those features into a compiler that must also accept code written 15 years ago.
- hacker_9 9y agoOne major difference is that Kotlin can tell you about nullability problems from simply analysing the method body, without going any further down the execution tree. this is a lot more reliable, and fast enough for real time IDE error reporting, than doing symbolic execution and diving into every method call to figure out all the possible paths. Same with Rust: it can determine multi threading issues from the surface layer, whereas in other languages in order to detect data races and contention you need some serious tracing analysis as well as doing a lot of runtime profiling.
- taeric 9y agoAh, please don't take my point as dismissing the new tools, either. Indeed, I would hope that a synergy between all tools lead to better tools for us all. I just have a pet peeve for the attitude of jumping rather quickly to "rewrite it all in new language" and not realizing you can bring a lot of this into existing codebases.
- stewbrew 9y agoI don't quite get the problem as stated in the article. I assume it's a contrived example and the real (kotlin) code rather looks like: val foo: T? = null Observable.just(foo).subscribe() Here foo is defined as nullable, hence the problem. If you make it val foo: T = null Observable.just(foo).subscribe() the kotlin compiler will bark on foo being null and you'll never have a runtime error. I think the motivating example should be described better but maybe I misunderstand it.
- kodablah 9y agoI think the point is different. Author is saying Java interop is fraught with runtime nullable issues that can't be caught at compile time for Java libs sans annotations. It has nothing to do with how it's set in Kotlin. Author says with nullable annotations on tge Java side, it'd bark at your first example too which is the real point.
- stewbrew 9y agoThe author introduces his example with "Let’s take a look at a very short Kotlin + RxJava sample." But IMHO his sample describes a Java-only problem (and solution). He could have written a quite similar article without any mentioning of kotlin.
- kodablah 9y ago> He could have written a quite similar article without any mentioning of kotlin Almost. The point about the language itself respecting the Java nullability annotations is apt.
- masklinn 9y ago> I don't quite get the problem as stated in the article. > Here foo is defined as nullable, hence the problem. How do you know it's "the problem"? The answer is that you don't until it blows up in your face, because this is a Java interop call. And of course the local could come from an other call into Java which returns a nullable reference, or it could make complete sense for it to be nullable.
- kodablah 9y agoThey could have gone hardcore and said any non-primitive, unanotated platform type is nullable but that would have made interop really ugly. And all of the null checks would have muddied up code and added (admittedly minimal) runtime costs. I want something in between Kotlin and Scala. I want option as a real type that is treated as a max-single-element collection that can be flattened with the same APIs as other collections. But I want no runtime penalty. People in Rust are so lucky to have zero cost abstractions for these things. I suppose I'll need better compile time support (even more than Scala macros) and whole program optimizations (i.e. cross JAR) to get zero cost optional on the JVM. Things like Scala Native use LLVM and surely Option things are inlined or otherwise optimized out.
- markcerqueira 9y ago> They could have gone hardcore and said any non-primitive, unanotated platform type is nullable but that would have made interop really ugly. Instead you just get a crash if it is null which is worse in my opinion. > And all of the null checks would have muddied up code and added (admittedly minimal) runtime costs. This already happens automatically for parameters to a Kotlin function; check out the Kotlin Intrinsics checks.
- kodablah 9y ago> Instead you just get a crash if it is null which is worse in my opinion Only in the same instances you would in Java-to-Java. So the ergonomics aren't improved or reduced. And really, this only helps on returns anyways. Marking every Java param nullable gets you nothing if the implementer didn't handle it well. > This already happens automatically for parameters to a Kotlin function; check out the Kotlin Intrinsics checks. Yup, and I don't like it. So even public Kotlin-to-Kotlin calls suffer. Haven't checked in a while, but I would like an option to use annotations only and skip those top-of-method checks.
- brabel 9y agoRelevant: https://discuss.kotlinlang.org/t/run-time-null-checks-and-performance/2086/2 https://discuss.kotlinlang.org/t/run-time-null-checks-and-pe... I've measured the null-checks and other people have too... the runtime performance cost is negligible, so disabling them would get you nothing useful. If they did, Kotlin would have made the compiler flag to disable them (which exists) public... but until someone comes up with an actual reason other than "I feel like it's better" that won't change.
- phreack 9y agoWell, the null issues in Java libraries have always been there, I don't really think it's a fault of Kotlin and its tooling that those bugs are not caught at all by the compiler, as all object types in Java are Kotlin optionals if not annotated. Maybe in time most of the necessary open source projects will be pure Kotlin ones, but for now I believe annotating Java code with null constraints is certainly a boon for everyone and should be encouraged.
- mkobit 9y agoI agree that the tooling is getting better. Some of the popular libraries are still figuring out the right patterns to apply to make them more "Kotlin friendly". Tools like the Checker Framework [1], JSR 308 [2], and compiler plugins like Traute [3] can also help with the safety issues on the Java side. Kotlin is beginning to understand these more, and some of the basic annotations are supported, but the tooling is only going to improve with all of the backing that Kotlin currently has. [1]: https://checkerframework.org/ https://checkerframework.org/ [2]: https://jcp.org/en/jsr/detail?id=308 https://jcp.org/en/jsr/detail?id=308 [3]: https://github.com/denis-zhdanov/traute https://github.com/denis-zhdanov/traute
- _Codemonkeyism 9y agoShameless little self promotion "Comparing Optionals and Null in Swift, Scala, Ceylon and Kotlin" http://codemonkeyism.com/comparing-optionals-and-null-in-swift-scala-ceylon-and-kotlin/ http://codemonkeyism.com/comparing-optionals-and-null-in-swi... Personally I like Monads better than the Kotlin solution. Best is probably the way Swift does it.
- trevor-e 9y agoIn Swift you can also reference the enum types directly, like `var h:String? = .some("Hello")`. You're also missing `??` for Swift's version of default values.
- greenhouse_gas 9y agoMy issue with Monads (and why I like kotlin and swift nullable types) is that it allows the compiler to infer nullability like the following code: fun f(a1: Int?, a2: Int?): Int { if (a1 == null || a2 == null) { return 0 } //now you can do something like return a1+a2 } In contrast, if it didn't, the code would look like: fn func1(q: Option<i32>, z: Option<i32>) -> i32 { if let Some(q_in) = q { if let Some(z_in) = z { return q_in + z_in } } return 0 } Note the deeply nested parenthesis.
- yiransheng 9y agoIn languages like Haskell and Scala that supports `do` / `for` comprehension sugar, this could be done without the nesting: fn :: Maybe Int -> Maybe Int -> Int fn q z = fromMaybe 0 $ do q_in <- q z_in <- z return (q_in + z_in)
- yxhuvud 9y agoAfter coding enough Crystal, which instead of optionals have raw and anonymous sum types, I can say this: I don't see the point of a type system where nil is a special case. I don't want optionals, I want to have nil as a totally separate type.
- anaphylactic 9y agoOptionals are sum types, too! No special-casing there. It's just that Crystal just has anonymous sum types, I guess, which can be a lot nicer to look at.
- brabel 9y agoOptional is a special case of a sum type where one of the types is Null. If you have Null as a type whose only value is null, then an Optional is just a T|Null. No need for Optional to exist.
- Groxx 9y agoRaw being something like `var x : int | str`, and anonymous like `var x = rand() ? 1 : ''` (where int|str is inferred)? Or do you mean something else?
- yxhuvud 9y agoYes, that is what I meant, yes.
- jontro 9y agoAt least for android kotlin programming, with the release of api level 27 + support libraries, it’s a lot better. They have placed the annotations all over the place. Using nullable annotations have good use cases in java too. I would think optional types and everything else being non nullable is the way forward for java too
- user2994cb 9y agoAren't null pointers just a variation on the Maybe Monad?
- anaphylactic 9y agoMaybe is just another Optional type, but with a much weirder name, so that's pretty obvious. But this makes me wonder - does Haskell optimize `Maybe`-like types down to a null-pointer, like Rust does?
- user2994cb 9y agoDunno, but I'd have thought that you can make None look like a null pointer, but Some(p) needs to be tagged in some way.
- user2994cb 9y agoActually, having Some(x) be just a pointer to x seems to work fine without an explicit tag, as dragonwriter says below.
- simon_o 9y agoNo, because Option/Maybe can nested, nulls can't.
- dragonwriter 9y agoNulls can be nested if you are using a language with explicit pointer types. E.g., Int*** x; Can be a lot like Maybe Maybe Maybe Int It's just that in languages where a type like Integer really means “an integer or a null", the nullability doesn't nest, and no one wants to deal with naked pointers.
- Zpalmtree 9y agoYou're forced to check for a Nothing value, wheras with a null pointer you can happily use it like a normal value.
- 9y ago
- xenadu02 9y ago>Unfortunately, it is not really possible to change Kotlin behave the same way. Apple uses a bridging mechanism to connect Objective-C and Swift binaries, when Kotlin uses the same bytecode as Java. To make a simpler mental picture, imagine Objective-C and Swift being connected side-by-side and Kotlin and Java as a stack, where Kotlin is on top. I presume it would be pretty challenging to provide a proper compatibility with Java, transforming all nullable Kotlin values to Optional and vice-versa, especially in such tight areas like Java reflection. This is not correct. The Kotlin compiler could treat every parameter and return value that doesn't have a nullability annotation as an implicitly unwrapped optional (Type!). It could even support all of the popular nullability annotations. There is no requirement to choose just one. The beauty of IUO is that you can ignore the nullability and you have the same amount of safety you have today: if you touch a null value you get an exception (or abort in Swift's case). The benefit is you can insert a null check in one place which then propagates through the rest of your Kotlin (or Swift) code.
- dcow 9y agoYeah (agree) that's not correct. Kotlin has to compile to Java bytecode, true, but it does not overlay Java like e.g. Groovy does. Kotlin and Java very much exist side-by-side in exactly the same way Swift and objective-c do: they (can) target the same machine but have different language scemantics.
- pvg 9y agoThe Chris Lattner quote at the end explains the fundamental difference. Kotlin is an interloper in a Java world - that includes the JVM, a VM designed to run Java. Swift integrates and interoperates with but is not based on the Objective C runtime. Apple also controls all the relevant bits and pieces and they can bridge, compile-to, shim, wrap, etc however they want. They can can pick the design approach and fiddle with all the parts to make them fit. This isn't a luxury Kotlin has.
- peterashford 9y agoThat would have been true to say if this was a decade ago. After the advent of .NET and CLI's ability to support multiple languages, Sun started moving the JVM away from being Java specific and including features to support other languages. Your criticism hasn't been correct for years.
- flareback 9y agoIt seems like a good article but the grammar makes it difficult to read at times.
- thought_alarm 9y ago.online is a terrible TLD.
- paulddraper 9y agoI wrote a post a few years ago comparing nullable types to maybe/optional. https://www.lucidchart.com/techblog/2015/08/31/the-worst-mistake-of-computer-science/ https://www.lucidchart.com/techblog/2015/08/31/the-worst-mis... A lot of it is a more specific cas of sum types vs union types.