6 ms·

Having fun with Go's nil, interfaces and errors

86 points by katcipis 10y ago

catnaroek 10y ago
> Why ? It seems to me that it happens because how interfaces are implemented Why on Earth should a user of a high-level language have to care about language implementation details? > What the actual fuck ??? Exactly.
- Vendan 10y ago
  Except it's not a language implementation detail, it's part of what the language is. You might as well ask why a user of another language needs to know what a class is, or how inheritance works. Too many people seem to be assuming "Oh, it's like void*, so it must act like that always!", but it's not.
  - catnaroek 10y ago
    > Except it's not a language implementation detail, it's part of what the language is. Okay, but what was the rationale for designing the language like that?
    - jerf 10y ago
      The last bit of information you need to understand that is that unlike some other languages you may be used to, "nil" is a perfectly valid value for a pointer to have. You can write methods like this: func (o *Object) ReturnSomething() int { if o == nil { return 0 } return o.SomeOtherValue } It is also therefore legal to return in an interface value a nil pointer to a struct of a particular kind, and therefore "an interface containing a nil pointer of a particular type" and "an interface containing nothing at all" are fundamentally different things that can not be collapsed together. So as Vendan says, it is just part of the language. All languages have this sort of wart in them, where two or more perfectly sensible decisions interact to create something that isn't sensible at first glance. (I'm still in favor of going back in time and changing Go to have non-nillable values, but that ship has certainly sailed.)
      - catnaroek 10y ago
        > unlike some other languages you may be used to, The languages I'm used to don't even have `nil`. > "nil" is a perfectly valid value for a pointer to have. So, just like C and Java. I don't see the difference. > You can write methods like this: (snippet) Oh, now I get it: `nil` itself isn't a value. `nil` is syntactic sugar that expands into a nil value. (In other words, confusingly enough, `nil` and “nil value” are different things!) This is unlike Java, where `null` itself is a value. It would be very helpful if people described things precisely. > All languages have this sort of wart in them, where two or more perfectly sensible decisions interact to create something that isn't sensible at first glance. This means that at least one of the two (or more) decisions was less sensible than it originally seemed. Good design decisions don't introduce warts into a system.
        Vendan 10y ago
        nil doesn't "expand", it's just what it is. You have to bear in mind that methods are roughly syntactic sugar from func (o *Object) ReturnSomething() int to func ReturnSomething(o *Object) int In the first case, it may "seem" that a nil Object would be invalid, but in the second case it's obviously valid, and makes complete sense. It may be described as a "wart", but it makes complete sense to me. The issue is, in my opinion, that people drag their own expectations into programming languages as they learn them.
        pcwalton 10y ago
        In this case, Java and C# have the same feature and don't have this behavior around null. I think it's reasonable for people who have seen identical features in other languages to expect similar behavior when learning new ones.
        Vendan 10y ago
        They don't have the same feature of "I can call a method on a null value", at least, not that I've seen... You are calling them identical features when the fact is that they aren't. C# and Java both have generics, do they perform identical? Haskell has generics too, but I bet they'd throw a shit-fit if you tried to say "Oh, C# has generics too, so why would I bother with Haskell?"
    - cakoose 10y ago
      Sometimes you try and make one feature work a certain way and that decision has unintended consequences. Most languages have surprises like this. In this case, it's an implicit conversion from "nil" to the "nil value" of a given type. This is convenient in many cases, but causes confusion in other cases.
      - catnaroek 10y ago
        > In this case, it's an implicit conversion from "nil" to the "nil value" of a given type. Upvoted because of this, because it clarified my earlier confusion. That being said, I don't really agree with the rest.
      - kjksf 10y ago
        In Go, it's called "zero value", not "nil value". It's also not an implicit conversion from nil to something. Nil isn't a value the way 0 is a value of an integer or "foo" is a value of string. Per Go spec (https://golang.org/ref/spec https://golang.org/ref/spec) nil is "predeclared identifier which has no type". That implies it doesn't have a value and therefore cannot be converted to a value. Nil is zero value of pointer types (which in practice end up being a pointer whose value interpreted as integer is 0) and also a zero value of interface type (which is interface without a type and without a value) and a few other types.
    - zeeboo 10y ago
      An interface being equal to nil just because the value it contains is nil would be even more confusing. For example, imagine if you had two interface values, one holding a nil * T and one holding a nil * S. They would both be equal to nil, but not equal to each other, losing transitivity of equality. Also, sometimes nil is a perfectly valid value of something and can implement interfaces. You can call methods on nil receivers because method calls are just curried function applications over the value. It would be odd if the interface compared equal to nil just because I happened to implement it with a value that's valid with nil.
      - katcipis 10y ago
        I had not thought about transitivity on equality, now it makes even more sense to behave like this. Thanks.
      - twic 10y ago
        Couldn't they also be equal to each other? If equality was defined only in terms of the value and not the vtable (to borrow pcwalton's terms), they would be equal. Are there any problems which would arise from this?
        slrz 10y ago
        Equality is not necessarily defined for values of different types ("with different vtables", if you will). You can't just compare values of different type and expect a meaningful result.
        twic 10y ago
        In this case, though, we're talking about interfaces; an interface is a pointer plus this bit we're calling a vtable. Pointers, even to different types, absolutely can be compared for equality, can't they? More specifically, i am pretty sure pointers can be compared to see if they are both nil, which is all that would be needed here.
  - sdegutis 10y ago
    After using Go for a few years, I've learned (the hard way) that Go does not separate the concepts of implementation details and the official language specification. The official implementation is the language, and they're inseparable. This is why a lot of design decisions about the language make you say "wait, what? why does it need to be that way?" and the community's official answer is "because that's how it works best with how it's implemented".
    - woah 10y ago
      In the end, you are programming a computer.
      - catnaroek 10y ago
        Yes, but the whole point to using abstractions (such as high-level languages) is to separate the context where implementation details are relevant (e.g., when writing a compiler), from the rest of the system, where implementation details don't matter (e.g., when using a compiler).
        brandonbloom 10y ago
        It's a stunningly difficult and surprisingly low-value proposition to create abstractions that don't leak. That's not the goal anyway: The goal is to leverage abstractions to build things. The implementation details frequently do matter. Even if your abstraction is airtight, it has to run on a real machine. When possible, it's preferable to design abstractions that have a clear implementation in terms of the lower level's abstractions. This way, when something goes wrong - and it will - you can understand and fix it.
        pcwalton 10y ago
        The behavior of Go's nil does not make the language any simpler to implement in any meaningful sense. The only thing it affects at the machine level is which cmp instructions are generated.
- aikah 10y ago
  The worse is not, that. I think the worse is that, unlike Java, a race condition in Go has a totally undefined behavior that can corrupt the memory of the entire system your Go app is running on. You can effectively shut down your computer with a race condition in a Go program, and it actually happened to me many times on Windows, with 3rd party Go tools that didn't handle concurrency carefully. Go concurrency is truly unsafe, C style.
  - mappu 10y ago
    This shouldn't happen on any kernel with memory protection. Userspace corruption, sure, but shutting down the computer is unrealistic unless you are running Go code on Windows 98 or classic Mac OS.
SirWart 10y ago
I got bit by this the other day when returning a nil pointer to a struct as an error. It was incredibly frustrating when I finally figured out what was going on, and I'm curious if there's a good reason to distinguish between these 2 types of nil or if it was just an oversight/mistake during the language design process.
- catnaroek 10y ago
  Most likely the reason is keeping the language implementation simple. Java-style `null` is actually quite subtle: http://stackoverflow.com/a/2707333/46571 http://stackoverflow.com/a/2707333/46571
  - pcwalton 10y ago
    Go has the same complexity. From the Go spec: "For an expression x of interface type and a type T, the primary expression x.(T) asserts that x is not nil and that the value stored in x is of type T.'"
- amscanne 10y ago
  It's not two types of nil. When a nil value is converted to an interface, it carries type information. When an interface carries type information, it isn't nil.
karmakaze 10y ago
I had run into the same situation of handling multiple errors. Assuming I wasn't the first to need such a thing I found multierror[1]. Searching now returns more choices some of which use a nil []error and some with structs. [1] github.com/joeshaw/multierror
feklar 10y ago
Nil interface/nil pointer in the interface errors also explained in this 'Go landmines' post https://gist.github.com/lavalamp/4bd23295a9f32706a48f https://gist.github.com/lavalamp/4bd23295a9f32706a48f
- katcipis 10y ago
  I already had a few surprises with the for loops + closures too :-). Thanks for sharing.
zalmoxes 10y ago
Francesc Campoy covers nil across all language types in Go https://www.youtube.com/watch?v=ynoY2xz-F8s https://www.youtube.com/watch?v=ynoY2xz-F8s
- katcipis 10y ago
  I mentioned this presentation on the beginning of the post, it is a great source of information on nil behaviour.
benguild 10y ago
Even Google's App Engine SDK has its own variant of the "multi-error" ... like an array of errors in one.
junke 10y ago
> This is a very interesting situation (and not a very common one for me) because it is where Go’s simple errors as values decisions shines. Doing this kind of thing with exceptions would be pretty clumsy, [...] Just because a language supports exceptions does not mean you always have to use them. You can always use a helper function to catch any error and return it as a value.
- fizzbatter 10y ago
  Wouldn't that basically be replicating a return value-error approach, in a more cumbersome way though? You could argue that Go can use Exceptions in the same manner as X lang too, by using panic and recover... but most people seem to not like that. With Go and Rust, return value error handling is just simply nice. There's no question about if a function might error out, just look at the signature.
  - junke 10y ago
    > Wouldn't that basically be replicating a return value-error approach, in a more cumbersome way though? This would not be "replicating" anything, exceptions already are values in most languages. The only thing to do is wrap your call with a function, like "err = catch-all(expr)". > ou could argue that Go can use Exceptions in the same manner as X lang too, by using panic and recover... Except that it would be cumbersome ;-) because panic/recover works only in combination with defer[0]. > With Go and Rust, return value error handling is just simply nice. Using non-exceptional control flow for errors can be useful depending on the circumstances. Using exceptions is simply nice in many other cases. > There's no question about if a function might error out, just look at the signature. Just look at the documentation. Code defensively. [0] http://stackoverflow.com/q/3413389/124319 http://stackoverflow.com/q/3413389/124319
    - junke 10y ago
      Just to clarify: err = catch-all(expr) If your language does not provide a similar tool, you can't just use a function because "expr" is evaluated before the function is called. I had Lisp macros in mind when writing this. A poor man's approach that is still generic is to wrap the expression in a closure: err = catch-all(() -> expr) Not too cumbersome, but YMMV.
karma_vaccum123 10y ago
I love the idea of a "read only" map (or any other type), but it seems it is just a side-effect landmine in this case. Yuck...almost strikes me as a bug as described in the article
- barsonme 10y ago
  A map is a pointer to the actual map implementation. When declared via var m map[T]T you're basically writing var m *runtime.hmap which is a nil pointer. Go has sane zero values, so it makes sense that reading from a nil map returns the zero value for whatever type its values are. However, there's a trade-off: either Go's maps allow nil assignment (in a similar fashion to slice's `append') _or_ it retains its "reference" (forgive me for using that term) semantics. Go's designers decided that maps should differ from slices in that regard, and I tend to agree. I think it'd be a mistake to have to always return a map _just_ in it's newly allocated.
dmcg 10y ago
Is this confusion just so that we can ask the type of a nil interface value at runtime? A man orders coffee in a restaurant. "Would you like it with or without cream?" "Without" A few minutes later the waiter returns. "I'm sorry sir, we've run out of cream. Would you like it without milk instead?"

86 points by katcipis 10y ago

> Why ? It seems to me that it happens because how interfaces are implemented Why on Earth should a user of a high-level language have to care about language implementation details? > What the actual fuck ??? Exactly.

Vendan 10y ago
Except it's not a language implementation detail, it's part of what the language is. You might as well ask why a user of another language needs to know what a class is, or how inheritance works. Too many people seem to be assuming "Oh, it's like void*, so it must act like that always!", but it's not.
- catnaroek 10y ago
  > Except it's not a language implementation detail, it's part of what the language is. Okay, but what was the rationale for designing the language like that?
  - jerf 10y ago
    The last bit of information you need to understand that is that unlike some other languages you may be used to, "nil" is a perfectly valid value for a pointer to have. You can write methods like this: func (o *Object) ReturnSomething() int { if o == nil { return 0 } return o.SomeOtherValue } It is also therefore legal to return in an interface value a nil pointer to a struct of a particular kind, and therefore "an interface containing a nil pointer of a particular type" and "an interface containing nothing at all" are fundamentally different things that can not be collapsed together. So as Vendan says, it is just part of the language. All languages have this sort of wart in them, where two or more perfectly sensible decisions interact to create something that isn't sensible at first glance. (I'm still in favor of going back in time and changing Go to have non-nillable values, but that ship has certainly sailed.)
    - catnaroek 10y ago
      > unlike some other languages you may be used to, The languages I'm used to don't even have `nil`. > "nil" is a perfectly valid value for a pointer to have. So, just like C and Java. I don't see the difference. > You can write methods like this: (snippet) Oh, now I get it: `nil` itself isn't a value. `nil` is syntactic sugar that expands into a nil value. (In other words, confusingly enough, `nil` and “nil value” are different things!) This is unlike Java, where `null` itself is a value. It would be very helpful if people described things precisely. > All languages have this sort of wart in them, where two or more perfectly sensible decisions interact to create something that isn't sensible at first glance. This means that at least one of the two (or more) decisions was less sensible than it originally seemed. Good design decisions don't introduce warts into a system.
      - Vendan 10y ago
        nil doesn't "expand", it's just what it is. You have to bear in mind that methods are roughly syntactic sugar from func (o *Object) ReturnSomething() int to func ReturnSomething(o *Object) int In the first case, it may "seem" that a nil Object would be invalid, but in the second case it's obviously valid, and makes complete sense. It may be described as a "wart", but it makes complete sense to me. The issue is, in my opinion, that people drag their own expectations into programming languages as they learn them.
        pcwalton 10y ago
        In this case, Java and C# have the same feature and don't have this behavior around null. I think it's reasonable for people who have seen identical features in other languages to expect similar behavior when learning new ones.
        Vendan 10y ago
        They don't have the same feature of "I can call a method on a null value", at least, not that I've seen... You are calling them identical features when the fact is that they aren't. C# and Java both have generics, do they perform identical? Haskell has generics too, but I bet they'd throw a shit-fit if you tried to say "Oh, C# has generics too, so why would I bother with Haskell?"
  - cakoose 10y ago
    Sometimes you try and make one feature work a certain way and that decision has unintended consequences. Most languages have surprises like this. In this case, it's an implicit conversion from "nil" to the "nil value" of a given type. This is convenient in many cases, but causes confusion in other cases.
    - catnaroek 10y ago
      > In this case, it's an implicit conversion from "nil" to the "nil value" of a given type. Upvoted because of this, because it clarified my earlier confusion. That being said, I don't really agree with the rest.
    - kjksf 10y ago
      In Go, it's called "zero value", not "nil value". It's also not an implicit conversion from nil to something. Nil isn't a value the way 0 is a value of an integer or "foo" is a value of string. Per Go spec (https://golang.org/ref/spec https://golang.org/ref/spec) nil is "predeclared identifier which has no type". That implies it doesn't have a value and therefore cannot be converted to a value. Nil is zero value of pointer types (which in practice end up being a pointer whose value interpreted as integer is 0) and also a zero value of interface type (which is interface without a type and without a value) and a few other types.
  - zeeboo 10y ago
    An interface being equal to nil just because the value it contains is nil would be even more confusing. For example, imagine if you had two interface values, one holding a nil * T and one holding a nil * S. They would both be equal to nil, but not equal to each other, losing transitivity of equality. Also, sometimes nil is a perfectly valid value of something and can implement interfaces. You can call methods on nil receivers because method calls are just curried function applications over the value. It would be odd if the interface compared equal to nil just because I happened to implement it with a value that's valid with nil.
    - katcipis 10y ago
      I had not thought about transitivity on equality, now it makes even more sense to behave like this. Thanks.
    - twic 10y ago
      Couldn't they also be equal to each other? If equality was defined only in terms of the value and not the vtable (to borrow pcwalton's terms), they would be equal. Are there any problems which would arise from this?
      - slrz 10y ago
        Equality is not necessarily defined for values of different types ("with different vtables", if you will). You can't just compare values of different type and expect a meaningful result.
        twic 10y ago
        In this case, though, we're talking about interfaces; an interface is a pointer plus this bit we're calling a vtable. Pointers, even to different types, absolutely can be compared for equality, can't they? More specifically, i am pretty sure pointers can be compared to see if they are both nil, which is all that would be needed here.
- sdegutis 10y ago
  After using Go for a few years, I've learned (the hard way) that Go does not separate the concepts of implementation details and the official language specification. The official implementation is the language, and they're inseparable. This is why a lot of design decisions about the language make you say "wait, what? why does it need to be that way?" and the community's official answer is "because that's how it works best with how it's implemented".
  - woah 10y ago
    In the end, you are programming a computer.
    - catnaroek 10y ago
      Yes, but the whole point to using abstractions (such as high-level languages) is to separate the context where implementation details are relevant (e.g., when writing a compiler), from the rest of the system, where implementation details don't matter (e.g., when using a compiler).
      - brandonbloom 10y ago
        It's a stunningly difficult and surprisingly low-value proposition to create abstractions that don't leak. That's not the goal anyway: The goal is to leverage abstractions to build things. The implementation details frequently do matter. Even if your abstraction is airtight, it has to run on a real machine. When possible, it's preferable to design abstractions that have a clear implementation in terms of the lower level's abstractions. This way, when something goes wrong - and it will - you can understand and fix it.
        pcwalton 10y ago
        The behavior of Go's nil does not make the language any simpler to implement in any meaningful sense. The only thing it affects at the machine level is which cmp instructions are generated.
aikah 10y ago
The worse is not, that. I think the worse is that, unlike Java, a race condition in Go has a totally undefined behavior that can corrupt the memory of the entire system your Go app is running on. You can effectively shut down your computer with a race condition in a Go program, and it actually happened to me many times on Windows, with 3rd party Go tools that didn't handle concurrency carefully. Go concurrency is truly unsafe, C style.
- mappu 10y ago
  This shouldn't happen on any kernel with memory protection. Userspace corruption, sure, but shutting down the computer is unrealistic unless you are running Go code on Windows 98 or classic Mac OS.

SirWart 10y ago

I got bit by this the other day when returning a nil pointer to a struct as an error. It was incredibly frustrating when I finally figured out what was going on, and I'm curious if there's a good reason to distinguish between these 2 types of nil or if it was just an oversight/mistake during the language design process.

catnaroek 10y ago
Most likely the reason is keeping the language implementation simple. Java-style `null` is actually quite subtle: http://stackoverflow.com/a/2707333/46571 http://stackoverflow.com/a/2707333/46571
- pcwalton 10y ago
  Go has the same complexity. From the Go spec: "For an expression x of interface type and a type T, the primary expression x.(T) asserts that x is not nil and that the value stored in x is of type T.'"
amscanne 10y ago
It's not two types of nil. When a nil value is converted to an interface, it carries type information. When an interface carries type information, it isn't nil.

karmakaze 10y ago

I had run into the same situation of handling multiple errors. Assuming I wasn't the first to need such a thing I found multierror[1]. Searching now returns more choices some of which use a nil []error and some with structs. [1] github.com/joeshaw/multierror

feklar 10y ago

Nil interface/nil pointer in the interface errors also explained in this 'Go landmines' post https://gist.github.com/lavalamp/4bd23295a9f32706a48f https://gist.github.com/lavalamp/4bd23295a9f32706a48f

katcipis 10y ago
I already had a few surprises with the for loops + closures too :-). Thanks for sharing.

zalmoxes 10y ago

Francesc Campoy covers nil across all language types in Go https://www.youtube.com/watch?v=ynoY2xz-F8s https://www.youtube.com/watch?v=ynoY2xz-F8s

katcipis 10y ago
I mentioned this presentation on the beginning of the post, it is a great source of information on nil behaviour.

benguild 10y ago

Even Google's App Engine SDK has its own variant of the "multi-error" ... like an array of errors in one.

junke 10y ago

> This is a very interesting situation (and not a very common one for me) because it is where Go’s simple errors as values decisions shines. Doing this kind of thing with exceptions would be pretty clumsy, [...] Just because a language supports exceptions does not mean you always have to use them. You can always use a helper function to catch any error and return it as a value.

fizzbatter 10y ago
Wouldn't that basically be replicating a return value-error approach, in a more cumbersome way though? You could argue that Go can use Exceptions in the same manner as X lang too, by using panic and recover... but most people seem to not like that. With Go and Rust, return value error handling is just simply nice. There's no question about if a function might error out, just look at the signature.
- junke 10y ago
  > Wouldn't that basically be replicating a return value-error approach, in a more cumbersome way though? This would not be "replicating" anything, exceptions already are values in most languages. The only thing to do is wrap your call with a function, like "err = catch-all(expr)". > ou could argue that Go can use Exceptions in the same manner as X lang too, by using panic and recover... Except that it would be cumbersome ;-) because panic/recover works only in combination with defer[0]. > With Go and Rust, return value error handling is just simply nice. Using non-exceptional control flow for errors can be useful depending on the circumstances. Using exceptions is simply nice in many other cases. > There's no question about if a function might error out, just look at the signature. Just look at the documentation. Code defensively. [0] http://stackoverflow.com/q/3413389/124319 http://stackoverflow.com/q/3413389/124319
  - junke 10y ago
    Just to clarify: err = catch-all(expr) If your language does not provide a similar tool, you can't just use a function because "expr" is evaluated before the function is called. I had Lisp macros in mind when writing this. A poor man's approach that is still generic is to wrap the expression in a closure: err = catch-all(() -> expr) Not too cumbersome, but YMMV.

karma_vaccum123 10y ago

I love the idea of a "read only" map (or any other type), but it seems it is just a side-effect landmine in this case. Yuck...almost strikes me as a bug as described in the article

barsonme 10y ago
A map is a pointer to the actual map implementation. When declared via var m map[T]T you're basically writing var m *runtime.hmap which is a nil pointer. Go has sane zero values, so it makes sense that reading from a nil map returns the zero value for whatever type its values are. However, there's a trade-off: either Go's maps allow nil assignment (in a similar fashion to slice's `append') _or_ it retains its "reference" (forgive me for using that term) semantics. Go's designers decided that maps should differ from slices in that regard, and I tend to agree. I think it'd be a mistake to have to always return a map _just_ in it's newly allocated.

dmcg 10y ago

Is this confusion just so that we can ask the type of a nil interface value at runtime? A man orders coffee in a restaurant. "Would you like it with or without cream?" "Without" A few minutes later the waiter returns. "I'm sorry sir, we've run out of cream. Would you like it without milk instead?"