5 ms·

The fact that Go is considered a “modern” language and nil pointer exceptions are downright common blows my mind every day. Is there a static checker config tha

The fact that Go is considered a “modern” language and nil pointer exceptions are downright common blows my mind every day. Is there a static checker config that folks use to reduce this risk? Because coming from a TS background it’s legitimately bizarre to me that at $COMP insufficient nil pointer checking regularly causes major production outages.

phplovesong 2y ago
This applies to all languages with nil. The alternative being options (like ocaml).
- nicoburns 2y ago
  > This applies to all languages with nil. The alternative being options (like ocaml) It actually doesn't. Languages like Kotlin, C# and TypeScript special case null in the type system and allow you to specify whether or not a given type should include null or not.
  - notpushkin 2y ago
    This. The bizarre way nils work in Go is completely over my head.
    - styluss 2y ago
      Nil receivers are such a thing to have
- leosanchez 2y ago
  C# at least came up with Nullable reference types.
- chris_overseas 2y ago
  Not entirely true - e.g. Kotlin has null safety built into its type system, and I find it much nicer than the option/optional approaches used in various other languages. https://kotlinlang.org/docs/null-safety.html https://kotlinlang.org/docs/null-safety.html
  - josephg 2y ago
    Looks similar to the syntax in Swift. I agree - it’s much nicer than all the mucking around you need to do in rust. It’ll never happen, but I wish rust added this kind of sugar on top of Option.
- BlackFly 2y ago
  You're replying to someone who is claiming to come from typescript where nullability must be expressly opted into and once opted into must be branched upon without use of a monad (isomorphism notwithstanding). Kotlin would be another example of that. Even with python type hints the same thing can be achieved or with Java and annotations. The problems in all of these cases is the boundaries between the languages where the nullability or lack thereof is often just assumed and not asserted upon. There are language decisions other than options that can work, but they boil down to the same thing: force a branch before use. If go does not have this sort of secondary type system now, it won't be too long before someone sees the value.
- fithisux 2y ago
  Scala has null for interoperability with JVM but Option is the way to go. Python these days has Optional[str] = str | None. Both are in the alternative. Dart hides optional as ?
  - hnlmorg 2y ago
    Nil pointers aren’t the same thing as optional types in python. You can achieve the same thing as optional types in Go using ‘…any’ and the standard library has made use of that pattern since the very first release of Go. Nil pointers in Go are a completely different problem and there isn’t really a direct comparison in dynamic languages like Python nor Typescript because they use references rather than pointers.
    - benmmurphy 2y ago
      I don’t think there is a difference between go pointers and references in python. For normal pointers to structs at runtime I suspect the implementation is very similar. Maybe it’s different for typescript because the compiler can protect you from null references statically and I assume if you are using python types the python type checker can also protect you at compile time.
      - hnlmorg 2y ago
        A nil pointer isn’t a type. It’s a pointer to a type and that type hasn’t been created. In short, it’s an initialisation problem rather than a type system problem. Where “references” differ is you have to initialise the type to get a reference. In Python and Typescript, you don’t create naked pointers. You create structs then pass that reference in functions. The problem with Go is that you can create those naked pointers and then you need to remember to create your struct and then point that pointer to the struct. And you’re sometimes you’re incentivised to write code this way because Gos garbage collector needs to do more work than if you create pointers adhoc after (like references). But you can use pointers like references if you wanted too. So there isn’t really a direct comparison with languages like Python and Typescript because they don’t have the same primitive to begin with. There are benefits to Gos approach: you get more control over memory usage as well as improved performance. But like any idiom, it comes with disadvantages too. And the reason Go panics is because the alternative is the risk of silent memory corruption, which is what a lot of other languages can suffer from with pointers. Though this isnt to say that there are also languages that do solve this problem in a much better way too. But they’re are also a completely different language to Python and Typescript again too. So the comparisons to Python and Typescript simply don’t work for this type of problem.
- josephg 2y ago
  Yeah. But I don’t really consider any language which has implicit nullability to be “modern”. It’s a bad design. It causes bugs. Plenty of recent languages have fixed this problem - like zig, Swift, kotlin, rust and typescript. Go is behind the curve.
- jahewson 2y ago
  It's not the null that's the issue, it's the type system. As long as you have algebraic data types you can have safe null. Ocaml's `None | Some of int` has the same basic meaning as TypeScript's `number | undefined`
  - hnlmorg 2y ago
    That’s not the same thing as a nil pointer though. Go data types cannot be null but a pointer to a data type can be. And what a nil pointers is, is a valid type of pointer but which points to an invalid point or memory. Typescript doesn’t have the same problem because it doesn’t have pointers. Rust somewhat this problem with its borrow checker but it’s still entirely possible to create a nil pointers in Rust and without using ‘unsafe’, but admittedly it’s not by writing idiomatic Rust. Go really should do more to prevent issues here but the real problem is around the creation of “objects” (since Go isn’t fully OOP) and how you can’t really create an image struct (nor even map) and have it default as empty. I’m sure the logic behind that is for performance, but why not expose the option of an uninitialised struct pointer behind ‘unsafe’ for those who are willing to accept that risk?
- Freedom2 2y ago
  > This applies to all languages with nil. This isn't true at all. C#, TypeScript and Kotlin are examples of languages where you have to explicitly opt into null.
hnlmorg 2y ago
Does it regularly cause major production outages? I’d love to see some data to back that claim because I’ve got Go services with uptimes of literally years. Also it might seem weird to you given Typescript is a completely different idiom and doesn’t even have pointers to begin with. That all said, I’m not going to defend the nil pointer problem in Go. It is something that can be largely mitigated with a few relatively small changes to the language. Though I couldn’t see them being entirely backwards compatible.
- go-nil-why 2y ago
  The issue isn’t the services that stay up forever, it’s the ones subject to a huge amount of churn where maintainers lose context on what is or isn’t possibly nil, and get loose with their checks. Edit, responded to pointer vs reference down thread.
  - hnlmorg 2y ago
    The issue I’m addressing is your exaggerated verbiage which sounds like it’s a reasoned argument but is actually just biased nonsense. I’m not disagreeing that nil pointers are an ugly sore for Go. What I’m arguing against is your claim that Go services are constantly failing in production because of that sore point. The reason I make that argument is because: - sore points are a problem for every programming language. designing a programming language is a constant trade off of concerns. so there will always be stuff that doesnt work well. And I know this as an author of a programming language myself. - however if sore points prevent a language from being usable in a production context (assuming that’s its intended role), then that becomes an insurmountable problem. Nil pointers aren’t an insurmountable problem for Go. They’re ugly and arguably a terrible design choice, but not an insurmountable problem. Which is why Go succeeds despite this sore spot. Typescript has its own WTF moments too. Every language does.
    - go-nil-why 2y ago
      Clearly, we don’t work at the same company. What I claimed applied to my company, exactly as I said. If it doesn’t apply to yours… ok? Congrats? Looking back, maybe you didn’t understand the shell variable syntax?
Mawr 2y ago
Assume all pointers are non-nil and use a generic Option type to represent values that may or may not exist: type Option[Value any] struct { value Value exists bool } func (o Option[Value]) Get() (Value, bool) { return o.value, o.exists } So long as all such values use this type instead of abusing the nullability of pointers, you'll eliminate the problem of programmers forgetting to check for nil. It's not without its issues though, the ecosystem does not use such a type and you need a little bit of discipline to stick with it. But it's not that big of a deal if you're getting regular major outages...
- go-nil-why 2y ago
  Issue is that isn’t generated by protoc. Looking at the go ecosystem you’d think static nil traceability is some sort of Very Hard Problem…
  - Mawr 2y ago
    Yep, the ecosystem doesn't support anything like it and a lot of Go people are downright hostile towards this solution :) You could also try your luck with https://github.com/uber-go/nilaway https://github.com/uber-go/nilaway

phplovesong 2y ago

This applies to all languages with nil. The alternative being options (like ocaml).

nicoburns 2y ago
> This applies to all languages with nil. The alternative being options (like ocaml) It actually doesn't. Languages like Kotlin, C# and TypeScript special case null in the type system and allow you to specify whether or not a given type should include null or not.
- notpushkin 2y ago
  This. The bizarre way nils work in Go is completely over my head.
  - styluss 2y ago
    Nil receivers are such a thing to have
leosanchez 2y ago
C# at least came up with Nullable reference types.
chris_overseas 2y ago
Not entirely true - e.g. Kotlin has null safety built into its type system, and I find it much nicer than the option/optional approaches used in various other languages. https://kotlinlang.org/docs/null-safety.html https://kotlinlang.org/docs/null-safety.html
- josephg 2y ago
  Looks similar to the syntax in Swift. I agree - it’s much nicer than all the mucking around you need to do in rust. It’ll never happen, but I wish rust added this kind of sugar on top of Option.
BlackFly 2y ago
You're replying to someone who is claiming to come from typescript where nullability must be expressly opted into and once opted into must be branched upon without use of a monad (isomorphism notwithstanding). Kotlin would be another example of that. Even with python type hints the same thing can be achieved or with Java and annotations. The problems in all of these cases is the boundaries between the languages where the nullability or lack thereof is often just assumed and not asserted upon. There are language decisions other than options that can work, but they boil down to the same thing: force a branch before use. If go does not have this sort of secondary type system now, it won't be too long before someone sees the value.
fithisux 2y ago
Scala has null for interoperability with JVM but Option is the way to go. Python these days has Optional[str] = str | None. Both are in the alternative. Dart hides optional as ?
- hnlmorg 2y ago
  Nil pointers aren’t the same thing as optional types in python. You can achieve the same thing as optional types in Go using ‘…any’ and the standard library has made use of that pattern since the very first release of Go. Nil pointers in Go are a completely different problem and there isn’t really a direct comparison in dynamic languages like Python nor Typescript because they use references rather than pointers.
  - benmmurphy 2y ago
    I don’t think there is a difference between go pointers and references in python. For normal pointers to structs at runtime I suspect the implementation is very similar. Maybe it’s different for typescript because the compiler can protect you from null references statically and I assume if you are using python types the python type checker can also protect you at compile time.
    - hnlmorg 2y ago
      A nil pointer isn’t a type. It’s a pointer to a type and that type hasn’t been created. In short, it’s an initialisation problem rather than a type system problem. Where “references” differ is you have to initialise the type to get a reference. In Python and Typescript, you don’t create naked pointers. You create structs then pass that reference in functions. The problem with Go is that you can create those naked pointers and then you need to remember to create your struct and then point that pointer to the struct. And you’re sometimes you’re incentivised to write code this way because Gos garbage collector needs to do more work than if you create pointers adhoc after (like references). But you can use pointers like references if you wanted too. So there isn’t really a direct comparison with languages like Python and Typescript because they don’t have the same primitive to begin with. There are benefits to Gos approach: you get more control over memory usage as well as improved performance. But like any idiom, it comes with disadvantages too. And the reason Go panics is because the alternative is the risk of silent memory corruption, which is what a lot of other languages can suffer from with pointers. Though this isnt to say that there are also languages that do solve this problem in a much better way too. But they’re are also a completely different language to Python and Typescript again too. So the comparisons to Python and Typescript simply don’t work for this type of problem.
josephg 2y ago
Yeah. But I don’t really consider any language which has implicit nullability to be “modern”. It’s a bad design. It causes bugs. Plenty of recent languages have fixed this problem - like zig, Swift, kotlin, rust and typescript. Go is behind the curve.
jahewson 2y ago
It's not the null that's the issue, it's the type system. As long as you have algebraic data types you can have safe null. Ocaml's `None | Some of int` has the same basic meaning as TypeScript's `number | undefined`
- hnlmorg 2y ago
  That’s not the same thing as a nil pointer though. Go data types cannot be null but a pointer to a data type can be. And what a nil pointers is, is a valid type of pointer but which points to an invalid point or memory. Typescript doesn’t have the same problem because it doesn’t have pointers. Rust somewhat this problem with its borrow checker but it’s still entirely possible to create a nil pointers in Rust and without using ‘unsafe’, but admittedly it’s not by writing idiomatic Rust. Go really should do more to prevent issues here but the real problem is around the creation of “objects” (since Go isn’t fully OOP) and how you can’t really create an image struct (nor even map) and have it default as empty. I’m sure the logic behind that is for performance, but why not expose the option of an uninitialised struct pointer behind ‘unsafe’ for those who are willing to accept that risk?
Freedom2 2y ago
> This applies to all languages with nil. This isn't true at all. C#, TypeScript and Kotlin are examples of languages where you have to explicitly opt into null.

hnlmorg 2y ago

Does it regularly cause major production outages? I’d love to see some data to back that claim because I’ve got Go services with uptimes of literally years. Also it might seem weird to you given Typescript is a completely different idiom and doesn’t even have pointers to begin with. That all said, I’m not going to defend the nil pointer problem in Go. It is something that can be largely mitigated with a few relatively small changes to the language. Though I couldn’t see them being entirely backwards compatible.

go-nil-why 2y ago
The issue isn’t the services that stay up forever, it’s the ones subject to a huge amount of churn where maintainers lose context on what is or isn’t possibly nil, and get loose with their checks. Edit, responded to pointer vs reference down thread.
- hnlmorg 2y ago
  The issue I’m addressing is your exaggerated verbiage which sounds like it’s a reasoned argument but is actually just biased nonsense. I’m not disagreeing that nil pointers are an ugly sore for Go. What I’m arguing against is your claim that Go services are constantly failing in production because of that sore point. The reason I make that argument is because: - sore points are a problem for every programming language. designing a programming language is a constant trade off of concerns. so there will always be stuff that doesnt work well. And I know this as an author of a programming language myself. - however if sore points prevent a language from being usable in a production context (assuming that’s its intended role), then that becomes an insurmountable problem. Nil pointers aren’t an insurmountable problem for Go. They’re ugly and arguably a terrible design choice, but not an insurmountable problem. Which is why Go succeeds despite this sore spot. Typescript has its own WTF moments too. Every language does.
  - go-nil-why 2y ago
    Clearly, we don’t work at the same company. What I claimed applied to my company, exactly as I said. If it doesn’t apply to yours… ok? Congrats? Looking back, maybe you didn’t understand the shell variable syntax?

Mawr 2y ago

Assume all pointers are non-nil and use a generic Option type to represent values that may or may not exist: type Option[Value any] struct { value Value exists bool } func (o Option[Value]) Get() (Value, bool) { return o.value, o.exists } So long as all such values use this type instead of abusing the nullability of pointers, you'll eliminate the problem of programmers forgetting to check for nil. It's not without its issues though, the ecosystem does not use such a type and you need a little bit of discipline to stick with it. But it's not that big of a deal if you're getting regular major outages...

go-nil-why 2y ago
Issue is that isn’t generated by protoc. Looking at the go ecosystem you’d think static nil traceability is some sort of Very Hard Problem…
- Mawr 2y ago
  Yep, the ecosystem doesn't support anything like it and a lot of Go people are downright hostile towards this solution :) You could also try your luck with https://github.com/uber-go/nilaway https://github.com/uber-go/nilaway