6 ms·

> Go does not allow many basic types, such as strings or booleans, to be nil. Instead, when a type is initialized without a value, it defaults to the “zero valu

> Go does not allow many basic types, such as strings or booleans, to be nil. Instead, when a type is initialized without a value, it defaults to the “zero value” for that type. This is frequently useful, but complicates database interactions, where null values are common. This sounds like they weren't defining their types properly. If your value can potentially be null, it should be a pointer to the type, not the type itself. A string can't be null, but a pointer to a string can. (In fact, there's no magic going on here - 'nil' is simply the zero value of a pointer. So you always get the zero value - you just need to choose the type that has the zero value you want... which is, in this case, a pointer, not a value). As explained well in this thread[0], this is the most accurate representation of the data itself. You could create your own type that automatically decodes all null values to whatever the zero value is (empty string, etc)., but then you lose that information. Yes, this forces you to do a check for the null value before using the data for the first time (or to invent your own monad for abstracting this), but at a high level, that's what you have to do in every language. [0] https://groups.google.com/forum/?fromgroups#!topic/golang-nuts/JOFWAqrTbUs https://groups.google.com/forum/?fromgroups#!topic/golang-nu...

lbarrow 14y ago
I think that's fair, and in retrospect we should have started with pointers to strings. If this is the best practice, though, we should change the standard library to reflect that, rather than providing NullStrings.
- chimeracoder 14y ago
  Well, it seems that they've just chosen the second option (no pun intended) that I've mentioned above - that is, to define a monad to abstract this. There may be some benefit to doing this instead of pointers, but as you've found yourself, using a monad requires you to pass that monadic type up the calling chain (OR to unwrap that and deal with both cases safely). I haven't used SQL databases with Go much, so I'm just looking at NullFoo types now for the first time, but every time I've thought I could improve on the design of Go in some way, the folks on #go-nuts have (kindly) shown me that I wasn't really thinking about things the right way. So, there's probably some reason that it's been designed this way, but I'm saying that purely based on prior experience with other Go matters, not with package "database/sql".
  - abraininavat 14y ago
    It's extremely generous... hell it's just wrong.. to call Go's NullString/NullBool/etc monads. "Monad" is not a synonym for "Wrapper type"
    - Tyr42 14y ago
      Well, they do correspond to the "Maybe a" for some fixed type a, and Maybe is a monad, however you are right that it would be clearer to to say Maybe or Nullable or Option type, since that's what is meant, not some general monad.
icey 14y ago
This is the approach I've taken for dealing with database nulls in go. I tried using NullString / NullTime / etc, but doing that required writing a custom JSON marshaller for every class with nullable data, which was a huge pain in the ass.
- chimeracoder 14y ago
  I don't know if you're using SQL or NoSQL databases, but if the latter (and your data is stored in JSON), you may find this package I wrote useful: https://www.github.com/ChimeraCoder/gojson https://www.github.com/ChimeraCoder/gojson Instead of unmarshalling to map[string]interface{} and then doing awkward type assertions and checks, etc., this will generate structs that you can unmarshal into. That way, if the unmarshalling succeeds, you're guaranteed to know the types of all your values instead of checking them each time. As for null values, it automatically uses a pointer in the struct definition if it finds a null value - writing that package was how I first learned about this entire topic, actually, which is why I mention it!
pcwalton 14y ago
Using a pointer adds an extra heap allocation, though. The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that.
- chimeracoder 14y ago
  > Using a pointer adds an extra heap allocation, though. If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time. > The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that. Sure you can - that's literally what the NullBool, etc. types do A pointer to a string IS a 'Nullable<String>' - what they need to do is define a way to unwrap this cleanly, which is easy.
  - pcwalton 14y ago
    > If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time. Sure, I'm not saying that the extra allocation will always matter. In most cases it won't. My point is just that this type system workaround does cost some performance. For example, a heap-allocated struct that contains two nullable ints using the pointer trick has 3 heap allocations, not one. > Sure you can - that's literally what the NullBool, etc. types do But that has to be done for each type. If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for.
    - chimeracoder 14y ago
      > If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for. I don't want to descend down the rabbit hole of 'Go doesn't have generics', but as I said above, in this case, pointers do exactly what is needed.
drbawb 14y ago
To avoid the nil pointer dereference panic, could you use an idiom borrowed from Go's map implementation? A library could take advantage of multiple assignment, a la maps: `i, ok := m["route"]` If the map has no entry for key "route", i will be the zero type [e.g: an empty string], and OK will be a bool indicating that the key wasn't found in the map. So in a SQL library it might look something like `func (s Statement) GetString(columnNum int) string,bool` Called as: `str,ok := s.GetString(i)` At least this way your program won't panic() if you forget to check that string != nil and dereference *string, but you still retain the information that the string is nil.
- chimeracoder 14y ago
  Yes, that's actually more or less what NullString does. It's a struct that contains two fields: String and Valid (a boolean). It also implements a couple of other interfaces, but at its core, it's what you describe. Functionally, they're basically the same thing - it's only a syntactic difference whether you check whether a pointer is nil vs. checking whether a boolean is false. At some point, you either have to do a check or coerce a null value into a reasonable value. (Or, I guess, just risk it!). Functional programming purists may cringe at the analogy, but this is is all more or less the same functionality as Scala's 'Option' monad - it just depends on how you want to express it. The only difference is that the Go compiler doesn't force you to check against nil before dereferencing, but if you want to ensure the safety before converting the database value to a String and forgetting about it, then the struct will do that.

lbarrow 14y ago

I think that's fair, and in retrospect we should have started with pointers to strings. If this is the best practice, though, we should change the standard library to reflect that, rather than providing NullStrings.

chimeracoder 14y ago
Well, it seems that they've just chosen the second option (no pun intended) that I've mentioned above - that is, to define a monad to abstract this. There may be some benefit to doing this instead of pointers, but as you've found yourself, using a monad requires you to pass that monadic type up the calling chain (OR to unwrap that and deal with both cases safely). I haven't used SQL databases with Go much, so I'm just looking at NullFoo types now for the first time, but every time I've thought I could improve on the design of Go in some way, the folks on #go-nuts have (kindly) shown me that I wasn't really thinking about things the right way. So, there's probably some reason that it's been designed this way, but I'm saying that purely based on prior experience with other Go matters, not with package "database/sql".
- abraininavat 14y ago
  It's extremely generous... hell it's just wrong.. to call Go's NullString/NullBool/etc monads. "Monad" is not a synonym for "Wrapper type"
  - Tyr42 14y ago
    Well, they do correspond to the "Maybe a" for some fixed type a, and Maybe is a monad, however you are right that it would be clearer to to say Maybe or Nullable or Option type, since that's what is meant, not some general monad.

icey 14y ago

This is the approach I've taken for dealing with database nulls in go. I tried using NullString / NullTime / etc, but doing that required writing a custom JSON marshaller for every class with nullable data, which was a huge pain in the ass.

chimeracoder 14y ago
I don't know if you're using SQL or NoSQL databases, but if the latter (and your data is stored in JSON), you may find this package I wrote useful: https://www.github.com/ChimeraCoder/gojson https://www.github.com/ChimeraCoder/gojson Instead of unmarshalling to map[string]interface{} and then doing awkward type assertions and checks, etc., this will generate structs that you can unmarshal into. That way, if the unmarshalling succeeds, you're guaranteed to know the types of all your values instead of checking them each time. As for null values, it automatically uses a pointer in the struct definition if it finds a null value - writing that package was how I first learned about this entire topic, actually, which is why I mention it!

pcwalton 14y ago

Using a pointer adds an extra heap allocation, though. The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that.

chimeracoder 14y ago
> Using a pointer adds an extra heap allocation, though. If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time. > The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that. Sure you can - that's literally what the NullBool, etc. types do A pointer to a string IS a 'Nullable<String>' - what they need to do is define a way to unwrap this cleanly, which is easy.
- pcwalton 14y ago
  > If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time. Sure, I'm not saying that the extra allocation will always matter. In most cases it won't. My point is just that this type system workaround does cost some performance. For example, a heap-allocated struct that contains two nullable ints using the pointer trick has 3 heap allocations, not one. > Sure you can - that's literally what the NullBool, etc. types do But that has to be done for each type. If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for.
  - chimeracoder 14y ago
    > If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for. I don't want to descend down the rabbit hole of 'Go doesn't have generics', but as I said above, in this case, pointers do exactly what is needed.

drbawb 14y ago

To avoid the nil pointer dereference panic, could you use an idiom borrowed from Go's map implementation? A library could take advantage of multiple assignment, a la maps: `i, ok := m["route"]` If the map has no entry for key "route", i will be the zero type [e.g: an empty string], and OK will be a bool indicating that the key wasn't found in the map. So in a SQL library it might look something like `func (s Statement) GetString(columnNum int) string,bool` Called as: `str,ok := s.GetString(i)` At least this way your program won't panic() if you forget to check that string != nil and dereference *string, but you still retain the information that the string is nil.

chimeracoder 14y ago
Yes, that's actually more or less what NullString does. It's a struct that contains two fields: String and Valid (a boolean). It also implements a couple of other interfaces, but at its core, it's what you describe. Functionally, they're basically the same thing - it's only a syntactic difference whether you check whether a pointer is nil vs. checking whether a boolean is false. At some point, you either have to do a check or coerce a null value into a reasonable value. (Or, I guess, just risk it!). Functional programming purists may cringe at the analogy, but this is is all more or less the same functionality as Scala's 'Option' monad - it just depends on how you want to express it. The only difference is that the Go compiler doesn't force you to check against nil before dereferencing, but if you want to ensure the safety before converting the database value to a String and forgetting about it, then the struct will do that.