6 ms·

Union types ('enum types') would be complicated in Go

28 points by misonic 2y ago

adastra22 2y ago
> One core requirement for this is what Rust calls an Enum and what is broadly known as a Union type Union types and enum types are not the same thing, and this misunderstanding invalidates the entire article. An enum type includes a marker which indicates which value it contains. The garbage collector would be able to read this tag value and know.
- hmry 2y ago
  I don't know what to tell you, there are just multiple ways to use the same words. What Rust calls enums are called tagged unions in most other languages. Enum usually refers to a simple collection of named values, what Rust calls fieldless enums or unit-only enums.
  - Tainnor 2y ago
    > are called tagged unions in most other languages Not sure about that. Swift also calls them enums, in Java and Kotlin (and maybe Scala? I forget) they're sealed classes/interfaces and in PL theory and many typed FP languages they're called "sum types".
    - pjmlp 2y ago
      Variant records in Pascal, and related linage language. Enumerations are a sequence of values.
  - adastra22 2y ago
    The "tagged" part is important. They are in fact not called "unions" in most languages, only C-derived languages. And in C, unions are untagged. Only later extensions added tagged union types. So I would contend that unqualified "union type" means an untagged union.
- cookiengineer 2y ago
  Wouldn't an enum in go not only be different in what the reflections API sees behind the scenes? In the sense that it first points to the type, and then to the value, similar to how other newer types are already implemented? type whatever enum { value1 iota value2 } enum would just be a similar shorthand to uint or whatever you want to identify it uniquely. I don't see much problem implementing this apart from the methods in the GC that have to be touched to trace the object tree correctly. The comparison handling must be touched in either case, so I don't think an additional pointer to the enum's definition there counts as much work.
- IshKebab 2y ago
  It sounds like you don't have exposure to other languages. C defaults to untagged unions but there are plenty of languages where a union is always tagged. To put it another way, a Rust `enum` is not the same as C's untagged unions but it is still a union.
  - adastra22 2y ago
    The article says the garbage collector can't know which variant the union type is. I don't know how to interpret that other than an (incorrect) assumption of untagged unions.
    - IshKebab 2y ago
      It is written a little confusingly but that's not what they're saying. They were saying you can't implement a union in user code with the current garbage collector because there's no way to tell it which variant your union is: > let's ask why we can't implement such a union type today using Go's unsafe package to perform suitable manipulation of a suitable memory region. They are aware that you could add union types to Go - their point is that it would require garbage collector modifications which may be difficult: > The corollary to all of this is that adding union types to Go as a language feature wouldn't be merely a modest change in the compiler. It would also require a bunch of work in how such types interact with garbage collection, Go's memory allocation systems (which in the normal Go toolchain allocate things with pointers into separate memory arenas than things without them), and likely other places in the runtime.
- aiono 2y ago
  Meaning of union and enum changes based on which programming language terminology you use. Enum of Rust is tagged union of C with compiler support. Enum in C is just basic enum of Rust without any fields. The list goes on like this.
skybrian 2y ago
Yes, the Go garbage collector would need to support some new memory layouts to make certain kinds of unions efficient. A union between a double, a pointer, and small integer types (as done in languages like JavaScript) might be a good start?
- tignaj 2y ago
  Here is a fairly efficient version of a simple union like that I did a while back https://github.com/tigrannajaryan/govariant https://github.com/tigrannajaryan/govariant
pjmlp 2y ago
No it wouldn't, as there are garbage collected systems programming languages with them, already 40 years ago, but as usual in Go, we ignore computing history. Additionally, just having actual enumerations, Pascal / Algo style, not ML style, would already be an improvement over the iota/const hack.
- philosopher1234 2y ago
  The author of this blog is an internet commenter, not a representative of Go. I don’t see how such a post justifies saying that Go ignores computing history.
  - tempodox 2y ago
    This post has nothing to do with Go ignoring computing history. The authors/maintainers of Go can do that just fine on their own.
  - pjmlp 2y ago
    Past history of most folks on Go community, including this post essay about Go's type system and related "difficulty" of implementation.
mjevans 2y ago
I've thought about this while working with Go more frequently over the past few months. Metadata. The same way shapes of functions let Go know which generic 'profile' to use for a given thing are metadata. Just like when a reference to something is taken that type is compile time metadata. Structure. The precise layout of structures and other data fields is also compile time metadata. They don't even need to remain the same between versions of go, or even builds if somehow they're randomized. That isn't how programmer's think (at least any who were also trained in assembly???). When I lay out a struct I do expect undersized fields to get padded, but I expect every field in order, and I'd prefer some way of forcing the issue for precise padding. However, 'union' of types is just syntax sugar. Give the programmer the above basics and add one more: builtin.*reshape()*. reshape() would allow any similarly shaped structures to replace the type of the reshaped item. E.G. reshape({x, y, z uint64},{x, y, z int64}, A, B) would convert a 192bit chunk of 3 ints of one type to the other. It could also convert anything else similarly. That's a trivial example, what about some private structure from a library? I'd think the unsafe package's version should allow violation of the private field space, but the normal safe version might force the unexported fields to 'pad' (inaccessible) space. I don't think this would alter garbage collection, as that process likely has to keep it's own track of regions of memory and places that point within them. That's runtime (maybe compiler time sometimes?) metadata which reshape() would have to work with. Generics even _sort_ of do this already when prefixed with ~type in the list of allowed types; the compiler's allowed to use the passed thing as that type of value and the return it back the same as input... and I really don't see why a reshape function couldn't do the same thing. There is something else though: reshape() likely needs to consume / claim the resource, since it wouldn't initialize anything. So maybe it needs to return the recast value to be assigned or passed somewhere and further invalidate usage of the variable past that use. Alternately it could take a single value variable and modify the type as part of it's call (also providing the value as it's return would be useful sometimes too).
melodyogonna 2y ago
The important detail that has bogged down almost every union type discussion is "zero values". What would be the zero value of a union type? If you've written Go you know the entire language is built around zero values, disabling it for some types is not an option.
- the_gipsy 2y ago
  Couldn't it be just the first enum item? Zero values are somewhat arbitrary anyway.
  - melodyogonna 2y ago
    Why would it be just the first enum item? How do you even determine how the enum is ordered?
    - riwsky 2y ago
      Source order, presumably?
      - eru 2y ago
        That's what eg Haskell does, eg when you derive comparisons for sum tyes, they compare by source order.
        pjmlp 2y ago
        As does Pascal, Modula-2, Ada, among plenty of others.
  - aiono 2y ago
    I don't think this is a good idea. Because zero value changes when you reorder the fields.
- IshKebab 2y ago
  You could use the first entry, or require an explicit annotation. That doesn't seem like a big issue.
lordofgibbons 2y ago
This is sorely needed to simplify error handling and getting rid of nil pointers panics. Would love to see a linter written for Go after something like this is created to ensure absolutely no naked pointer is ever returned.
- Ferret7446 2y ago
  I have never encountered a nil pointer panic in Go in ~5 years, after the first few months learning to actually use the language. It's basically a non-issue IMO. Just stop explicitly ignoring the errors returned from constructors.
  - lordofgibbons 2y ago
    I've been using the Go full time since about 2013 and it's a massive issue, and has woken me and my teammates up at night. Specially, when junior engineers are involved. Why hope to catch these issues with code review if the type system/compiler can do it for you instead? Some of the most common areas that infest the code with nullable pointer types are when you have to deal with de-serializing data a lot. This is due to lack of a common built-in Optional type (I know you can define one easily, but you can't force libraries that you rely on to use that type). The best we have now is https://github.com/uber-go/nilaway https://github.com/uber-go/nilaway and it's improving with time. It does static analysis, but it has a very difficult job to do, so right now it's super slow to run, and is prone to having false-positives.
brundolf 2y ago
Forget Result, just allow the type system to express non-nullable object references. Use the same layout, just let the compiler know when something is guaranteed to exist and force null-checking when it isn't This doesn't cover everything people might want to do with unions, but it covers the billion-dollar mistake and doesn't run against the grain of the entire language (as far as I know)
- mrgriffin 2y ago
  > doesn't run against the grain of the entire language Not an expert, but my gut says maybe it runs against zero values? As in, "what's the zero value for a non-nullable reference?" Maybe the answer is something like "you can only use this type for parameters", but that seems very limiting.
  - rudiksz 2y ago
    Half of the language is already non-nullable and is accomplished by allowing for zero values. Non pointer variables are guaranteed to be never nil. What is missing is the ability to have pointer variables and have the compiler ensure that it will be never nil. I believe this was a design choice, not some technical limitation.
  - keybored 2y ago
    Like the sibling comment seems to be saying: a non-nil pointer would have to be set to some real (non-nil) pointer value anyway. So having a zero value does not seem to apply?
- hellcow 2y ago
  I've been using `Null[T any] struct { V T, Valid bool }` for this, as the pattern comes from database/sql. Works fine.
eru 2y ago
> At one level we easily do something that looks like a Result type in Go, especially now that we have generics. You make a generic struct that has private fields for an error, a value of type T, and a flag that says which is valid, and then give it some methods to set and get values and ask it which it currently contains. If you ask for a sort of value that's not valid, it panics. However, this struct necessarily has space for three fields, where the Rust enums (and generally union types) act more like C unions, only needing space for the largest type possible in them and sometimes a marker of what type is in the union right now. That seems better than not having algebraic data types at all.
aiono 2y ago
I don't know what's the point of the post is. Safe unions require compiler work? Sure who objects to that?
foldr 2y ago
I think this is mistaken. Go already has a way to represent 'open' union types (interfaces), so all of these runtime problems have already been solved. What's missing is just the type system support to do exhaustive matching on the members of the union. With the addition of generics, 'all' that would be necessary is to make the following a legal variable definition: var foo interface { struct { A int } | struct { B string } } It currently fails with the following error: "cannot use type interface{struct{A int} | struct{B string}} outside a type constraint: interface contains type constraints"
- jerf 2y ago
  The problem with that is that in Go, I need to be able to put methods on those things, for reasons possibly unrelated to the interface in question. For that they need to be named. For that you might as well do what has worked since Go 1.0 and just put an unexported method in the interface and declare several instances of that interface in your package. Honestly interfaces with unexported methods are 90%+ of what people want. It's just not spelled the way they expect. And if you're not going to be happy except at absolutely 100%, a position I can and do respect, there's no point waiting for Go to get any better because I can guarantee you no Go proposal for sum types will fix that you will be forced to have a "nil" value in the sum type, so there's no point in waiting.
  - wbl 2y ago
    The zero would obviously be the zero value of one of the constituents. Coproducts in most of Go are easy: it's interfaces that make it hard.
    - masklinn 2y ago
      > The zero would obviously be the zero value of one of the constituents. If interfaces are used for union types then the obvious zero is nil, not any constituent. Nil is the zero value of interfaces. It’s perfectly consistent and in line with the rest of the langage.

28 points by misonic 2y ago

> One core requirement for this is what Rust calls an Enum and what is broadly known as a Union type Union types and enum types are not the same thing, and this misunderstanding invalidates the entire article. An enum type includes a marker which indicates which value it contains. The garbage collector would be able to read this tag value and know.

hmry 2y ago
I don't know what to tell you, there are just multiple ways to use the same words. What Rust calls enums are called tagged unions in most other languages. Enum usually refers to a simple collection of named values, what Rust calls fieldless enums or unit-only enums.
- Tainnor 2y ago
  > are called tagged unions in most other languages Not sure about that. Swift also calls them enums, in Java and Kotlin (and maybe Scala? I forget) they're sealed classes/interfaces and in PL theory and many typed FP languages they're called "sum types".
  - pjmlp 2y ago
    Variant records in Pascal, and related linage language. Enumerations are a sequence of values.
- adastra22 2y ago
  The "tagged" part is important. They are in fact not called "unions" in most languages, only C-derived languages. And in C, unions are untagged. Only later extensions added tagged union types. So I would contend that unqualified "union type" means an untagged union.
cookiengineer 2y ago
Wouldn't an enum in go not only be different in what the reflections API sees behind the scenes? In the sense that it first points to the type, and then to the value, similar to how other newer types are already implemented? type whatever enum { value1 iota value2 } enum would just be a similar shorthand to uint or whatever you want to identify it uniquely. I don't see much problem implementing this apart from the methods in the GC that have to be touched to trace the object tree correctly. The comparison handling must be touched in either case, so I don't think an additional pointer to the enum's definition there counts as much work.
IshKebab 2y ago
It sounds like you don't have exposure to other languages. C defaults to untagged unions but there are plenty of languages where a union is always tagged. To put it another way, a Rust `enum` is not the same as C's untagged unions but it is still a union.
- adastra22 2y ago
  The article says the garbage collector can't know which variant the union type is. I don't know how to interpret that other than an (incorrect) assumption of untagged unions.
  - IshKebab 2y ago
    It is written a little confusingly but that's not what they're saying. They were saying you can't implement a union in user code with the current garbage collector because there's no way to tell it which variant your union is: > let's ask why we can't implement such a union type today using Go's unsafe package to perform suitable manipulation of a suitable memory region. They are aware that you could add union types to Go - their point is that it would require garbage collector modifications which may be difficult: > The corollary to all of this is that adding union types to Go as a language feature wouldn't be merely a modest change in the compiler. It would also require a bunch of work in how such types interact with garbage collection, Go's memory allocation systems (which in the normal Go toolchain allocate things with pointers into separate memory arenas than things without them), and likely other places in the runtime.
aiono 2y ago
Meaning of union and enum changes based on which programming language terminology you use. Enum of Rust is tagged union of C with compiler support. Enum in C is just basic enum of Rust without any fields. The list goes on like this.

skybrian 2y ago

Yes, the Go garbage collector would need to support some new memory layouts to make certain kinds of unions efficient. A union between a double, a pointer, and small integer types (as done in languages like JavaScript) might be a good start?

tignaj 2y ago
Here is a fairly efficient version of a simple union like that I did a while back https://github.com/tigrannajaryan/govariant https://github.com/tigrannajaryan/govariant

pjmlp 2y ago

No it wouldn't, as there are garbage collected systems programming languages with them, already 40 years ago, but as usual in Go, we ignore computing history. Additionally, just having actual enumerations, Pascal / Algo style, not ML style, would already be an improvement over the iota/const hack.

philosopher1234 2y ago
The author of this blog is an internet commenter, not a representative of Go. I don’t see how such a post justifies saying that Go ignores computing history.
- tempodox 2y ago
  This post has nothing to do with Go ignoring computing history. The authors/maintainers of Go can do that just fine on their own.
- pjmlp 2y ago
  Past history of most folks on Go community, including this post essay about Go's type system and related "difficulty" of implementation.

mjevans 2y ago

I've thought about this while working with Go more frequently over the past few months. Metadata. The same way shapes of functions let Go know which generic 'profile' to use for a given thing are metadata. Just like when a reference to something is taken that type is compile time metadata. Structure. The precise layout of structures and other data fields is also compile time metadata. They don't even need to remain the same between versions of go, or even builds if somehow they're randomized. That isn't how programmer's think (at least any who were also trained in assembly???). When I lay out a struct I do expect undersized fields to get padded, but I expect every field in order, and I'd prefer some way of forcing the issue for precise padding. However, 'union' of types is just syntax sugar. Give the programmer the above basics and add one more: builtin.*reshape()*. reshape() would allow any similarly shaped structures to replace the type of the reshaped item. E.G. reshape({x, y, z uint64},{x, y, z int64}, A, B) would convert a 192bit chunk of 3 ints of one type to the other. It could also convert anything else similarly. That's a trivial example, what about some private structure from a library? I'd think the unsafe package's version should allow violation of the private field space, but the normal safe version might force the unexported fields to 'pad' (inaccessible) space. I don't think this would alter garbage collection, as that process likely has to keep it's own track of regions of memory and places that point within them. That's runtime (maybe compiler time sometimes?) metadata which reshape() would have to work with. Generics even _sort_ of do this already when prefixed with ~type in the list of allowed types; the compiler's allowed to use the passed thing as that type of value and the return it back the same as input... and I really don't see why a reshape function couldn't do the same thing. There is something else though: reshape() likely needs to consume / claim the resource, since it wouldn't initialize anything. So maybe it needs to return the recast value to be assigned or passed somewhere and further invalidate usage of the variable past that use. Alternately it could take a single value variable and modify the type as part of it's call (also providing the value as it's return would be useful sometimes too).

melodyogonna 2y ago

The important detail that has bogged down almost every union type discussion is "zero values". What would be the zero value of a union type? If you've written Go you know the entire language is built around zero values, disabling it for some types is not an option.

the_gipsy 2y ago
Couldn't it be just the first enum item? Zero values are somewhat arbitrary anyway.
- melodyogonna 2y ago
  Why would it be just the first enum item? How do you even determine how the enum is ordered?
  - riwsky 2y ago
    Source order, presumably?
    - eru 2y ago
      That's what eg Haskell does, eg when you derive comparisons for sum tyes, they compare by source order.
      - pjmlp 2y ago
        As does Pascal, Modula-2, Ada, among plenty of others.
- aiono 2y ago
  I don't think this is a good idea. Because zero value changes when you reorder the fields.
IshKebab 2y ago
You could use the first entry, or require an explicit annotation. That doesn't seem like a big issue.

lordofgibbons 2y ago

This is sorely needed to simplify error handling and getting rid of nil pointers panics. Would love to see a linter written for Go after something like this is created to ensure absolutely no naked pointer is ever returned.

Ferret7446 2y ago
I have never encountered a nil pointer panic in Go in ~5 years, after the first few months learning to actually use the language. It's basically a non-issue IMO. Just stop explicitly ignoring the errors returned from constructors.
- lordofgibbons 2y ago
  I've been using the Go full time since about 2013 and it's a massive issue, and has woken me and my teammates up at night. Specially, when junior engineers are involved. Why hope to catch these issues with code review if the type system/compiler can do it for you instead? Some of the most common areas that infest the code with nullable pointer types are when you have to deal with de-serializing data a lot. This is due to lack of a common built-in Optional type (I know you can define one easily, but you can't force libraries that you rely on to use that type). The best we have now is https://github.com/uber-go/nilaway https://github.com/uber-go/nilaway and it's improving with time. It does static analysis, but it has a very difficult job to do, so right now it's super slow to run, and is prone to having false-positives.

brundolf 2y ago

Forget Result, just allow the type system to express non-nullable object references. Use the same layout, just let the compiler know when something is guaranteed to exist and force null-checking when it isn't This doesn't cover everything people might want to do with unions, but it covers the billion-dollar mistake and doesn't run against the grain of the entire language (as far as I know)

mrgriffin 2y ago
> doesn't run against the grain of the entire language Not an expert, but my gut says maybe it runs against zero values? As in, "what's the zero value for a non-nullable reference?" Maybe the answer is something like "you can only use this type for parameters", but that seems very limiting.
- rudiksz 2y ago
  Half of the language is already non-nullable and is accomplished by allowing for zero values. Non pointer variables are guaranteed to be never nil. What is missing is the ability to have pointer variables and have the compiler ensure that it will be never nil. I believe this was a design choice, not some technical limitation.
- keybored 2y ago
  Like the sibling comment seems to be saying: a non-nil pointer would have to be set to some real (non-nil) pointer value anyway. So having a zero value does not seem to apply?
hellcow 2y ago
I've been using `Null[T any] struct { V T, Valid bool }` for this, as the pattern comes from database/sql. Works fine.

eru 2y ago

> At one level we easily do something that looks like a Result type in Go, especially now that we have generics. You make a generic struct that has private fields for an error, a value of type T, and a flag that says which is valid, and then give it some methods to set and get values and ask it which it currently contains. If you ask for a sort of value that's not valid, it panics. However, this struct necessarily has space for three fields, where the Rust enums (and generally union types) act more like C unions, only needing space for the largest type possible in them and sometimes a marker of what type is in the union right now. That seems better than not having algebraic data types at all.

aiono 2y ago

I don't know what's the point of the post is. Safe unions require compiler work? Sure who objects to that?

foldr 2y ago

I think this is mistaken. Go already has a way to represent 'open' union types (interfaces), so all of these runtime problems have already been solved. What's missing is just the type system support to do exhaustive matching on the members of the union. With the addition of generics, 'all' that would be necessary is to make the following a legal variable definition: var foo interface { struct { A int } | struct { B string } } It currently fails with the following error: "cannot use type interface{struct{A int} | struct{B string}} outside a type constraint: interface contains type constraints"

jerf 2y ago
The problem with that is that in Go, I need to be able to put methods on those things, for reasons possibly unrelated to the interface in question. For that they need to be named. For that you might as well do what has worked since Go 1.0 and just put an unexported method in the interface and declare several instances of that interface in your package. Honestly interfaces with unexported methods are 90%+ of what people want. It's just not spelled the way they expect. And if you're not going to be happy except at absolutely 100%, a position I can and do respect, there's no point waiting for Go to get any better because I can guarantee you no Go proposal for sum types will fix that you will be forced to have a "nil" value in the sum type, so there's no point in waiting.
- wbl 2y ago
  The zero would obviously be the zero value of one of the constituents. Coproducts in most of Go are easy: it's interfaces that make it hard.
  - masklinn 2y ago
    > The zero would obviously be the zero value of one of the constituents. If interfaces are used for union types then the obvious zero is nil, not any constituent. Nil is the zero value of interfaces. It’s perfectly consistent and in line with the rest of the langage.