5 ms·

Lack of sum types is probably one of the worst things about working in Go, and I think it is a much bigger problem than the lack of generics ever was. Sadly, th

by jchw 1y ago

Lack of sum types is probably one of the worst things about working in Go, and I think it is a much bigger problem than the lack of generics ever was. Sadly, though, I don't think you can really just bolt sum types onto an already complete programming language design.

andrewflnr 1y ago
I wonder if a version of OCaml that had a better concurrency story 10-15 years ago could have taken Go's place.
- jchw 1y ago
  Maybe, but I think focusing on concurrency is the wrong idea. I think Go + sum types could be good. Maybe. But, honestly, it's hard to say for sure. First-order effects are great: we have sum types and can use them to model problems. Second-order effects get muddy: We have sum types and they are the ideal solution for a bunch of problems, but without other features they're awkward to use. For example... now you can do a Result type... but if you want to return multiple values in a Result, you need a set/tuple type too. If you do that, is Go's multiple return values concept still a good idea? I could probably go on but hopefully this illustrates the point. I think a lot of people don't acknowledge why Go is truly so successful. Well OK, first elephant in the room, it's obviously successful because it's backed by Google, a company who can and did throw immense resources at making the implementation and standard library pretty damn good as well as helping to push it, but that alone wouldn't have propelled it to where it is today (I mean, compare it to Dart.) But beyond that, Go is successful because Go code is very simple to write and leaves the programmer with relatively few things to be concerned about, and yet in spite of that, Go code generally runs reasonably fast, uses relatively small amounts of memory and is generally not very crash-prone. (Some people would happily debate that, but I trust software like restic, esbuild, rclone and Syncthing every day without fail, among other smaller Go programs. I'm OK with making that assertion.) If you put in the effort to make really good Rust code, the effort is not wasted, but it is a lot of effort when the stupid Go code often does the trick. Consider Discord's presence service: they switched from Go to Rust and massively cut costs and improved latency. Rust wins, Rust better? But... most services will never scale to the point where marginal improvements in latency, throughput or RAM are going to be worth a lot of man-hours worth of programming and ongoing maintenance. Usually throwing a couple more VMs at the problem is just going to be the path of lesser resistance. This was always the case when comparing lower-level to higher-level, but Go amplifies it because the performance gap isn't as big, but the complexity gap remains very large or maybe gets larger. Is writing Rust code really that hard? No, not at all, it's more that writing good Go code is so relatively easy, the language is simple and the standard library is loaded with useful tools. Go's CLI flag parser is pretty unimpressive, but so many projects just need a very basic flag parser and it works totally fine for that, you just don't need to overthink it 99.99% of the time. Same for net/http, the built-in image and compression codecs, TLS stack, and more. Meanwhile, designing and maintaining good high-quality Rust crates is just relatively hard. You've got to worry about async vs sync, various cfg options like nostd and noalloc, and dealing with a lot of wrinkles in the language. Want to use a dyn trait? You'll need to make sure the trait doesn't have any functions with generic parameters or async; makes perfect sense, but adds tension; you want to avoid unnecessary runtime costs in the ideal case but still have the flexibility to use a dyn trait in other cases. Not to mention the borrow checker and how it interacts with a lot of these design choices. Go code is much dumber, but often times it's sufficient. And that's the crux of it. Go is simple in a stupid way, rather than an elegant way, but it really makes the most of it. If you want to supplant Go at what it does best, trying to make a better programming language overall may be a bit misguided. It's probably not that hard to come up with a programming language design that is "better" than Go. What's hard, IMO, is coming up with a programming language where the cognitive load increase relative to Go is met with a pay-off that people using Go would consider genuinely worth it. Something like guaranteed data race safety is definitely good enough if it's something someone needs or at least strongly wants for a given use case. Sum types, on the other hand, are very nice thing to have that make modelling data easier and less error-prone, but not having them isn't exactly the end of the world... In Go, people sometimes emulate them with interfaces and type-switches, and it's not fantastic, but it's usually sufficient. Ocaml probably could/should be more successful, but I'm not sure it competes with Go, I think they're in entirely different wheelhouses. Ocaml feels like it doesn't want to compete with Go, but then again, I only have relatively surface-level knowledge of Ocaml, so I can't really say for sure.
  - alain_gilbert 1y ago
    Check out this toy project I made. It's basically a fork of the Go lexer/parser that adds Result/Option/Tuple/Set... propagation operators (and more) and it compiles down to Go code. https://github.com/alaingilbert/agl https://github.com/alaingilbert/agl
  - noelwelsh 1y ago
    I feel OCaml is as easy to write as Go, though perhaps that is not the case for someone with a typical imperative programmer background. I feel a good part of Go's success is that the core language is very similar to Python, which is obviously very widely known. Most of the issues you discuss with Rust are not issues in OCaml, as OCaml has GC. The language is simpler, in that programs are concerned with fewer concepts (e.g. no lifetimes), but less expressive in that they cannot formally talk about these concepts (though see the Jane Street work to add in some Rust-like concerns: https://blog.janestreet.com/oxidizing-ocaml-locality/ https://blog.janestreet.com/oxidizing-ocaml-locality/)
  - delifue 1y ago
    Go with sum type: Borgo https://github.com/borgo-lang/borgo https://github.com/borgo-lang/borgo
- noelwelsh 1y ago
  I believe it could have, and OCaml didn't because of some bad predictions by the core team (when asked about better concurrency, the response was along the lines that we'll all have tens of CPUs soon, so better in process support wasn't necessary), and the restrictions of academia (there is little value given to engineering work).
- dvdkon 1y ago
  Sadly I don't think so, just because it looks "too functional". If comments on the web are anything to go by, being familiar to C and JS programmers is one of the main reasons for Go's success. I think it has plenty of its own specifics, it's not like e.g. C# which really did start as a straightforward Java clone, but OCaml has even more differences.
  - jchw 1y ago
    I think it's worth considering the possibility that imperative style is just more approachable than functional style. On one hand, some tasks that are basically trivial few-line solutions in a good functional programming language balloon to hundreds of lines of arguably more error prone code in especially dumb imperative languages like Go. On the other hand, it's really really easy to understand what the vast majority of Go code is doing by just reading it line-by-line. Don't get me wrong: it's easy to write broken and unreadable code in pretty much any programming language (I can write bad code in imperative or functional style, I'm a multi-paradigm bad programmer) but at the very least for most imperative code you can really just follow everything line-by-line. It's harder to pull apart complex compositions of map/reduce/filter/fold/etc. (especially if you involve other complex topics like async.) This does seem to be quite a conundrum. I don't think there's any argument regarding the merits of functional programming as a paradigm, but imperative just seems easier to wrap one's head around, and frankly most code is really boring and doesn't need to be able to do the super powerful things that functional programming can do with relative ease. I feel like I'm playing with fire by saying this here, but I think it's true.
    - andrewflnr 1y ago
      That's the thing about OCaml, though, it's relatively friendly with imperative programming too. At least, it's not picky about side effects the way Haskell is. It might still be a pain with loops, though. It's been a while since I used it.
    - suprtx 1y ago
      I think you are correct, perhaps even being too careful in your claims. I think functional and imperative programming, in practice, tend to have different types of constraints, and there are different things that are easy to reason about in them, and encourage different reasoning methods and approaches, and ways of proving properties about them. I think that for learning functional programming, being all-in on purely functional programming and never having side-effects or mutation, can be useful. But for practical programming, functional programming is not in itself a virtue just by itself. One should always consider the trade-offs of the specific case one is facing if one is not learning, as such. If a project has decided on a trade-off where purely functional programming is chosen, then maybe continuing with pure FP is fine; but even in such a case, a function that from outside is purely functional, but uses imperative programming internally, for instance for the sake of optimization, can make sense. There are different advantages and disadvantages to having: pure FP; imperative programming; or different kinds of mixes of FP and imperative programming. Overall, my approximate opinion is that it does not make sense to be dogmatic about functional programming, except when learning FP. https://stackoverflow.com/questions/7717691/why-is-the-minimalist-example-haskell-quicksort-not-a-true-quicksort https://stackoverflow.com/questions/7717691/why-is-the-minim... is arguably a relevant example, though a proper in-place implementation in Haskell might be more fair for comparison. Though I am not certain whether Haskell's lazy evaluation would help or hinder analysis, Ocaml or SML might be better languages for implementing and analysing an in-place implementation variant. Also, Hacker News is a censorship hole.
- pjmlp 1y ago
  Go only enjoys widespread adoption thanks to Docker and Kubernetes success in the industry, had it not been the case, we would be talking about it just like we talk about Limbo and Oberon-2, its influences. The kind of folks that dropped Python and Java for Go, would not have picked up OCaml even if it had a better concurrency story 10-15 years ago.
  - andrewflnr 1y ago
    I guess part of the hypothetical is, "a version of OCaml supported and pushed by Google". But the same idea probably applies, Rob Pike et al (he was involved right?) probably never go for a functional language, even a relatively pragmatic one.
    - pjmlp 1y ago
      That would be Kotlin, Google's C#, given how history goes. I has enough ML influence on it, as a simplified Scala, and also can compile to native in various ways. On the other hand you will notice that the only thing Android team only cared about Go, was the Soong build syste, and parts of the GPU debugger.
eru 1y ago
> Lack of sum types is probably one of the worst things about working in Go, and I think it is a much bigger problem than the lack of generics ever was. Maybe, but once you have eg an Option or Either (a.k.a. Result) type, you typically really want to have some functions that work generically on all versions of them. (Though you could probably get away with using Go's version of void*, the empty interface, in a lot of cases?)
- jchw 1y ago
  Basically, anything that makes it possible to do an Etiher/Result type blows the whole damn thing up. It calls everything into question, e.g. whether multiple returns really makes sense. It's kind of a shame, because I would really like to be able to model things better (and ideally more efficiently too) when working in Go.
  - eru 1y ago
    The 'multiple return values' thing is really stupid in any case, even without sum-types: it's like "we have tuples, but only for this one special case." Just add first-class tuples to the language.
    - dontlaugh 1y ago
      It’s an actual liability, since you can’t uniformly compose the results of functions with different numbers of return values.
      - aatd86 1y ago
        I think it could be retroffited but would require a syntax to have a variable marked as being a tuple. If we look behind the curtains, go functions somehow return tuples that are automatically unrolled into respective variables, sometimes. That's why if you have f(a, b, c){} and g() (b,c){} you can't do f(a, g()) currently. It does not unroll it automatically for some reason here. The result remains in "tuple" form ? I think it may leave some space to have tuples if needed. In any case, I don't see it as technically infeasible.
pjmlp 1y ago
Personally, by now I would already be happy with Pascal like enumerations, instead of the stupid iota/const hack, yes it is an hack. type StatusCodes = (Success, Ongoing, Done) or if you prefer, while bikeshedding what keyword to use, type StatusCodes enumeration (Success, Ongoing, Done) Naturally, it is so much better to write type StatusCodes int const ( Success StatusCodes = iota Ongoing Done )
jerf 1y ago
The major problem with "bolting on" sum types to Go is that it is way too redundant with interfaces and the interactions get weird, because this already works: type Color interface { isColor() } type Red struct {} func (r Red) isColor() {} type Green struct {} func (g Green) isColor() {} type RGB struct { Red byte Green byte Blue byte } func (r RGB) isColor() {} func PrintColor(color Color, text string) { switch c := color.(type) { case Red: // print in red here case Green: // print in green here case RGB: // handle the RGB } } This doesn't prevent the zero value of a Color being 'nil' but it does make "a value of type Color" effectively a sum type because you can't do anything useful to it without unpacking it in a type-aware way. There is no way in Go to have a Red but act like you have an RGB, values always have their types. (There is a common misconception among Go programmers that this doesn't seal the type because you can implement a "isColor" method on your own types in other packages, but it won't work. Try it if you want to be sure.) One could argue they've effectively already been bolted on, all that is missing is some syntax gloss like sum Color { Red{} Green{} RGB{ Red byte Green byte Blue byte } } that would simply unsugar to the above.
- jchw 1y ago
  Yes, this is all true. However, this approach has more downsides than just nil and cumbersome syntax imo. It takes more memory and pointer-chasing than "proper" sum types would require, and while you can "seal" the type, there's no exhaustiveness checking.

by jchw 1y ago

andrewflnr 1y ago

I wonder if a version of OCaml that had a better concurrency story 10-15 years ago could have taken Go's place.

jchw 1y ago
Maybe, but I think focusing on concurrency is the wrong idea. I think Go + sum types could be good. Maybe. But, honestly, it's hard to say for sure. First-order effects are great: we have sum types and can use them to model problems. Second-order effects get muddy: We have sum types and they are the ideal solution for a bunch of problems, but without other features they're awkward to use. For example... now you can do a Result type... but if you want to return multiple values in a Result, you need a set/tuple type too. If you do that, is Go's multiple return values concept still a good idea? I could probably go on but hopefully this illustrates the point. I think a lot of people don't acknowledge why Go is truly so successful. Well OK, first elephant in the room, it's obviously successful because it's backed by Google, a company who can and did throw immense resources at making the implementation and standard library pretty damn good as well as helping to push it, but that alone wouldn't have propelled it to where it is today (I mean, compare it to Dart.) But beyond that, Go is successful because Go code is very simple to write and leaves the programmer with relatively few things to be concerned about, and yet in spite of that, Go code generally runs reasonably fast, uses relatively small amounts of memory and is generally not very crash-prone. (Some people would happily debate that, but I trust software like restic, esbuild, rclone and Syncthing every day without fail, among other smaller Go programs. I'm OK with making that assertion.) If you put in the effort to make really good Rust code, the effort is not wasted, but it is a lot of effort when the stupid Go code often does the trick. Consider Discord's presence service: they switched from Go to Rust and massively cut costs and improved latency. Rust wins, Rust better? But... most services will never scale to the point where marginal improvements in latency, throughput or RAM are going to be worth a lot of man-hours worth of programming and ongoing maintenance. Usually throwing a couple more VMs at the problem is just going to be the path of lesser resistance. This was always the case when comparing lower-level to higher-level, but Go amplifies it because the performance gap isn't as big, but the complexity gap remains very large or maybe gets larger. Is writing Rust code really that hard? No, not at all, it's more that writing good Go code is so relatively easy, the language is simple and the standard library is loaded with useful tools. Go's CLI flag parser is pretty unimpressive, but so many projects just need a very basic flag parser and it works totally fine for that, you just don't need to overthink it 99.99% of the time. Same for net/http, the built-in image and compression codecs, TLS stack, and more. Meanwhile, designing and maintaining good high-quality Rust crates is just relatively hard. You've got to worry about async vs sync, various cfg options like nostd and noalloc, and dealing with a lot of wrinkles in the language. Want to use a dyn trait? You'll need to make sure the trait doesn't have any functions with generic parameters or async; makes perfect sense, but adds tension; you want to avoid unnecessary runtime costs in the ideal case but still have the flexibility to use a dyn trait in other cases. Not to mention the borrow checker and how it interacts with a lot of these design choices. Go code is much dumber, but often times it's sufficient. And that's the crux of it. Go is simple in a stupid way, rather than an elegant way, but it really makes the most of it. If you want to supplant Go at what it does best, trying to make a better programming language overall may be a bit misguided. It's probably not that hard to come up with a programming language design that is "better" than Go. What's hard, IMO, is coming up with a programming language where the cognitive load increase relative to Go is met with a pay-off that people using Go would consider genuinely worth it. Something like guaranteed data race safety is definitely good enough if it's something someone needs or at least strongly wants for a given use case. Sum types, on the other hand, are very nice thing to have that make modelling data easier and less error-prone, but not having them isn't exactly the end of the world... In Go, people sometimes emulate them with interfaces and type-switches, and it's not fantastic, but it's usually sufficient. Ocaml probably could/should be more successful, but I'm not sure it competes with Go, I think they're in entirely different wheelhouses. Ocaml feels like it doesn't want to compete with Go, but then again, I only have relatively surface-level knowledge of Ocaml, so I can't really say for sure.
- alain_gilbert 1y ago
  Check out this toy project I made. It's basically a fork of the Go lexer/parser that adds Result/Option/Tuple/Set... propagation operators (and more) and it compiles down to Go code. https://github.com/alaingilbert/agl https://github.com/alaingilbert/agl
- noelwelsh 1y ago
  I feel OCaml is as easy to write as Go, though perhaps that is not the case for someone with a typical imperative programmer background. I feel a good part of Go's success is that the core language is very similar to Python, which is obviously very widely known. Most of the issues you discuss with Rust are not issues in OCaml, as OCaml has GC. The language is simpler, in that programs are concerned with fewer concepts (e.g. no lifetimes), but less expressive in that they cannot formally talk about these concepts (though see the Jane Street work to add in some Rust-like concerns: https://blog.janestreet.com/oxidizing-ocaml-locality/ https://blog.janestreet.com/oxidizing-ocaml-locality/)
- delifue 1y ago
  Go with sum type: Borgo https://github.com/borgo-lang/borgo https://github.com/borgo-lang/borgo
noelwelsh 1y ago
I believe it could have, and OCaml didn't because of some bad predictions by the core team (when asked about better concurrency, the response was along the lines that we'll all have tens of CPUs soon, so better in process support wasn't necessary), and the restrictions of academia (there is little value given to engineering work).
dvdkon 1y ago
Sadly I don't think so, just because it looks "too functional". If comments on the web are anything to go by, being familiar to C and JS programmers is one of the main reasons for Go's success. I think it has plenty of its own specifics, it's not like e.g. C# which really did start as a straightforward Java clone, but OCaml has even more differences.
- jchw 1y ago
  I think it's worth considering the possibility that imperative style is just more approachable than functional style. On one hand, some tasks that are basically trivial few-line solutions in a good functional programming language balloon to hundreds of lines of arguably more error prone code in especially dumb imperative languages like Go. On the other hand, it's really really easy to understand what the vast majority of Go code is doing by just reading it line-by-line. Don't get me wrong: it's easy to write broken and unreadable code in pretty much any programming language (I can write bad code in imperative or functional style, I'm a multi-paradigm bad programmer) but at the very least for most imperative code you can really just follow everything line-by-line. It's harder to pull apart complex compositions of map/reduce/filter/fold/etc. (especially if you involve other complex topics like async.) This does seem to be quite a conundrum. I don't think there's any argument regarding the merits of functional programming as a paradigm, but imperative just seems easier to wrap one's head around, and frankly most code is really boring and doesn't need to be able to do the super powerful things that functional programming can do with relative ease. I feel like I'm playing with fire by saying this here, but I think it's true.
  - andrewflnr 1y ago
    That's the thing about OCaml, though, it's relatively friendly with imperative programming too. At least, it's not picky about side effects the way Haskell is. It might still be a pain with loops, though. It's been a while since I used it.
  - suprtx 1y ago
    I think you are correct, perhaps even being too careful in your claims. I think functional and imperative programming, in practice, tend to have different types of constraints, and there are different things that are easy to reason about in them, and encourage different reasoning methods and approaches, and ways of proving properties about them. I think that for learning functional programming, being all-in on purely functional programming and never having side-effects or mutation, can be useful. But for practical programming, functional programming is not in itself a virtue just by itself. One should always consider the trade-offs of the specific case one is facing if one is not learning, as such. If a project has decided on a trade-off where purely functional programming is chosen, then maybe continuing with pure FP is fine; but even in such a case, a function that from outside is purely functional, but uses imperative programming internally, for instance for the sake of optimization, can make sense. There are different advantages and disadvantages to having: pure FP; imperative programming; or different kinds of mixes of FP and imperative programming. Overall, my approximate opinion is that it does not make sense to be dogmatic about functional programming, except when learning FP. https://stackoverflow.com/questions/7717691/why-is-the-minimalist-example-haskell-quicksort-not-a-true-quicksort https://stackoverflow.com/questions/7717691/why-is-the-minim... is arguably a relevant example, though a proper in-place implementation in Haskell might be more fair for comparison. Though I am not certain whether Haskell's lazy evaluation would help or hinder analysis, Ocaml or SML might be better languages for implementing and analysing an in-place implementation variant. Also, Hacker News is a censorship hole.
pjmlp 1y ago
Go only enjoys widespread adoption thanks to Docker and Kubernetes success in the industry, had it not been the case, we would be talking about it just like we talk about Limbo and Oberon-2, its influences. The kind of folks that dropped Python and Java for Go, would not have picked up OCaml even if it had a better concurrency story 10-15 years ago.
- andrewflnr 1y ago
  I guess part of the hypothetical is, "a version of OCaml supported and pushed by Google". But the same idea probably applies, Rob Pike et al (he was involved right?) probably never go for a functional language, even a relatively pragmatic one.
  - pjmlp 1y ago
    That would be Kotlin, Google's C#, given how history goes. I has enough ML influence on it, as a simplified Scala, and also can compile to native in various ways. On the other hand you will notice that the only thing Android team only cared about Go, was the Soong build syste, and parts of the GPU debugger.

eru 1y ago

> Lack of sum types is probably one of the worst things about working in Go, and I think it is a much bigger problem than the lack of generics ever was. Maybe, but once you have eg an Option or Either (a.k.a. Result) type, you typically really want to have some functions that work generically on all versions of them. (Though you could probably get away with using Go's version of void*, the empty interface, in a lot of cases?)

jchw 1y ago
Basically, anything that makes it possible to do an Etiher/Result type blows the whole damn thing up. It calls everything into question, e.g. whether multiple returns really makes sense. It's kind of a shame, because I would really like to be able to model things better (and ideally more efficiently too) when working in Go.
- eru 1y ago
  The 'multiple return values' thing is really stupid in any case, even without sum-types: it's like "we have tuples, but only for this one special case." Just add first-class tuples to the language.
  - dontlaugh 1y ago
    It’s an actual liability, since you can’t uniformly compose the results of functions with different numbers of return values.
    - aatd86 1y ago
      I think it could be retroffited but would require a syntax to have a variable marked as being a tuple. If we look behind the curtains, go functions somehow return tuples that are automatically unrolled into respective variables, sometimes. That's why if you have f(a, b, c){} and g() (b,c){} you can't do f(a, g()) currently. It does not unroll it automatically for some reason here. The result remains in "tuple" form ? I think it may leave some space to have tuples if needed. In any case, I don't see it as technically infeasible.

pjmlp 1y ago

Personally, by now I would already be happy with Pascal like enumerations, instead of the stupid iota/const hack, yes it is an hack. type StatusCodes = (Success, Ongoing, Done) or if you prefer, while bikeshedding what keyword to use, type StatusCodes enumeration (Success, Ongoing, Done) Naturally, it is so much better to write type StatusCodes int const ( Success StatusCodes = iota Ongoing Done )

jerf 1y ago

The major problem with "bolting on" sum types to Go is that it is way too redundant with interfaces and the interactions get weird, because this already works: type Color interface { isColor() } type Red struct {} func (r Red) isColor() {} type Green struct {} func (g Green) isColor() {} type RGB struct { Red byte Green byte Blue byte } func (r RGB) isColor() {} func PrintColor(color Color, text string) { switch c := color.(type) { case Red: // print in red here case Green: // print in green here case RGB: // handle the RGB } } This doesn't prevent the zero value of a Color being 'nil' but it does make "a value of type Color" effectively a sum type because you can't do anything useful to it without unpacking it in a type-aware way. There is no way in Go to have a Red but act like you have an RGB, values always have their types. (There is a common misconception among Go programmers that this doesn't seal the type because you can implement a "isColor" method on your own types in other packages, but it won't work. Try it if you want to be sure.) One could argue they've effectively already been bolted on, all that is missing is some syntax gloss like sum Color { Red{} Green{} RGB{ Red byte Green byte Blue byte } } that would simply unsugar to the above.

jchw 1y ago
Yes, this is all true. However, this approach has more downsides than just nil and cumbersome syntax imo. It takes more memory and pointer-chasing than "proper" sum types would require, and while you can "seal" the type, there's no exhaustiveness checking.