6 ms·

Hacking sum types with Go generics

34 points by lawrjone 5y ago

lawrjone 5y ago
Author here With generics coming in go 1.18, we can finally make a sum type abstraction that is a bit friendlier than the previous alternatives. Obviously having this in the language would be best, but just having a construct that can check you've handled each case is a big improvement in my mind.
- slaymaker1907 5y ago
  Why not just use the visitor pattern? Sum types are obviously better, but the Church encoding for sum types is clunkier than visitors in my opinion.
howeyc 5y ago
I had a hard time getting your examples into a state that worked. Even after that, I don't see how what you are doing improves over a switch. It actually looks less readable to me, with no gain in functionality at all. https://gotipplay.golang.org/p/xpuAIC5xdL_7 https://gotipplay.golang.org/p/xpuAIC5xdL_7
- lawrjone 5y ago
  Hey! Your example is actually working, in that you've initialised a field type of an empty string which isn't a valid enum value. I agree this is slightly less readable than the native switch, but it means the compiler can tell you when parts of your code try handling all enum values but don't, in fact, handle them all. For the default switch you'll only find out at runtime, which isn't very useful when developing. Does that make sense? The compiler warning in the article is probably the best example of what I mean here.
  - howeyc 5y ago
    But in my example it only found out at runtime too.
  - konart 5y ago
    >For the default switch you'll only find out at runtime, which isn't very useful when developing. Test are useful when developing. Tests should've told you that you are not handling the input as expected.
    - lawrjone 5y ago
      You're right, which is why we don't need types, I guess? I'm being facetious, but if you're using a statically typed language you've already bought into types helping you across your codebase being worthwhile. Equally, this situation is one where the testing point doesn't work. Your tests for the existing functionality won't, obviously, be testing the new functionality you're creating. So they won't be able to catch that one part of your codebase doesn't support the new feature, while this solution can help you with that!
      - konart 5y ago
        Let's say you have a gun at home. This doesn't mean you have to shoot it every time the door bell rings. >Your tests for the existing functionality won't, obviously, be testing the new functionality you're creating. Well... duh, that's why write test in the first place. To test new functionality. In fact some people prefer writing test before the functionality in question. >this solution can help you with that! While making the codebase (subjectively) less readable at least. This maybe a viable option for a pet project but I wouldn't be happy to have it for a code review with the only reason being "now you don't have to test it" (you will have to test it anyway most likely)
        lawrjone 5y ago
        > you will have to test it anyway most likely I think we're cross talking. One of the primary advantages of this approach is that when you add a new value to that switch, your compiler will tell you about all the parts of the codebase that need updating to handle it. Then you can use the compiler errors to help you write the tests for the new feature. You might say you'd prefer to just search the codebase for parts you need to update, and you're ok with forgetting the odd thing and finding out when it's broken after shipping. That's ok, but I'd prefer to have the compiler help me do this rather than doing the work myself.
- woah 5y ago
  There's a huge gain in functionality: The compiler will tell you if you messed up
  - rplnt 5y ago
    Go is very fast to lint. Having linter to check whether your switch statement is exhaustive is a good idea regardless.
    - woah 5y ago
      That's irrelevant to the task at hand. The linter can't tell you if your type switch can handle every `interface{}` that might be passed into it some day.
  - lawrjone 5y ago
    100%
ngalaiko 5y ago
well, here it comes... introducing generics was a mistake. give it a couple of years and golang would be indistinguishable from java
- pjmlp 5y ago
  That is the thing, Go could have learned from the past and been designed properly fromt the get go instead of accumulating hacks to fix design shortcomings.
- akavi 5y ago
  Java doesn’t have union types
  - _old_dude_ 5y ago
    sealed typed are union types, granted they have been introduced very recently.
    - akavi 5y ago
      Oh, interesting. TIL, thanks!
- LambdaComplex 5y ago
  Is there an actual issue that you have with sum types?
  - mutatio 5y ago
    I think (at least from my perspective) the issue here is that sum types are not first class citizens supported by the compiler, for example non-exhaustive matching in Rust would result in a compile time error. Implementations like this are a mere approximation - and takes a while to grep exactly what is going on (at least for me).
- lawrjone 5y ago
  Just curious on this, what about this do you think is a mistake? You could do this before without generics, you'd just be unsafely casting types (which can panic at runtime) or assigning to a closured variable, which is quite indirect.
jerf 5y ago
This is just a case of non-idiomatic programming in the first place, and it's not even non-idiomatic "Go" programming, it's non-idiomatic for object-oriented languages in general. The correct solution to this sort of problem is more like this: type OptionParser interface { ParseValue(data string) (*Value, error) } type OptionField struct {} func (of *OptionField) ParseValue(data string) (*Value, error) { return &Value{OptionID: data}, nil } type NumericField struct {} func (nf *NumericField) ParseValue(data string) (*Value, error) { value, err := strconv.ParseFloat(data, 64) if err != nil { return nil, errors.New("invalid value, must be a number") } return &Value{Numeric: value}, nil } and so on. A non-Go language would use whatever the local Abstract Data Type abstraction is. The parseValue function just melts away entirely, you just call a method on the value you have in hand. And now, instead of having already spent a great deal of design budget on this abstraction, and having a hard time integrating it into any future abstractions you may want to make, you're using language constructs in the way they were designed, and there's more space here to make further adaptations. Generics could even help with some of those perhaps, but they're not needed here. It's possible the Value type could be helped a bit with generics, but it's not clear. Probably the old-school existing "sum types" techniques would work just fine: http://www.jerf.org/iri/post/2917 http://www.jerf.org/iri/post/2917 But "a field per possible value" isn't necessarily a bad choice, especially if it's only a handful of fields. Sometimes it's the best performing implementation, even if it may seem skeevy at the programmer level, because it involves the least indirection even if it seems to involve the largest (in RAM size) objects, and being able to pass them around by value rather than pointers or interfaces means inlining may end up doing more for you too. In general, nowadays, you're better off using 64 bytes in one place than you are using 16 bytes here and 16 bytes somewhere else through a pointer, even if 64 > 32. It is helpful to understand the expression problem to understand what's going on here: https://en.wikipedia.org/wiki/Expression_problem https://en.wikipedia.org/wiki/Expression_problem In short, OO excels in adding values that implement an existing set of operations, functional programming excels in adding operations to existing sets of data types. Forcing sum types here into a language that doesn't really want them means that you're taking a techique that is good at the wrong dimension (you are adding more data types here that fit into the existing operations, what OO is already good at), then having to twist it around in a way it isn't really comfortable with to get it to work with what you want (in this case it'll manifest as needing to add this code to every method, of which I'm sure you have more than the ones shown; in properly-applied FP you get usually get the advantage by adding many fewer clauses to your pattern matches, often just one), when the correct solution was already there. While I often complain about trying to jam FP into OO without understanding in the context of map/filter/reduce-based programming, this is actually the even more profound difference between the two. It's nice to have the FP toolkit around in an OO language for when you encounter a problem it can help with, but when you're in an OO language and right in OO's wheelhouse, use the OO. Don't jam the FP in because FP is Platonically better or something. It actually has well-characterizable weaknesses vs. object orientation, and at least as it stands now is absolutely not universally better at everything. (Some up-and-coming languages claim to have solved the expression problem. I have not evaluated any of these claims. More power to them if they are correct!)
skybrian 5y ago
How does this affect performance? Maybe sticking with switch statements and writing a linter of some sort to do exhaustiveness checks would be a better idea?
- lawrjone 5y ago
  I should say, we have absolutely no performance requirements, at least in terms of what this will do. If we could add 10% developer efficiency for 20% performance hit, we'd take that in a heartbeat. The nanoseconds this construct might cost are just irrelevant to us, our customers, or our bottomline. Obviously if you're writing some low-level performance sensitive code then this won't be free, but I expect most people are in our situation.
- wlamartin 5y ago
  Such a thing exists: https://github.com/BurntSushi/go-sumtype https://github.com/BurntSushi/go-sumtype
flippinburgers 5y ago
Frankly painful to read that code.

34 points by lawrjone 5y ago

Author here With generics coming in go 1.18, we can finally make a sum type abstraction that is a bit friendlier than the previous alternatives. Obviously having this in the language would be best, but just having a construct that can check you've handled each case is a big improvement in my mind.

slaymaker1907 5y ago
Why not just use the visitor pattern? Sum types are obviously better, but the Church encoding for sum types is clunkier than visitors in my opinion.

howeyc 5y ago

I had a hard time getting your examples into a state that worked. Even after that, I don't see how what you are doing improves over a switch. It actually looks less readable to me, with no gain in functionality at all. https://gotipplay.golang.org/p/xpuAIC5xdL_7 https://gotipplay.golang.org/p/xpuAIC5xdL_7

lawrjone 5y ago
Hey! Your example is actually working, in that you've initialised a field type of an empty string which isn't a valid enum value. I agree this is slightly less readable than the native switch, but it means the compiler can tell you when parts of your code try handling all enum values but don't, in fact, handle them all. For the default switch you'll only find out at runtime, which isn't very useful when developing. Does that make sense? The compiler warning in the article is probably the best example of what I mean here.
- howeyc 5y ago
  But in my example it only found out at runtime too.
- konart 5y ago
  >For the default switch you'll only find out at runtime, which isn't very useful when developing. Test are useful when developing. Tests should've told you that you are not handling the input as expected.
  - lawrjone 5y ago
    You're right, which is why we don't need types, I guess? I'm being facetious, but if you're using a statically typed language you've already bought into types helping you across your codebase being worthwhile. Equally, this situation is one where the testing point doesn't work. Your tests for the existing functionality won't, obviously, be testing the new functionality you're creating. So they won't be able to catch that one part of your codebase doesn't support the new feature, while this solution can help you with that!
    - konart 5y ago
      Let's say you have a gun at home. This doesn't mean you have to shoot it every time the door bell rings. >Your tests for the existing functionality won't, obviously, be testing the new functionality you're creating. Well... duh, that's why write test in the first place. To test new functionality. In fact some people prefer writing test before the functionality in question. >this solution can help you with that! While making the codebase (subjectively) less readable at least. This maybe a viable option for a pet project but I wouldn't be happy to have it for a code review with the only reason being "now you don't have to test it" (you will have to test it anyway most likely)
      - lawrjone 5y ago
        > you will have to test it anyway most likely I think we're cross talking. One of the primary advantages of this approach is that when you add a new value to that switch, your compiler will tell you about all the parts of the codebase that need updating to handle it. Then you can use the compiler errors to help you write the tests for the new feature. You might say you'd prefer to just search the codebase for parts you need to update, and you're ok with forgetting the odd thing and finding out when it's broken after shipping. That's ok, but I'd prefer to have the compiler help me do this rather than doing the work myself.
woah 5y ago
There's a huge gain in functionality: The compiler will tell you if you messed up
- rplnt 5y ago
  Go is very fast to lint. Having linter to check whether your switch statement is exhaustive is a good idea regardless.
  - woah 5y ago
    That's irrelevant to the task at hand. The linter can't tell you if your type switch can handle every `interface{}` that might be passed into it some day.
- lawrjone 5y ago
  100%

ngalaiko 5y ago

well, here it comes... introducing generics was a mistake. give it a couple of years and golang would be indistinguishable from java

pjmlp 5y ago
That is the thing, Go could have learned from the past and been designed properly fromt the get go instead of accumulating hacks to fix design shortcomings.
akavi 5y ago
Java doesn’t have union types
- _old_dude_ 5y ago
  sealed typed are union types, granted they have been introduced very recently.
  - akavi 5y ago
    Oh, interesting. TIL, thanks!
LambdaComplex 5y ago
Is there an actual issue that you have with sum types?
- mutatio 5y ago
  I think (at least from my perspective) the issue here is that sum types are not first class citizens supported by the compiler, for example non-exhaustive matching in Rust would result in a compile time error. Implementations like this are a mere approximation - and takes a while to grep exactly what is going on (at least for me).
lawrjone 5y ago
Just curious on this, what about this do you think is a mistake? You could do this before without generics, you'd just be unsafely casting types (which can panic at runtime) or assigning to a closured variable, which is quite indirect.

jerf 5y ago

This is just a case of non-idiomatic programming in the first place, and it's not even non-idiomatic "Go" programming, it's non-idiomatic for object-oriented languages in general. The correct solution to this sort of problem is more like this: type OptionParser interface { ParseValue(data string) (*Value, error) } type OptionField struct {} func (of *OptionField) ParseValue(data string) (*Value, error) { return &Value{OptionID: data}, nil } type NumericField struct {} func (nf *NumericField) ParseValue(data string) (*Value, error) { value, err := strconv.ParseFloat(data, 64) if err != nil { return nil, errors.New("invalid value, must be a number") } return &Value{Numeric: value}, nil } and so on. A non-Go language would use whatever the local Abstract Data Type abstraction is. The parseValue function just melts away entirely, you just call a method on the value you have in hand. And now, instead of having already spent a great deal of design budget on this abstraction, and having a hard time integrating it into any future abstractions you may want to make, you're using language constructs in the way they were designed, and there's more space here to make further adaptations. Generics could even help with some of those perhaps, but they're not needed here. It's possible the Value type could be helped a bit with generics, but it's not clear. Probably the old-school existing "sum types" techniques would work just fine: http://www.jerf.org/iri/post/2917 http://www.jerf.org/iri/post/2917 But "a field per possible value" isn't necessarily a bad choice, especially if it's only a handful of fields. Sometimes it's the best performing implementation, even if it may seem skeevy at the programmer level, because it involves the least indirection even if it seems to involve the largest (in RAM size) objects, and being able to pass them around by value rather than pointers or interfaces means inlining may end up doing more for you too. In general, nowadays, you're better off using 64 bytes in one place than you are using 16 bytes here and 16 bytes somewhere else through a pointer, even if 64 > 32. It is helpful to understand the expression problem to understand what's going on here: https://en.wikipedia.org/wiki/Expression_problem https://en.wikipedia.org/wiki/Expression_problem In short, OO excels in adding values that implement an existing set of operations, functional programming excels in adding operations to existing sets of data types. Forcing sum types here into a language that doesn't really want them means that you're taking a techique that is good at the wrong dimension (you are adding more data types here that fit into the existing operations, what OO is already good at), then having to twist it around in a way it isn't really comfortable with to get it to work with what you want (in this case it'll manifest as needing to add this code to every method, of which I'm sure you have more than the ones shown; in properly-applied FP you get usually get the advantage by adding many fewer clauses to your pattern matches, often just one), when the correct solution was already there. While I often complain about trying to jam FP into OO without understanding in the context of map/filter/reduce-based programming, this is actually the even more profound difference between the two. It's nice to have the FP toolkit around in an OO language for when you encounter a problem it can help with, but when you're in an OO language and right in OO's wheelhouse, use the OO. Don't jam the FP in because FP is Platonically better or something. It actually has well-characterizable weaknesses vs. object orientation, and at least as it stands now is absolutely not universally better at everything. (Some up-and-coming languages claim to have solved the expression problem. I have not evaluated any of these claims. More power to them if they are correct!)

skybrian 5y ago

How does this affect performance? Maybe sticking with switch statements and writing a linter of some sort to do exhaustiveness checks would be a better idea?

lawrjone 5y ago
I should say, we have absolutely no performance requirements, at least in terms of what this will do. If we could add 10% developer efficiency for 20% performance hit, we'd take that in a heartbeat. The nanoseconds this construct might cost are just irrelevant to us, our customers, or our bottomline. Obviously if you're writing some low-level performance sensitive code then this won't be free, but I expect most people are in our situation.
wlamartin 5y ago
Such a thing exists: https://github.com/BurntSushi/go-sumtype https://github.com/BurntSushi/go-sumtype

flippinburgers 5y ago

Frankly painful to read that code.