9 ms·

Generics facilitators in Go

legulere 5y ago
> Go type system is not a traditional type system How so, I don't see anything special about Go's type system in the linked article.
- noah_buddy 5y ago
  I believe they're talking about the semi-duck typing used in go for interfaces but I'm not certain. Interfaces are implicitly implemented with a runtime check determining whether a type x assigned to interface y is valid.This is why you'll often see an empty declaration in packages for structs intended to implement an interface like so: var _ y = x{} // syntax might be slightly wrong After writing this, I found the linked article in the OP link to what the author meant. https://rakyll.org/typesystem/ https://rakyll.org/typesystem/ There's really a bunch of small things that are different about golang that individually you can find elsewhere but are on the whole unique.
  - politician 5y ago
    var _ = ThingerInterface(&Struct{}) Creates a compile time error if Struct is not a ThingerInterface.
assbuttbuttass 5y ago
A nice pattern that will probably become common. I assume the limitation on generic methods is just an implementation issue, the vtable for an interface has to have a known number of methods, and they can't insert specializations for each type on the fly into the vtable.
- jerf 5y ago
  There is discussion of this in the proposal: https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#no-parameterized-methods https://go.googlesource.com/proposal/+/refs/heads/master/des... I'd call it a fundamental semantics issue. That discussion is mostly framed in terms of the compile-time problems that arise, but that brief aside about the reflection package is deceptively short, because it means that the entire compile time set of problems is also lifted into runtime, where the problem is even worse. (To translate for someone who isn't a Go expert, the reflection package has enough power that it can dynamically instantiate new types that the compiler did not and can not even in principle have seen at compile time, which may then require new methods to be created. The current Go runtime has no run-time code creation capabilities. [1]) Without the reflection problem, it's annoying but at least solvable in principle, but I don't see any reasonable design (without a fundamental overhaul to specify a runtime-available compiler in the language spec itself, which is a huge step) that solves the reflection issue. [1] Which personally I consider almost a feature. YMMV. Runtime compilation is very cool in a lot of ways but is a hell of a subsystem to try to secure, just by its very nature. There's a lot to be said for simply not having that capability unless it is integral to some particular program.
  - klodolph 5y ago
    Yes, I agree. I think one of the key problems here is that any solution would create a distinction between polymorphic and monomorphic methods. This is why languages are so much more than a bag of features... every feature interacts with other features, often in subtle ways. For example, take a "simple" feature like operator overloading... if you look at how operator overloading actually works in C++ and C#, it's an incredibly complex feature in both languages, with some surprising behavior. That surprising behavior arises because features interact in weird ways that you wouldn't ordinarily think about. Saying "no generic methods" is a nice out.
  - _old_dude_ 5y ago
    > it means that the entire compile time set of problems is also lifted into runtime It's what Swift does if inlining is defeated, the type arguments are passed at runtime as a classical argument on stack. You don't need full reflection as you suggest, only to be able to materialize the types as structs + an interface at runtime. Yes, it's a little slower but most of the overhead comes from the function not be inlined.
parhamn 5y ago
I recently saw a go generic data structures library in the wild: https://github.com/zyedidia/generic https://github.com/zyedidia/generic. Anyone have any more? I'm curious what people come up with for goroutines/channels.
- kodablah 5y ago
  Undocumented early impl/design of futures: https://github.com/cretz/fut https://github.com/cretz/fut. Examples: https://github.com/cretz/fut/blob/main/ops_test.go https://github.com/cretz/fut/blob/main/ops_test.go.
- nkcmr 5y ago
  I made this: https://github.com/nkcmr/async https://github.com/nkcmr/async so I can experiment with the idea of Promises in Go. Keep in mind it's experimental and I've only lightly played around with the pattern.
  - Zababa 5y ago
    That's interesting, I always found Go's handling of asynchronous stuff way better than other languages, especially the ones that relies on async.
    - cube2222 5y ago
      For those interested to learn more about Go's handling of async, I wrote a comment a while ago where I describe why I too think the Go model is superior: https://news.ycombinator.com/item?id=27545031 https://news.ycombinator.com/item?id=27545031 EDIT: Superior for most non-performance sensitive use cases, that is.
      - Zababa 5y ago
        I personally prefer Go's handling of async too. I'm not sure if I would call it superior, because from what I understand monadic async has its uses, especially when you don't have a runtime. But for the kind of programming that I do, which doesn't need every last bit of performance, monadic async is a pain. For people not familiar with "monadic async", it's the formal way to refer to the function coloring problem [1]. Monads offer you ways to "lift" your functions/values in the monadic world (paint the blue values red), but you can't do the opposite (paint the red functions/values blue). [1]: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...
arwineap 5y ago
Not terribly excited for generics.
- sigmonsays 5y ago
  I have this concern too. I wish to remain optimistic though. i've enjoyed go for its simplicity. At times it requires more code (or code generation) but that generated code was easier to debug and have tooling operate on. With generics, Some things will suffer. Complexity of code and tooling increases, tooling is probably going to suffer and become more costly. compile time and performance will probably also be affected. Generics are likely only great for collections of things, ie []*T, where, you need to do the same thing on many of T.
svnpenn 5y ago
I really dont like this. It just feels like people are trying to ram a square peg into a round hole. Personally I dont get why generics were needed in the first place. Every single example Ive seen so far, could be pretty easily replaced with `interface{}`. And now that we finally have generics, people still arent happy. Now that they have their round peg, they are trying to now ram it into a star shaped hole, as is being done in this post. I really wish people would just use the language as it is. Yes, it does cause for some repetition, but thats not the worst thing in the world. Compared to some Java codebases I have seen, Go is extremely simple already, in terms of "flatness" in the filesystem, and flatness in the type system. I think people should just know when to leave well enough alone. I dont think the complexity of generics is worth it for 99/100 use cases.
- fsdjkflsjfsoij 5y ago
  > Every single example Ive seen so far, could be pretty easily replaced with `interface{}` Of course they could just use interface{}... But by doing that they lose both type safety and performance.
  - svnpenn 5y ago
    > they lose both type safety and performance I guess you think generics is a magic pixie dust that has the exact performance envelope as non-generic code?
    - fsdjkflsjfsoij 5y ago
      That depends entirely on the implementation. Almost every respectable implementation strategy is going to have better performance characteristics than an interface{} though.
    - sidlls 5y ago
      Not exact in all cases, but in some and very close in most others. And it isn't "magic pixie dust"--it is a lot of hard work to implement generics in a way that is both ergonomic for the programmer and efficient.
mattarm 5y ago
"All problems in computer science can be solved by another level of indirection" - Butler Lampson See https://en.wikipedia.org/wiki/Indirection https://en.wikipedia.org/wiki/Indirection
- deleted 5y ago
  [deleted]
  - deleted 5y ago
    [deleted]
- infogulch 5y ago
  "Except for the problem of too many levels of indirection" - ?
  - adgjlsfhk1 5y ago
    No, you solve that problem by adding another layer of indirection (aka a caching layer)
kubb 5y ago
> Go type system is not a traditional type system and it was not possible just to bring an existing generics implementation from other language and be done. I claim that there are no original ideas in Go's type system and in the way that Go does generics, and the reason it took so long was because it came out of an environment with actors (including a part of the community) pushing against the feature. You might disagree, I'm curious what original features of Go you will point out to me.
- verdverm 5y ago
  While not necessarily part of the language, but arguably part of Go, something novel is the dep mgmt system. I have not encountered the MVS algorithm in another language. CUE is intending to adopt it.
  - pxc 5y ago
    Where can non-Go (for now!) developers read about this to get a sense of what makes it interesting?
- randomdata 5y ago
  Go's generics have been in development (by the same person) since the pre-1.0 days, with various proposals submitted for their inclusion. Are you suggesting that those earlier proposals failed for frivolous reasons rather than real technical concerns? It seems to me that that it was less about active pushback and more that people who cared about having generics in Go was limited to an exceedingly small group of people who weren't necessarily experts in the subject matter. In fact, the Go team hired an external expert to lay the groundwork for what became the winning proposal.
- anonymoushn 5y ago
  A great innovation of Go is that it includes a built-in reference type with both inner nils and outer nils.
  - kubb 5y ago
    It is a bit like void *outer = 0; // outer can be nil int val = 5; int *inner = &val; // inner can also be nil outer = (void*)(&inner) // can have an inner value, inner nil or outer nil
jayd16 5y ago
Reminds me of Java's use of listener objects because it lacked (at the time) lambdas and method references for callbacks. Hopefully Go generics continue to improve.
- kubb 5y ago
  I wonder if the limitation of not allowing type parameters for methods came from wanting to prevent people from using it for certain features that would be considered "too complicated for Go", or from it being "too complicated to support" or from "we won't be needing it anyways". Another option to circumvent this is to use top-level functions, and accept the receiver as the first argument of the function. This is effectively the same thing, the only thing it's missing is the object.method() syntax that's familiar to OO programmers.
  - randomdata 5y ago
    No need to wonder. The proposal goes into lengthy detail as to why they are not included. There is hope that they will be added in the future, but solutions to the problems will need to be found first.
    - kubb 5y ago
      I wouldn't call it lengthy, but if anyone is wondering, the proposal points out that parametrized methods don't have an obvious behavior _inside interfaces_. No mention of why they left them out in non-interface types.
      - Ericson2314 5y ago
        Yeah they are just being ignorant once again. Even "static" interface usage, a la Rust Traits, is fine. But Go doesn't distinguish between the static stuff and the dynamic vtable stuff very well at all. That's the problem they should fix.
        randomdata 5y ago
Groxx 5y ago
You can also cast between these "facilitators", since the unused type doesn't affect memory layout: https://gotipplay.golang.org/p/2w2y1KEjXVE https://gotipplay.golang.org/p/2w2y1KEjXVE (from https://news.ycombinator.com/item?id=29586456 https://news.ycombinator.com/item?id=29586456 ) I think they'll be pretty common. It's a simple structure, and simple to manipulate.
kune 5y ago
Go 1.18 supports generic functions. Wouldn't a generic function much simpler than an additional type for that use case?
deleted 5y ago
[deleted]
flippinburgers 5y ago
Why not just use an interface for this?
- cpuguy83 5y ago
  Do you have an example of how to make this work with an interface?
  - flippinburgers 5y ago
    The author conveniently leaves out the implementation of the actual method calls themselves, which is frankly the important part. interface Query { All ... Insert ... Update ... Delete ... WhateverElse ... } Now admittedly, the implementation is probably cleaner with "generics", but you can generically approach the problem with interfaces.
    - cpuguy83 5y ago
      But how do you get the thing that implements that interface? Either the framework has to know how to create a querier for each specific type, drop to using empty interfaces, or use generics.
      - flippinburgers 5y ago
        The point of the interface is that the person authoring the code defines the internals.
asoliveira 5y ago
C++ supports generics "template" since 1989. It is a shame for Golang hasn't been designed with this feature since from the beginning.

> Go type system is not a traditional type system How so, I don't see anything special about Go's type system in the linked article.

noah_buddy 5y ago
I believe they're talking about the semi-duck typing used in go for interfaces but I'm not certain. Interfaces are implicitly implemented with a runtime check determining whether a type x assigned to interface y is valid.This is why you'll often see an empty declaration in packages for structs intended to implement an interface like so: var _ y = x{} // syntax might be slightly wrong After writing this, I found the linked article in the OP link to what the author meant. https://rakyll.org/typesystem/ https://rakyll.org/typesystem/ There's really a bunch of small things that are different about golang that individually you can find elsewhere but are on the whole unique.
- politician 5y ago
  var _ = ThingerInterface(&Struct{}) Creates a compile time error if Struct is not a ThingerInterface.

assbuttbuttass 5y ago

A nice pattern that will probably become common. I assume the limitation on generic methods is just an implementation issue, the vtable for an interface has to have a known number of methods, and they can't insert specializations for each type on the fly into the vtable.

jerf 5y ago
There is discussion of this in the proposal: https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#no-parameterized-methods https://go.googlesource.com/proposal/+/refs/heads/master/des... I'd call it a fundamental semantics issue. That discussion is mostly framed in terms of the compile-time problems that arise, but that brief aside about the reflection package is deceptively short, because it means that the entire compile time set of problems is also lifted into runtime, where the problem is even worse. (To translate for someone who isn't a Go expert, the reflection package has enough power that it can dynamically instantiate new types that the compiler did not and can not even in principle have seen at compile time, which may then require new methods to be created. The current Go runtime has no run-time code creation capabilities. [1]) Without the reflection problem, it's annoying but at least solvable in principle, but I don't see any reasonable design (without a fundamental overhaul to specify a runtime-available compiler in the language spec itself, which is a huge step) that solves the reflection issue. [1] Which personally I consider almost a feature. YMMV. Runtime compilation is very cool in a lot of ways but is a hell of a subsystem to try to secure, just by its very nature. There's a lot to be said for simply not having that capability unless it is integral to some particular program.
- klodolph 5y ago
  Yes, I agree. I think one of the key problems here is that any solution would create a distinction between polymorphic and monomorphic methods. This is why languages are so much more than a bag of features... every feature interacts with other features, often in subtle ways. For example, take a "simple" feature like operator overloading... if you look at how operator overloading actually works in C++ and C#, it's an incredibly complex feature in both languages, with some surprising behavior. That surprising behavior arises because features interact in weird ways that you wouldn't ordinarily think about. Saying "no generic methods" is a nice out.
- _old_dude_ 5y ago
  > it means that the entire compile time set of problems is also lifted into runtime It's what Swift does if inlining is defeated, the type arguments are passed at runtime as a classical argument on stack. You don't need full reflection as you suggest, only to be able to materialize the types as structs + an interface at runtime. Yes, it's a little slower but most of the overhead comes from the function not be inlined.

parhamn 5y ago

I recently saw a go generic data structures library in the wild: https://github.com/zyedidia/generic https://github.com/zyedidia/generic. Anyone have any more? I'm curious what people come up with for goroutines/channels.

kodablah 5y ago
Undocumented early impl/design of futures: https://github.com/cretz/fut https://github.com/cretz/fut. Examples: https://github.com/cretz/fut/blob/main/ops_test.go https://github.com/cretz/fut/blob/main/ops_test.go.
nkcmr 5y ago
I made this: https://github.com/nkcmr/async https://github.com/nkcmr/async so I can experiment with the idea of Promises in Go. Keep in mind it's experimental and I've only lightly played around with the pattern.
- Zababa 5y ago
  That's interesting, I always found Go's handling of asynchronous stuff way better than other languages, especially the ones that relies on async.
  - cube2222 5y ago
    For those interested to learn more about Go's handling of async, I wrote a comment a while ago where I describe why I too think the Go model is superior: https://news.ycombinator.com/item?id=27545031 https://news.ycombinator.com/item?id=27545031 EDIT: Superior for most non-performance sensitive use cases, that is.
    - Zababa 5y ago
      I personally prefer Go's handling of async too. I'm not sure if I would call it superior, because from what I understand monadic async has its uses, especially when you don't have a runtime. But for the kind of programming that I do, which doesn't need every last bit of performance, monadic async is a pain. For people not familiar with "monadic async", it's the formal way to refer to the function coloring problem [1]. Monads offer you ways to "lift" your functions/values in the monadic world (paint the blue values red), but you can't do the opposite (paint the red functions/values blue). [1]: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...

arwineap 5y ago

Not terribly excited for generics.

sigmonsays 5y ago
I have this concern too. I wish to remain optimistic though. i've enjoyed go for its simplicity. At times it requires more code (or code generation) but that generated code was easier to debug and have tooling operate on. With generics, Some things will suffer. Complexity of code and tooling increases, tooling is probably going to suffer and become more costly. compile time and performance will probably also be affected. Generics are likely only great for collections of things, ie []*T, where, you need to do the same thing on many of T.

svnpenn 5y ago

I really dont like this. It just feels like people are trying to ram a square peg into a round hole. Personally I dont get why generics were needed in the first place. Every single example Ive seen so far, could be pretty easily replaced with `interface{}`. And now that we finally have generics, people still arent happy. Now that they have their round peg, they are trying to now ram it into a star shaped hole, as is being done in this post. I really wish people would just use the language as it is. Yes, it does cause for some repetition, but thats not the worst thing in the world. Compared to some Java codebases I have seen, Go is extremely simple already, in terms of "flatness" in the filesystem, and flatness in the type system. I think people should just know when to leave well enough alone. I dont think the complexity of generics is worth it for 99/100 use cases.

fsdjkflsjfsoij 5y ago
> Every single example Ive seen so far, could be pretty easily replaced with `interface{}` Of course they could just use interface{}... But by doing that they lose both type safety and performance.
- svnpenn 5y ago
  > they lose both type safety and performance I guess you think generics is a magic pixie dust that has the exact performance envelope as non-generic code?
  - fsdjkflsjfsoij 5y ago
    That depends entirely on the implementation. Almost every respectable implementation strategy is going to have better performance characteristics than an interface{} though.
  - sidlls 5y ago
    Not exact in all cases, but in some and very close in most others. And it isn't "magic pixie dust"--it is a lot of hard work to implement generics in a way that is both ergonomic for the programmer and efficient.

mattarm 5y ago

"All problems in computer science can be solved by another level of indirection" - Butler Lampson See https://en.wikipedia.org/wiki/Indirection https://en.wikipedia.org/wiki/Indirection

deleted 5y ago
[deleted]
- deleted 5y ago
  [deleted]
infogulch 5y ago
"Except for the problem of too many levels of indirection" - ?
- adgjlsfhk1 5y ago
  No, you solve that problem by adding another layer of indirection (aka a caching layer)

kubb 5y ago

> Go type system is not a traditional type system and it was not possible just to bring an existing generics implementation from other language and be done. I claim that there are no original ideas in Go's type system and in the way that Go does generics, and the reason it took so long was because it came out of an environment with actors (including a part of the community) pushing against the feature. You might disagree, I'm curious what original features of Go you will point out to me.

verdverm 5y ago
While not necessarily part of the language, but arguably part of Go, something novel is the dep mgmt system. I have not encountered the MVS algorithm in another language. CUE is intending to adopt it.
- pxc 5y ago
  Where can non-Go (for now!) developers read about this to get a sense of what makes it interesting?
randomdata 5y ago
Go's generics have been in development (by the same person) since the pre-1.0 days, with various proposals submitted for their inclusion. Are you suggesting that those earlier proposals failed for frivolous reasons rather than real technical concerns? It seems to me that that it was less about active pushback and more that people who cared about having generics in Go was limited to an exceedingly small group of people who weren't necessarily experts in the subject matter. In fact, the Go team hired an external expert to lay the groundwork for what became the winning proposal.
anonymoushn 5y ago
A great innovation of Go is that it includes a built-in reference type with both inner nils and outer nils.
- kubb 5y ago
  It is a bit like void *outer = 0; // outer can be nil int val = 5; int *inner = &val; // inner can also be nil outer = (void*)(&inner) // can have an inner value, inner nil or outer nil

jayd16 5y ago

Reminds me of Java's use of listener objects because it lacked (at the time) lambdas and method references for callbacks. Hopefully Go generics continue to improve.

kubb 5y ago
I wonder if the limitation of not allowing type parameters for methods came from wanting to prevent people from using it for certain features that would be considered "too complicated for Go", or from it being "too complicated to support" or from "we won't be needing it anyways". Another option to circumvent this is to use top-level functions, and accept the receiver as the first argument of the function. This is effectively the same thing, the only thing it's missing is the object.method() syntax that's familiar to OO programmers.
- randomdata 5y ago
  No need to wonder. The proposal goes into lengthy detail as to why they are not included. There is hope that they will be added in the future, but solutions to the problems will need to be found first.
  - kubb 5y ago
    I wouldn't call it lengthy, but if anyone is wondering, the proposal points out that parametrized methods don't have an obvious behavior _inside interfaces_. No mention of why they left them out in non-interface types.
    - Ericson2314 5y ago
      Yeah they are just being ignorant once again. Even "static" interface usage, a la Rust Traits, is fine. But Go doesn't distinguish between the static stuff and the dynamic vtable stuff very well at all. That's the problem they should fix.
      - randomdata 5y ago

Groxx 5y ago

You can also cast between these "facilitators", since the unused type doesn't affect memory layout: https://gotipplay.golang.org/p/2w2y1KEjXVE https://gotipplay.golang.org/p/2w2y1KEjXVE (from https://news.ycombinator.com/item?id=29586456 https://news.ycombinator.com/item?id=29586456 ) I think they'll be pretty common. It's a simple structure, and simple to manipulate.

kune 5y ago

Go 1.18 supports generic functions. Wouldn't a generic function much simpler than an additional type for that use case?

deleted 5y ago

[deleted]

flippinburgers 5y ago

Why not just use an interface for this?

cpuguy83 5y ago
Do you have an example of how to make this work with an interface?
- flippinburgers 5y ago
  The author conveniently leaves out the implementation of the actual method calls themselves, which is frankly the important part. interface Query { All ... Insert ... Update ... Delete ... WhateverElse ... } Now admittedly, the implementation is probably cleaner with "generics", but you can generically approach the problem with interfaces.
  - cpuguy83 5y ago
    But how do you get the thing that implements that interface? Either the framework has to know how to create a querier for each specific type, drop to using empty interfaces, or use generics.
    - flippinburgers 5y ago
      The point of the interface is that the person authoring the code defines the internals.

asoliveira 5y ago

C++ supports generics "template" since 1989. It is a shame for Golang hasn't been designed with this feature since from the beginning.