19 ms·
Go generics are not bad
- mirashii 4y agoA lot of folks have commented on this being a cherry-picked example, but I haven't yet seen someone link to a set of examples that show the opposite: pathologically bad performance with Go's generics. https://planetscale.com/blog/generics-can-make-your-go-code-slower https://planetscale.com/blog/generics-can-make-your-go-code-... is a much more in depth dive into how Go's generics work that shows some of the easy cases, like this article, but also the more general cases.
- bediger4000 4y agoThis is good to hear. There's something like a cognitive bias that makes me leary of genetics (in any language). Once you have some cross cutting feature, generics or object orientation, or in functional programming, those functions of type 'a->'a, or assembly language address modes, people get bogged down by trying to make everything in their work generic or object oriented or "orthogonal". Lemire's example is relevant to his topic, but it could be a poster child if someone else did it. They need to sum an array of int. So they spend 4 days getting a function and a generic type that can sum every numerical type, rather than 15 minutes to sum int arrays. What's this called?
- alanbernstein 4y agoPremature abstraction?
- nicoburns 4y ago> They need to sum an array of int. So they spend 4 days getting a function and a generic type that can sum every numerical type, rather than 15 minutes to sum int arrays. More likely they install a 3rd party generic sum function and don't have to write their own at all. Well, in the case of a sum function it's probably only 15 minutes to write to generic version, and there may even be one in the standard library. But consider something like a concurrent hash map. In Rust, You just `cargo add dashmap` and you have one that works with any type that works with a regular hashmap. This isn't possible without generics.
- Beltalowda 4y agoThe sum example is simple, but very often have I seen people write general solutions for business logic or whatnot that you can't just get from a 3rd party package, when it wasn't at all clear that's needed. I have done this myself too.
- e12e 4y ago>> They need to sum an array of int. So they spend 4 days getting a function and a generic type that can sum every numerical type, rather than 15 minutes to sum int arrays > More likely they install a 3rd party generic sum function and don't have to write their own at all. Well, in the case of a sum function it's probably only 15 minutes to write to generic version I'm not sure "sum over an array of int" is a great "simple" example - how do you guarantee that the result can fit in an int? This is (one reason) why Julia implemented a full, scheme-style, numeric tower right in the core language. Sure there are some examples where you can sum N Mbit numbers into an Mbit number, and either want overflow, or is somehow "confident" things will fit. But in general it's just lazy "no one will ever have such a large shopping cart"-design...
- anonymoushn 4y ago> This is (one reason) why Julia implemented a full, scheme-style, numeric tower right in the core language. What's the result of adding two i32 then? Is it an i64? Doesn't the accumulator need to get wider on some additions, since adding an i64 to an i32 might also overflow an i64?
- nvartolomei 4y agoThis was just discussed here in another front page post “Why DRY is the most over-rated programming principle” https://news.ycombinator.com/item?id=32010699 https://news.ycombinator.com/item?id=32010699
- TheDong 4y ago> What's this called? The blub paradox https://wiki.c2.com/?BlubParadox https://wiki.c2.com/?BlubParadox
- mseepgood 4y agoI don't think that's what the blub paradox is. I think the parent comment is arguing the opposite. People who believe in the blub paradox are usually so smug and full of themselves that they don't pursue simple solutions.
- TheDong 4y agoThe parent commenter is dismissing generics, and various other features, as something that "bogs people down" and makes them write worse code. This is the essence of the blub paradox, thinking that people who use a feature are simply misusing it, after all in blub you don't have generics and things are fine. They're just making things more complicated than they need to be. The blub programmer isn't necessarily smug, they just make excuses to dismiss more powerful language features that they are not sufficiently familiar with.
- ShamelessC 4y agoSame thing happens in the other direction too e.g. Rust coders trying to use Python without familiarity with dynamic scripting languages and dismisses the whole language as a footgun (despite Rust’s learning curve being harder, for instance).
- fauigerzigerk 4y agoThe problem with the blub argument is that it heavily relies on the existence of an undisputed power hierarchy among programming languages. Only s/he who is an expert in the most powerful language is in a position to pass judgement on all other languages. I don't know how to resolve disputes over the relative power of programming languages on that basis. Are dynamic languages more or less powerful than static languages with advanced type systems? I don't know. What's the definition of power? But I think what we can take from blub is that familiarity with one language or set of features is an insufficient basis for dismissing other, unfamiliar, languages or features.
- anyfoo 4y ago> those functions of type 'a->'a Polymorphism. But note that there is only one function of type 'a->'a for all 'a, and that's the identity function.
- rachitnigam 4y agoNot quite. This also has the type ‘a -> ‘a: def nitpick(x): throw “foo”
- deleted 4y ago[deleted]
- anyfoo 4y agoYou're right, and foo x = foo x also qualifies as 'a -> 'a. The only function that always terminates is the identity function, then.
- tsimionescu 4y agoWe could throw some unsafePerformIO in there and do anything else as well, but if course we would have a function in the mathematical sense anymore.
- xigoi 4y agoA function in the mathematical sense is total (always terminates).
- VirusNewbie 4y agothat's not a function (in the pedantic sense). technically a partial function, or you could say that every function is really of the type A -> Exception \/ A
- Cobord 4y agoBut you should not be writing functions that make you think about the fact that Hask is not really a category.
- School-Cotton 4y agohttps://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=cb69809c31d60f8abeb04e14bd952db1 https://play.rust-lang.org/?version=stable&mode=debug&editio... Took me five minutes, not four days…
- dataangel 4y agoWriting a generic function to sum an array of any numbers type is trivial in C++. There's no 4 day research period, even if you're new to templates. template<class T> T sum(T* x, size_t size) { T acc = 0; for(size_t i = 0; i < size; ++i) acc += x[i]; return acc; }
- xigoi 4y agofunc sum[Number](nums: openArray[Number]): Number = for num in nums: result += num Took about a minute.
- glmdev 4y agoI think Go's generics are reasonable, but not quite powerful enough to scale. It's all well and good to generalize basic containers and functions (good, in fact), but I wish the language was better at inferring types. I'm sure this is something that will improve with time, but a bit after generics were released, I tried to build a type-safe language evaluator in Go using generics and found them lacking the kind of type-narrowing necessary for fully-generic architecture. Short write-up of my conundrum on SO, if anyone is interested: https://stackoverflow.com/questions/71955121/how-to-type-switch-values-with-generic-types https://stackoverflow.com/questions/71955121/how-to-type-swi... TypeScript has me spoiled. :)
- bbkane 4y agoYes, I was trying to get it to infer the types for some code I was writing as well (a generic funcopt for two different types) and it couldn't - made me specify the type manually.
- nemothekid 4y agoI haven't looked into Go's actual implementation but isn't this kind of downcasted better suited for interfaces. Your code, as is, wouldn't translate to something like Rust which uses monomorphization; you are doing a runtime "inspection" of the types. This looks more in the domain of duck-typing than something that generics handles.
- SekstiNi 4y agoHere's a writeup exploring this very topic, and showing how Go's generics will currently give worse performance than just using an interface. https://planetscale.com/blog/generics-can-make-your-go-code-slower https://planetscale.com/blog/generics-can-make-your-go-code-...
- socialdemocrat 4y agoI think I am glad that kind of stuff doesn’t work, because that was pretty hard to read. Go is trying to keep things simple. If you want a sophisticated type system with a language they complied to native code then use Swift or Rust. I personally think it is good to have some choice in complexity level and that at least one language, Go, tries to carve out a niche in simplicity. I am only barely convinced that generics was the right choice for Go.
- avgcorrection 4y agoKey phrase in the article: > So, at least in this one instance, Yeah, one example which is not even that useful. It might be cherry-picked for all I know.
- lma21 4y agoIt's been a while since i've went through any Java code. Why wouldn't the Java code sample compile?
- saagarjha 4y agoJava primitive types are not objects :/
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- levoea 4y agoyou can't do `T summer = 0`, nor `summer += v[k]` as there is no arbitrary sum operation for Number (e.g a method `Number add(Number);` or `<T extends Number> T add(T t);`). though this can be solved either with types, as mentioned in another comment (https://news.ycombinator.com/item?id=32029871 https://news.ycombinator.com/item?id=32029871), or simply by providing the sum operation: static <T extends Number> T sum(T[] v, BinaryOperator<T> adder) { T summer = v[0]; for (int i = 1; i < v.length; i++) { summer = adder.apply(summer, v[i]); } return summer; } Float[] ft = { 0f, 1f, 2f, 3f }; // which can also be written as // `float z = sum(ft, Float::sum);` float z = sum(ft, (x, y) -> x + y);
- Cobord 4y ago{}
- EdwardDiego 4y agoBingo. You could use the longValue() or doubleValue() on Number to do math on if you insisted.
- saagarjha 4y agoThis is like the most cherry-picked example that could've been chosen. Java can't do this because the primitive types are not objects. Mentioning the performance of this is hilarious because 1. it's Daniel Lemire, he should know better and 2. the performance of pretty much every other thing that uses generics is terrible because of gcshapes being passed everywhere and 3. Java literally has a JIT to make generics fast. And bear in mind I don't even like Java generics that much, it's just that Go does even worse. (But still better than when it didn't have it at all, I guess…)
- Thaxll 4y agoHow can Go does it even worse that Java where generics implementation is the worse of all modern language with type erasure. Java is way worse than Go on that aspect.
- drdaeman 4y agoFor example, by not allowing one to create a parametrized method. `func (self SomeNonGenericType) Foo[T any](param T) T` is not allowed, best one can do is `func Foo[T any](self SomeNonGenericType, param T) T`. The reasons are explained here: https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#No-parameterized-methods https://go.googlesource.com/proposal/+/refs/heads/master/des... but the point is, you can have this in Java, but not in Go.
- thegeekpirate 4y agoYou can certainly use generic parameters in methods—the [T any] just has to be on SomeNonGenericType, even if it's not used inside the type itself https://rakyll.org/generics-facilititators https://rakyll.org/generics-facilititators
- drdaeman 4y agoYes, but that's completely different thing. If you move T you would change the semantics. - func (self SomeType[T]) Foo(...) is a method for a class parametrized on T. So if you want foo.Foo(1) then it only works for foo that's SomeType[int]. You cannot then easily call foo.Foo("one") on the same instance. - func (self SomeType) Foo[T any](...) - if that'd be a thing - would be a generic method on SomeType that works for any T. So you can foo.Foo(1) and foo.Foo("one") on the very next line and that would work.
- ClumsyPilot 4y agoaha, so Go has some kind of generics now, so since it has become a real programming language I might actually give it a spin! /s Never understoon why Generics where subject of a holywar in go community. It's an important nad basic feature these days
- zer0-c00l 4y agothis article is hilarious: 1. go generics do this one thing 2. therefore they are good Go generics are better than no generics but they're still limited, and that frustrates me.
- deleted 4y ago[deleted]
- socialdemocrat 4y agoCould you give some concrete examples? Something which requires a type of Go generics but which cannot easily be solved in another way?
- tgv 4y agoThe article really doesn't claim to briefly explore more than precisely that one thing. People just jump on the title plus some weird hate for Go.
- moelf 4y ago"not bad" is a weird bar to cross for a language that: 1. decided generics is not needed 2. went on for a decade 3. implemented a version of it that is not performance-sensitive 3.5. $$ by google
- deleted 4y ago[deleted]
- google234123 4y agoNot sure how 3.5 follows.
- Beltalowda 4y agoAnd 1. is demonstrably false as has been stated a million times already.
- erik_seaberg 4y agoIf they had launched without supporting odd numbers, nobody would be saying “they always intended to add that later” in their defense.
- tsimionescu 4y agoThey demonstrably decided generics are not needed for Go at release, or for the following 5+ years, I don't know how that could be seen as false. If they had thought generics are required for Go, they wouldn't have released the language before they had them. What is often claimed instead, and is indeed false, is that they believed generics should not exist in Go. This is a position that many Go proponents take, but the designers of the language never did.
- kgeist 4y ago>decided generics is not needed >went on for a decade The FAQ on the official site already back in 2013 (I couldn't find an earlier snapshot on webarchive) stated they were open to adding generics but weren't sure how to properly design/implement them without overcomplicating the language, citing also other priorities. >implemented a version of it that is not performance-sensitive It's less performant than the ideal but it's a tradeoff to avoid exponential code bloat found in C++. In our large C++ project which extensively used templated boost.signals the code bloat from using templates alone added a whopping 200 MB of machine code which we were able to shed off by switching to our in-house library. IIRC C# uses an approach similar to Go to share generated code with some type-specific conditions at runtime (at least Mono I remember shared generic code for all reference types)
- bouncycastle 4y agoThe site gives assembly code of the inner loop, but isn't the problem that there is additional overhead on the calling of generic functions?
- wahern 4y agoAFAIU, as compared to pure monomorphization, yes, sometimes the model imposes runtime indirection. This is particularly the case for methods. However, for simple yet common cases, such as functions taking []T, the model permits inlining into the caller.
- throwaway894345 4y agoI’m also hoping that the more monomorphization will take place over time.
- mcronce 4y agoGo's issues are mostly not performance related, so more monomorphization will really not solve a significant portion of them.
- throwaway894345 4y agoThey aren’t performance related because idiomatic Go pushes people toward performance. Generics currently push people away from performance (of course there are exceptions). I don’t expect it will be a huge deal either way, but it would be nice not to have to choose between performance and expressiveness.
- chrchang523 4y agoI believe Lemire's point is that the compiler is still able to inline the function despite the generic definition, so at least in this somewhat narrow case generics are a zero-cost abstraction.
- wyuenho 4y agoThe title says Go generics are not bad, and then in less than a page this guy only compared one use case with Java's generics. I'd expect a CS professor to be able to add a little more rigor in a blog post but I guess everyone is losing patience to read and write these days, even for the supposedly erudite.
- liuliu 4y agoThis is his blog, he can write whatever he wants. It is not a peer reviewed paper or even arXiv. Someone submitted and people liked it, and it ends up in HN front page. All I am saying, please don't put pressure on random blog writers. You can criticize the content without saying "CS professor can be more rigor".
- wyuenho 4y agoI only expected a CS professor to discover some of issues listed here: https://go101.org/generics/888-the-status-quo-of-go-custom-generics.html https://go101.org/generics/888-the-status-quo-of-go-custom-g... I mean, my cousin still in high school can do better than that...
- deleted 4y ago[deleted]
- bborud 4y agoIf someone doesn't know how to address an audience you can hardly blame the audience for not responding. Academic papers are written to further academic careers. Their audience is not developers, but other people who are also not developers. Of course there is going to be an impedance mismatch. What makes people important, effective and useful isn't what silly snobbery they use to put other people down, but whether they can effectively communicate their ideas to the audience.
- webkike 4y ago> so far I am giving go generics an A I guess I’ll wait until the full review then
- throwaway234232 4y agoI think in Java, you would need to do a type-class approach. interface Numeric<T> { T zero(); T add(T a, T b); } static <T> T sum(Numeric<T> n, T[] v) { T summer = n.zero(); for (int k = 0; k < v.length; k++) { summer = n.add(summer, v[k]); } return summer; }
- Avlin67 4y agoalso in .Net but then you enter a generic hell and you start thinking vanilla es6 is better
- nine_k 4y agoAFAICT this is basically how numbers work in Haskell, and I see no particular problems with them. You usually don't need too many number types of different nature.
- maxbond 4y agoGenerics are great when you need them, but they will cut you badly if you misuse them. Generics are viral. When you make something generic, you often have to make the things that touch or contain it generic, too. Generics also create tight coupling. When you change the definition of a generic interface/class, you'll need to update your usage across the codebase. As opposed to, say, adding a new field to a class, that can be safely ignored anywhere it isn't used. When this component you're updating is highly connected to other parts of your code, perhaps add another generic parameter to it, it completely explodes and you have to jump all around your codebase adding generics. The kicker is that you may be updating components which are themselves generic and highly connected, setting off secondary explosions. Pretty soon you're throwing that codebase out, starting over, and swearing to yourself that you'll never touch generics again. My advice is to assume generics are a premature abstraction until you've exhausted what you can do with more concrete approaches.
- rowanG077 4y ago
- Avlin67 4y agoproper generics would be C++ but it is called templates and could be hardcore.. https://stackoverflow.com/a/498329/729738 https://stackoverflow.com/a/498329/729738 .Net generics do have same issues mentioned in the article
- adgjlsfhk1 4y agoJulia gives you the same (but dynamic rather than static), but without the awful syntax.
- halter73 4y ago.NET 7 improves this with static abstract members in interfaces and generic math. https://github.com/dotnet/csharplang/issues/4436 https://github.com/dotnet/csharplang/issues/4436 https://devblogs.microsoft.com/dotnet/dotnet-7-generic-math/ https://devblogs.microsoft.com/dotnet/dotnet-7-generic-math/
- wyuenho 4y agoProper generics would be parametric polymorphism with full program type inference, i.e. an ML derivative.
- germandiago 4y agoC++ generic programming is super effective and efficient even if not super user-friendly til concepts landed. The good thing about Go generics is thatis has monomorphic generics for value types. That is fundamental. Java messed it up with type erasure and now they are wirh all this Valhalla project for years.
- tapirl 4y agoThis article is too simple to get a full picture of Go custom generics. There are actually both good points and bad points. To get a comprehensive understanding of the current Go custom generics, please read: https://go101.org/generics/101.html https://go101.org/generics/101.html
- deleted 4y ago[deleted]
- silisili 4y agoYour statement may be correct, and your book may be excellent. But insulting a free knowledge sharing article while posting your own site which is laden with 'buy buy buy' is rather poor taste, IMO.
- tapirl 4y agoYou don't need to buy it. It is free for reading online, and most contents in my website are free. "insulting"? That is a too heavy word.
- silisili 4y agoI rather agree, and must apologize as English seems to be slipping from me lately. I spent some time trying to think of the right word, but ended on insult, debating whether that was too negative of a word to use. Perhaps dismissive would have been better?
- tapirl 4y agoIt is just too simple to draw a conclusion. And IMHO, it is not very generics related.
- danielscrubs 4y agoEither you start making a language from a sound theoretical foundation and then implement it (Koka springs to mind) or you implement a language from some syntactic gripes and then try to find the theoretical foundation later… Go was impressive in the practical sense (compiler speed, channels in std and good tooling) but a shit show otherwise. I’ve completely lost any confidence that FANG will be able to make some great language because they always take the easy way (makes more business sense, see Dart). Academia on the other hand will make great languages that never reaches critical mass.
- truffdog 4y agoIan Lance Taylor worked on generics for like a decade, and the final design was heavily based on the Featherweight Go paper, which had formal proofs of the design given a simplified version of the go language. This is a very odd complaint around this feature.
- grumpyprole 4y agoYes retrofitting is hard. However, "Generics", also known as parametric polymorphism is well understood and goes back to Milners ML from the 70s.
- bborud 4y agoWhile something may be well understood in an academic sense doesn't mean that the application of that understanding to implementation humans will take a liking to is well understood. If you look at the programming language landscape I think one wouldn't be unreasonable in pointing out that very little is understood in terms of how you go from good ideas to widely adopted language. What affects reality is important.
- grumpyprole 4y ago> doesn't mean that the application of that understanding to implementation humans will take a liking to is well understood. I really like how ML implements it, does that make me somehow less human?
- zaptheimpaler 4y agoThis seems very basic. Not to dunk on prof. Lemire, i know he has some great posts. This one is just barely scratching the surface of what makes a language good at supporting polymorphism or not.
- deleted 4y ago[deleted]
- morelisp 4y agoIf I was a researcher focused on developing efficient algorithms for numerical data this example would also seem to be one of the most critical for my work.
- cookiengineer 4y agoWhat I still don't understand is why golang has exceptions for "language constructs" like make() and append()... while those are unimplementable in golang. It's pretty annoying that golang embraces static and functional patterns in its core, but it doesn't even have map/filter/reduce/forEach as generic implementations for all data types. The lack of a "new" keyword and default values for struct's property syntax is also pretty annoying. Let alone the "hack" with annotations for decoding/encoding everywhere, just so that you can use reflect API in the encoder implementations. For loop fatigue in golang makes it so unreadable in practice, and leads everybody to offloading manipulation of arrays/slices to helper methods, just to have the for loop there. The library that fixes this as good as possible with generics is the "lo" library (inspired by lodash) [1]. It's worth taking a look at it if you're curious how to implement it for your structs. [1] https://github.com/samber/lo https://github.com/samber/lo
- hsn915 4y agoWhat is for loop fatigue? Seems like a fake problem that one would only feel if they buy too much into functional programming. Some ideas in fuctional programming are useful, but overall, for loops are much more preferable to map/filter/reduce. For loops are much more easily amenable to incremental changes. "reduce" is particularly bad. It obfuscates control flow for no benefit at all.
- RhodesianHunter 4y ago>but overall, for loops are much more preferable to map/filter/reduce I strongly disagree on this one. A .map or a .filter are clear and concise while filtering or mapping, particularly in a streaming situation where you're only iterating the collection once. You can trivially write code using this paradigm that's easily readable, performant, and takes up a handful of lines that would take dozens in Go.
- throwaway894345 4y agoThere’s a whole lot wrong about this comment, but the most obvious is the claim that Go lacks a new keyword. Firstly, Go has a new keyword and secondly who makes a big deal about whether or not a language has a new keyword?
- davidkuennen 4y agoUpgraded my production service from go 1.17 to 1.18 today. A nice 30% performance boost. Love go. It keeps on giving.
- goto11 4y agoNumeric types are weird in Java and .net. I'm more familiar with C#, but in C# adding two int's result an int, but adding two short's also result in an int. So addition is not polymorphic the way you would expect it to be. C# has a large number of numeric types, but below the hood it only supports arithmetic operations on three of them. The problem is not with the implementation of generics per se, the problems is numeric types in those languages is a really leaky abstraction.
- maffydub 4y agoI don't seem to be able to post on his blog (400 Bad Request), so posting here instead... Interestingly, it looks like Rust (and C via clang) both generate something like the following as their one-at-a-time main loop: .LBB5_14: addl (%rcx), %eax addq $4, %rcx cmpq %rdx, %rcx jne .LBB5_14 It's one fewer instruction than the Go version (because it has combined the MOV and the ADD), although I don't know whether that would actually result in higher performance. I say their "one-at-a-time" main loop, because they actually seem to generate lots of code to try to take advantage of AVX or AVX2 to add 4 or 8 values at a time if available and the array is long enough! It might be cheating, but summing an iterator over numbers is built-in to Rust, so summing an array is just: v.iter().sum() The fun thing with this is that it's not just generic over number types, it's also generic over any type of iterator, so I can even use it to sum over the values in a HashMap: hashmap.values().sum()