6 ms·

Go Chainable: .map().filter().reduce() in Go

50 points by davidashe 5y ago

davidashe 5y ago
Now that generics are in beta, here's a library for those who want ergonomic slice/map wrappers that make chainable operations easy. Or, a gateway drug for ECMAscripters who can't drop their `.map(x => y).filter(x => y).reduce(x => y)` habit.
- eyelidlessness 5y ago
  Yep, nobody pipes data through any processing in say Haskell or Clojure.
- mushyhammer 5y ago
  That habit makes zero sense. Reduce already is a map and filter and one, why do you need to loop it 3 times? Reduce itself is awful to begin with. https://twitter.com/jaffathecake/status/1213077702300852224 https://twitter.com/jaffathecake/status/1213077702300852224
  - dragonwriter 5y ago
    it's a readable, maintainable composable functional pattern that unfortunately scales badly in JS because the methods are all eager methods; there are libraries with lazy versions that return generators so that chaining them produces a pipeline that does one loop rather than one per chained operation, though, which most sensible languages do, or at least support, out of the box. Still, if you know you’ll have a small working set, and you don't what the extra deependency, there's lots of cases where bet readability, composability, and maintainability wins over efficiency.
  - Zababa 5y ago
    > Reduce already is a map and filter and one, why do you need to loop it 3 times? Because being explicit about intent is important. Map means "transforming each element of the collection", filter means "keeping only some elements of the collection". With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read. You don't have to spend energy to understand if the loop is only a transformation, or also filtering. Since you chain them, you can easily isolate each transformation, which makes code easier to read and easier to test. > Reduce itself is awful to begin with. I don't really understand what's supposed to be awful about reduce. It's just another form of for loop. I don't see people arguing that for loops are awful. The thread you linked is just someone that's biased against reduce for some reason that I don't quite understand. Reduce makes explicit that when using a for loop and accumulating a result, you need both a base case, and what to do when going from n to n+1. Again, this allows you to isolate the "going from n to n+1" part easily. Recursion itself is great for some problems.
    - mushyhammer 5y ago
      > what to do when going from n to n+1 You mention the only worthwhile exception: sums and other 3-character operation. If you’re using reduce to construct objects you’re just better off with a for loop. Having a random awkward named function that only makes sense when plugged into reduce is not any better than just have a function that includes the loop itself: - newArray = array.reduce(operation, []) + newArray = operation(array) If you’re used to having infinite chains of loops then of course that’s not going to work. The problem is that reduce can do anything and most people use it to do anything. It has no advantage over regular for loops unless you’re proficient with real functional programming (if you use forEach() in JavaScript, you’re not) > With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read. Filter and map are ok. Filter, map and reduce are not, at least not in JavaScript. Try writing the equivalent for loop and you’ll find that it’s just as simple to follow.
ayastrebov 5y ago
https://github.com/AlexanderYastrebov/stream https://github.com/AlexanderYastrebov/stream is my take on this that does not re-allocate full slice on every filter/map operation
adam_arthur 5y ago
Are there people with experience in a wide variety of languages that prefer Go? I've only used it in passing, but everytime I see examples they're verbose and look clunky. For example, the chainable methods are nice, but comparing to JS looks more like ES5 than modern code. Do you find Go preferable to other languages for solo projects?
- mohanmcgeek 5y ago
  People who prefer golang use it _because_ it is verbose. Nothing is implicit and that makes good easy to read.
  - ironmagma 5y ago
    There are definitely people who prefer to use Go who think the verbosity is a downside. Someone had to craft the proposal for generics after all.
  - throw_m239339 5y ago
    > People who prefer golang use it _because_ it is verbose. I don't prefer Go because it is verbose, error as values without constructs to manage them is a pain in the ass. I'd use go over language X,Y or Z because of its standard library and ease of deployment. The language itself has great things but is verbosity, mainly when it comes to dealing with error C style, its absolutely not a feature. Now that we have generics, things will get more interesting. > Nothing is implicit and that makes good easy to read. I mean this is a language with garbage collection, that thing certainly is implicit.
    - mindwok 5y ago
      For me, it is a feature. Coming from languages with exceptions, where you have no idea if a function call is going to explode without praying that the library has its exceptions documented or reading the source code, having error values that you cannot (easily) ignore is a blessing. It's not as good as something like Rust's Result sum type, but it's pretty close. It makes you explicitly handle, or bubble up, every error in your program which in my experience is a huge cause of unexpected exits in other languages.
chrismorgan 5y ago
Hmm. Implemented on a custom type that wraps []T, so you have to create a List to get the methods, meaning more boilerplate (a common theme in Go); eager, like JavaScript’s Array methods rather than like the iterator methods in Rust or most/all functional programming languages; and since methods can’t have generics (which really surprised me in the Go generics proposal), Map() can only work on the same type (T → T, rather than T → U), and Reduce()’s output type is a generic parameter on the list, so you can only ever reduce to one type (unless you deconstruct and reconstruct the List) which must be specified at instantiation time. As one who works mostly in Rust and JavaScript and is passingly familiar with Go (I used it for a couple of projects eight and nine years ago), these seem some pretty severe limitations. Rust’s trait-based iterator system is delightful, so that you can map, filter, reduce, &c. on any iterator, lazily, and even define an extension trait to define your own methods on any iterator, thereafter accessible by just importing that trait. In the end, I think the current scope of generics and interfaces won’t be enough to produce any feeling other than “shoehorned” for this functional style in Go. It’s just not a style that works well in all types of programming languages.
- gtramont 5y ago
  After taking a stab at some more functional approach with generics in Go — https://github.com/gtramontina/go-extlib https://github.com/gtramontina/go-extlib — I do agree that some of it does feel shoehorned. Although for some other constructs, it feels quite nice. As I mentioned on the readme of the linked repository, this post https://hypirion.com/musings/type-safe-http-servers-in-go-via-generics https://hypirion.com/musings/type-safe-http-servers-in-go-vi... presents pros and cons nicely.
- tptacek 5y ago
  I don't love libraries like this and feel like they work against the idiom in Go; I'm skeptical that things like this will be part of the idiom going forward. But Rust isn't all wine and roses with this stuff either; obviously, it's a much more powerful generics system, but closures are much more annoying to work with than they are in Go, where everything just magically escapes to the heap when you need it to.
svnpenn 5y ago
I hate this. I don't know why people insist on forcing this programming style into every language. Loops exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap. Not every program needs to be golfed down to one line.
- ironmagma 5y ago
  Then don’t use it? Bad code is written in every language, you don’t need a library to make that possible.
- deleted 5y ago
  [deleted]
- bobbylarrybobby 5y ago
  The point of these functions is to give names to common operations that happen in loops so that you can read the code more quickly. Map: we're transforming values. Filter: we're dropping values. Reduce: we're accumulating. Yes you can write these out explicitly in a loop but then you actually have to read all of the associated noise to grok what's happening. In addition, the method-based approach scopes the variables to each individual call so there's no risk of, say, transforming the loop variable but then accidentally filtering based on the original instead of the transformed.
- erik_seaberg 5y ago
  Goto was simple and fast and did anything. We replaced that with if/then/else/while/foreach/return/throw because each of those clearly expresses what we’re currently doing and especially what we aren’t.
- chlorion 5y ago
  You could very easily say the same thing about goto vs structured loops, for example: "I hate this. I don't know why people insist on forcing this programming style into every language. Goto and labels exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap." It's just a more structured way to express common operations that we use loops for, and it does has objective technological advantages over raw loops when implemented and used correctly. As for performance, that is going to vary language to language, but in general iterator processing pipelines are not slower and can even be faster than raw loops. Whether this is the case in Go, I have no idea. With all of that being said, I am not sure if I would use this style of programming in Go since it doesn't feel idiomatic (yet?).
avl999 5y ago
This actually became a problem in Java after the streams API was introduced. A bunch of people in my company started writing Java code that looked nothing like Java with a bunch of chained functional style functions with liberal use of the Optional API where trying to decipher what was actually being done required stopping in your tracks and using the help of the IDE to figure out what type was being returned by each level of the chain. I myself fell victim to this and wrote code which made me feel clever in the moment but looking at it a couple of days later even I couldn't understand my own code. If people want to write functional code they should use functional languages rather than writing non-idiomatic code in other languages. Thankfully this style is unlikely to gain popularity in Go because they make the syntax for doing this verbose enough and ugly enough that most people aren't gonna bother with it except a top level map or filter.
- jackling 5y ago
  I work with Java day to day. And I definitely fall victim to overuse of a functional style. Starting to think I shouldn't try to fit everything into a stream, but it's hard because that seems to be the general direction my company is going with their SDK. I do heavily prefer Optionals however when dealing with Values that can be null. Dealing with nullity checks is annoying.

50 points by davidashe 5y ago

Now that generics are in beta, here's a library for those who want ergonomic slice/map wrappers that make chainable operations easy. Or, a gateway drug for ECMAscripters who can't drop their `.map(x => y).filter(x => y).reduce(x => y)` habit.

eyelidlessness 5y ago
Yep, nobody pipes data through any processing in say Haskell or Clojure.
mushyhammer 5y ago
That habit makes zero sense. Reduce already is a map and filter and one, why do you need to loop it 3 times? Reduce itself is awful to begin with. https://twitter.com/jaffathecake/status/1213077702300852224 https://twitter.com/jaffathecake/status/1213077702300852224
- dragonwriter 5y ago
  it's a readable, maintainable composable functional pattern that unfortunately scales badly in JS because the methods are all eager methods; there are libraries with lazy versions that return generators so that chaining them produces a pipeline that does one loop rather than one per chained operation, though, which most sensible languages do, or at least support, out of the box. Still, if you know you’ll have a small working set, and you don't what the extra deependency, there's lots of cases where bet readability, composability, and maintainability wins over efficiency.
- Zababa 5y ago
  > Reduce already is a map and filter and one, why do you need to loop it 3 times? Because being explicit about intent is important. Map means "transforming each element of the collection", filter means "keeping only some elements of the collection". With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read. You don't have to spend energy to understand if the loop is only a transformation, or also filtering. Since you chain them, you can easily isolate each transformation, which makes code easier to read and easier to test. > Reduce itself is awful to begin with. I don't really understand what's supposed to be awful about reduce. It's just another form of for loop. I don't see people arguing that for loops are awful. The thread you linked is just someone that's biased against reduce for some reason that I don't quite understand. Reduce makes explicit that when using a for loop and accumulating a result, you need both a base case, and what to do when going from n to n+1. Again, this allows you to isolate the "going from n to n+1" part easily. Recursion itself is great for some problems.
  - mushyhammer 5y ago
    > what to do when going from n to n+1 You mention the only worthwhile exception: sums and other 3-character operation. If you’re using reduce to construct objects you’re just better off with a for loop. Having a random awkward named function that only makes sense when plugged into reduce is not any better than just have a function that includes the loop itself: - newArray = array.reduce(operation, []) + newArray = operation(array) If you’re used to having infinite chains of loops then of course that’s not going to work. The problem is that reduce can do anything and most people use it to do anything. It has no advantage over regular for loops unless you’re proficient with real functional programming (if you use forEach() in JavaScript, you’re not) > With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read. Filter and map are ok. Filter, map and reduce are not, at least not in JavaScript. Try writing the equivalent for loop and you’ll find that it’s just as simple to follow.

ayastrebov 5y ago

https://github.com/AlexanderYastrebov/stream https://github.com/AlexanderYastrebov/stream is my take on this that does not re-allocate full slice on every filter/map operation

adam_arthur 5y ago

Are there people with experience in a wide variety of languages that prefer Go? I've only used it in passing, but everytime I see examples they're verbose and look clunky. For example, the chainable methods are nice, but comparing to JS looks more like ES5 than modern code. Do you find Go preferable to other languages for solo projects?

mohanmcgeek 5y ago
People who prefer golang use it _because_ it is verbose. Nothing is implicit and that makes good easy to read.
- ironmagma 5y ago
  There are definitely people who prefer to use Go who think the verbosity is a downside. Someone had to craft the proposal for generics after all.
- throw_m239339 5y ago
  > People who prefer golang use it _because_ it is verbose. I don't prefer Go because it is verbose, error as values without constructs to manage them is a pain in the ass. I'd use go over language X,Y or Z because of its standard library and ease of deployment. The language itself has great things but is verbosity, mainly when it comes to dealing with error C style, its absolutely not a feature. Now that we have generics, things will get more interesting. > Nothing is implicit and that makes good easy to read. I mean this is a language with garbage collection, that thing certainly is implicit.
  - mindwok 5y ago
    For me, it is a feature. Coming from languages with exceptions, where you have no idea if a function call is going to explode without praying that the library has its exceptions documented or reading the source code, having error values that you cannot (easily) ignore is a blessing. It's not as good as something like Rust's Result sum type, but it's pretty close. It makes you explicitly handle, or bubble up, every error in your program which in my experience is a huge cause of unexpected exits in other languages.

chrismorgan 5y ago

Hmm. Implemented on a custom type that wraps []T, so you have to create a List to get the methods, meaning more boilerplate (a common theme in Go); eager, like JavaScript’s Array methods rather than like the iterator methods in Rust or most/all functional programming languages; and since methods can’t have generics (which really surprised me in the Go generics proposal), Map() can only work on the same type (T → T, rather than T → U), and Reduce()’s output type is a generic parameter on the list, so you can only ever reduce to one type (unless you deconstruct and reconstruct the List) which must be specified at instantiation time. As one who works mostly in Rust and JavaScript and is passingly familiar with Go (I used it for a couple of projects eight and nine years ago), these seem some pretty severe limitations. Rust’s trait-based iterator system is delightful, so that you can map, filter, reduce, &c. on any iterator, lazily, and even define an extension trait to define your own methods on any iterator, thereafter accessible by just importing that trait. In the end, I think the current scope of generics and interfaces won’t be enough to produce any feeling other than “shoehorned” for this functional style in Go. It’s just not a style that works well in all types of programming languages.

gtramont 5y ago
After taking a stab at some more functional approach with generics in Go — https://github.com/gtramontina/go-extlib https://github.com/gtramontina/go-extlib — I do agree that some of it does feel shoehorned. Although for some other constructs, it feels quite nice. As I mentioned on the readme of the linked repository, this post https://hypirion.com/musings/type-safe-http-servers-in-go-via-generics https://hypirion.com/musings/type-safe-http-servers-in-go-vi... presents pros and cons nicely.
tptacek 5y ago
I don't love libraries like this and feel like they work against the idiom in Go; I'm skeptical that things like this will be part of the idiom going forward. But Rust isn't all wine and roses with this stuff either; obviously, it's a much more powerful generics system, but closures are much more annoying to work with than they are in Go, where everything just magically escapes to the heap when you need it to.

svnpenn 5y ago

I hate this. I don't know why people insist on forcing this programming style into every language. Loops exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap. Not every program needs to be golfed down to one line.

ironmagma 5y ago
Then don’t use it? Bad code is written in every language, you don’t need a library to make that possible.
deleted 5y ago
[deleted]
bobbylarrybobby 5y ago
The point of these functions is to give names to common operations that happen in loops so that you can read the code more quickly. Map: we're transforming values. Filter: we're dropping values. Reduce: we're accumulating. Yes you can write these out explicitly in a loop but then you actually have to read all of the associated noise to grok what's happening. In addition, the method-based approach scopes the variables to each individual call so there's no risk of, say, transforming the loop variable but then accidentally filtering based on the original instead of the transformed.
erik_seaberg 5y ago
Goto was simple and fast and did anything. We replaced that with if/then/else/while/foreach/return/throw because each of those clearly expresses what we’re currently doing and especially what we aren’t.
chlorion 5y ago
You could very easily say the same thing about goto vs structured loops, for example: "I hate this. I don't know why people insist on forcing this programming style into every language. Goto and labels exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap." It's just a more structured way to express common operations that we use loops for, and it does has objective technological advantages over raw loops when implemented and used correctly. As for performance, that is going to vary language to language, but in general iterator processing pipelines are not slower and can even be faster than raw loops. Whether this is the case in Go, I have no idea. With all of that being said, I am not sure if I would use this style of programming in Go since it doesn't feel idiomatic (yet?).

avl999 5y ago

This actually became a problem in Java after the streams API was introduced. A bunch of people in my company started writing Java code that looked nothing like Java with a bunch of chained functional style functions with liberal use of the Optional API where trying to decipher what was actually being done required stopping in your tracks and using the help of the IDE to figure out what type was being returned by each level of the chain. I myself fell victim to this and wrote code which made me feel clever in the moment but looking at it a couple of days later even I couldn't understand my own code. If people want to write functional code they should use functional languages rather than writing non-idiomatic code in other languages. Thankfully this style is unlikely to gain popularity in Go because they make the syntax for doing this verbose enough and ugly enough that most people aren't gonna bother with it except a top level map or filter.

jackling 5y ago
I work with Java day to day. And I definitely fall victim to overuse of a functional style. Starting to think I shouldn't try to fit everything into a stream, but it's hard because that seems to be the general direction my company is going with their SDK. I do heavily prefer Optionals however when dealing with Values that can be null. Dealing with nullity checks is annoying.