7 ms·

Collection Pipeline

39 points by chillax 12y ago

deleted 12y ago
[deleted]
rikkus 12y ago
Half my code looks like this, these days: return products .Where(p => p.Type == ProductType.Bean) .OrderBy(p => p.Group) .ThenBy(p => p.Name) .Select(p => p.Name) ; Sometimes I'll write it like this: return from p in Products where p.ProductType == ProductType.Bean orderby ... etc. Everything's a query, or a sh pipeline, or... One of the nice things about this idiom is that it's supported all over the place, so you can write similar code in many different languages, for many different platforms.
- balakk 12y ago
  Here's where F# shines - you could write your query like: products |> Seq.filter(fun a -> a.Type = ProductType.Bean) |> Seq.sortBy(fun a -> a.Group, a.Name) |> Seq.map(fun a -> a.Name) It doesn't save a lot of lines here, but the big difference is this: suddenly you aren't constrained by the syntax of the single Linq query. So you want an additional filter,join,merge happening in between? No problem!
  - virtualwhys 12y ago
    Pretty noisy. Instead of: products |> Seq.filter(fun a -> a.foo =...) In another language you can do: products.filter(_.foo =...) Those `|> Seq...(fun a ->` all over the place look Java-ish ;-)
    - CmonDev 12y ago
      They had to deal with the OCAML baggage instead of starting from scratch.
- pekk 12y ago
  It is something you can do. It's fun to write. But is it better? Why?
CoffeeDregs 12y ago
I've generally appreciated Fowler's work on cataloging software patterns (and based my MBA thesis on doing something similar for organizational design), but I find this post a bit puzzling. Having spent a bunch of time with Haskell 6-7 years ago, the treatment of functions such as these was much more rigorous and generalized (e.g. http://en.wikibooks.org/wiki/Haskell/The_Functor_class http://en.wikibooks.org/wiki/Haskell/The_Functor_class). However the items this time are generic collection types themselves - where an OO collection pipeline would use objects, a functional language would use a hashmap. Again, I'm a bit confused (though I might have missed some context). Functional languages tend to be much more generic so that a "collection pipeline" or map can operate over a wide range of types. I'm reasonably certain that I'm missing some critical point, but I'm not sure about what?
- sparkie 12y ago
  > cataloging software patterns Otherwise known as inventing new terms for old concepts. Please stop doing this Mr Fowler! The programmer's glossary is already full of redundancy. Just stick to the language everyone else is using (ie, the language that was used by whoever wrote the original research papers in the given topics)
  - Dewie 12y ago
    But, we need to rename things like Functor to Mappable and Monad to Bindable because everything has to be a vaguely relatable (even if possibly misleading) adjective, damnit.
platz 12y ago
Not really a mention of whether these 'chained' methods should be returning a new collection, mutating an existing collection, or returning something completely different. Would expect to at least see some kind of discussion on this here - otherwise every method call is a "pipeline"
- munificent 12y ago
  From the article: > ...returns a collection of only those articles for which the lambda resolves as true. > Like select it returns a new collection so I can continue the pipeline... > ...return a collection of records. > Each operation takes a collection as an input and emits another collection (expect the last operation, which may be a terminal that emits a single value).
jgrodziski 12y ago
I would have liked to have seen Martin Fowler mentioned the parallelism and state implication of the unix pipeline, somewhat limited (see http://www.quora.com/Parallel-Computing/Do-pipelined-unix-commands-run-faster-on-multicore http://www.quora.com/Parallel-Computing/Do-pipelined-unix-co...) but this is for me a very profond way of looking at this kind of pattern: whether you finish completely your processing on the data structure or you emit the result downstream as soon as you have finished processing one element. Stream is for me the fundamental nature of the unix pipeline, and this is rarely met in programming language (channels to the rescue like in Clojure and Go). The Java Streams seems to work this way with the distinction between stateless and stateful processing element (http://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html http://docs.oracle.com/javase/8/docs/api/java/util/stream/pa...).

39 points by chillax 12y ago

deleted 12y ago

[deleted]

Half my code looks like this, these days: return products .Where(p => p.Type == ProductType.Bean) .OrderBy(p => p.Group) .ThenBy(p => p.Name) .Select(p => p.Name) ; Sometimes I'll write it like this: return from p in Products where p.ProductType == ProductType.Bean orderby ... etc. Everything's a query, or a sh pipeline, or... One of the nice things about this idiom is that it's supported all over the place, so you can write similar code in many different languages, for many different platforms.

balakk 12y ago
Here's where F# shines - you could write your query like: products |> Seq.filter(fun a -> a.Type = ProductType.Bean) |> Seq.sortBy(fun a -> a.Group, a.Name) |> Seq.map(fun a -> a.Name) It doesn't save a lot of lines here, but the big difference is this: suddenly you aren't constrained by the syntax of the single Linq query. So you want an additional filter,join,merge happening in between? No problem!
- virtualwhys 12y ago
  Pretty noisy. Instead of: products |> Seq.filter(fun a -> a.foo =...) In another language you can do: products.filter(_.foo =...) Those `|> Seq...(fun a ->` all over the place look Java-ish ;-)
  - CmonDev 12y ago
    They had to deal with the OCAML baggage instead of starting from scratch.
pekk 12y ago
It is something you can do. It's fun to write. But is it better? Why?

CoffeeDregs 12y ago

I've generally appreciated Fowler's work on cataloging software patterns (and based my MBA thesis on doing something similar for organizational design), but I find this post a bit puzzling. Having spent a bunch of time with Haskell 6-7 years ago, the treatment of functions such as these was much more rigorous and generalized (e.g. http://en.wikibooks.org/wiki/Haskell/The_Functor_class http://en.wikibooks.org/wiki/Haskell/The_Functor_class). However the items this time are generic collection types themselves - where an OO collection pipeline would use objects, a functional language would use a hashmap. Again, I'm a bit confused (though I might have missed some context). Functional languages tend to be much more generic so that a "collection pipeline" or map can operate over a wide range of types. I'm reasonably certain that I'm missing some critical point, but I'm not sure about what?

sparkie 12y ago
> cataloging software patterns Otherwise known as inventing new terms for old concepts. Please stop doing this Mr Fowler! The programmer's glossary is already full of redundancy. Just stick to the language everyone else is using (ie, the language that was used by whoever wrote the original research papers in the given topics)
- Dewie 12y ago
  But, we need to rename things like Functor to Mappable and Monad to Bindable because everything has to be a vaguely relatable (even if possibly misleading) adjective, damnit.

platz 12y ago

Not really a mention of whether these 'chained' methods should be returning a new collection, mutating an existing collection, or returning something completely different. Would expect to at least see some kind of discussion on this here - otherwise every method call is a "pipeline"

munificent 12y ago
From the article: > ...returns a collection of only those articles for which the lambda resolves as true. > Like select it returns a new collection so I can continue the pipeline... > ...return a collection of records. > Each operation takes a collection as an input and emits another collection (expect the last operation, which may be a terminal that emits a single value).

jgrodziski 12y ago

I would have liked to have seen Martin Fowler mentioned the parallelism and state implication of the unix pipeline, somewhat limited (see http://www.quora.com/Parallel-Computing/Do-pipelined-unix-commands-run-faster-on-multicore http://www.quora.com/Parallel-Computing/Do-pipelined-unix-co...) but this is for me a very profond way of looking at this kind of pattern: whether you finish completely your processing on the data structure or you emit the result downstream as soon as you have finished processing one element. Stream is for me the fundamental nature of the unix pipeline, and this is rarely met in programming language (channels to the rescue like in Clojure and Go). The Java Streams seems to work this way with the distinction between stateless and stateful processing element (http://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html http://docs.oracle.com/javase/8/docs/api/java/util/stream/pa...).