6 ms·

Did Martin really need to invent the important-sounding term "collection pipeline" for this?

aikah 11y ago
well, it takes what it takes to sell consultancy.You have to be a bit "creative", make the old stuff new, marketing is at the center of any consultancy /s I mean that's how things work, you don't have to invent anything, just make up some new expressions.
- deleted 11y ago
  [deleted]
- Afton 11y ago
  > I mean that's how things work, you don't have to invent anything, just make up some new expressions. Let's be somewhat more charitable, and say "you don't have to invent anything, just provide an insight". Which is pretty reasonably, I think.
  - aikah 11y ago
    agreed.
zenogais 11y ago
Agreed. The real title should be "Refactor loops with map, filter, and reduce"
- nilliams 11y ago
  There are languages that have map, filter and reduce as built-ins but aren't very accommodating of -- or discourage 'fluid interfaces' that allow the piping style in his examples. E.g. in python, you can't really 'pipe' to compose its filter and map built-ins, you have to do: map(lambda x: x * 2, filter(lambda x: x % 2 != 0, [1, 2, 3])) (Or use list comprehensions which also are not pipe-like). He's also specifically talking about 'collections' as in 'lists of dicts' in Python parlance, or 'arrays of objects' in other languages. The idea that you pass this 'collection' through a 'pipe' of operations is pretty powerful and obviously maps naturally to query-like behaviour. So to give Python back some dues, the place where you do of course see some 'collection piping' is when using the Django ORM. So yeah it may sound a bit grandiose but I think he's highlighting those terms on purpose because it is about more than just map, filter and reduce.
  - mdpopescu 11y ago
    Thanks for this; I am starting Python coming from C# and I kept looking for LINQ equivalents. Apparently I can stop looking, it was driving me nuts :)
    - deleted 11y ago
      [deleted]
    - ackalker 11y ago
      How about this one? mcoll - Modern API for python collections: https://github.com/nvbn/mcoll https://github.com/nvbn/mcoll
      - nilliams 11y ago
        That looks pretty neat, thanks! I'd only previously come across underscore.py [0] (python port of underscore.js) which looked a little unmaintained though perhaps because it's 'done'. [0] https://github.com/serkanyersen/underscore.py https://github.com/serkanyersen/underscore.py
    - shill 11y ago
      It's not using pipes, but you can refactor the example with this Python list comprehension. def filter_authors(authors, company): return [author['twitter_handle'] for author in authors if author['company'] == company and author['twitter_handle']] EDIT: HN code formatting seems to be broken so you may need to scroll right to see the entire comprehension. You can do other SQL-like operations with list comprehensions too. http://code.activestate.com/recipes/159974-sql-like-set-operations-with-list-comprehension-on/ http://code.activestate.com/recipes/159974-sql-like-set-oper...
prezjordan 11y ago
The term is older than this article. http://martinfowler.com/articles/collection-pipeline/ http://martinfowler.com/articles/collection-pipeline/ I also don't think he invented the term. A quick google search shows it's connected with LINQ.
- bshimmin 11y ago
  I don't wish to be argumentative, but I googled the term before I posted, too, and I don't see many references to it that predate Fowler's first article on the subject, or, indeed, many articles that don't actually credit Fowler for the term. Maybe he didn't invent the term, I don't know. It just feels to me like it's two words which, when used in combination, don't really add a whole lot of useful meaning to something that many of us will have been using for quite some years.
  - cgopalan 11y ago
    I concur. I have noticed that people that initially come into the industry as "object bigots" (Fowler's phrase to describe himself, not mine) end up discovering patterns inherent in functional programming or other paradigms and declaring them as novel ways to do stuff in the object world or otherwise. Its probably not optimal to keep adding terms to what is essentially one single concept.
skybrian 11y ago
This is what design patterns are all about. You take some common practice, give it a name (or pick a name from commonly used names), and write documentation clearly describing it. Then we can talk about about more easily. Maybe you don't like the name, but there's nothing unusual about this.

aikah 11y ago

well, it takes what it takes to sell consultancy.You have to be a bit "creative", make the old stuff new, marketing is at the center of any consultancy /s I mean that's how things work, you don't have to invent anything, just make up some new expressions.

deleted 11y ago
[deleted]
Afton 11y ago
> I mean that's how things work, you don't have to invent anything, just make up some new expressions. Let's be somewhat more charitable, and say "you don't have to invent anything, just provide an insight". Which is pretty reasonably, I think.
- aikah 11y ago
  agreed.

zenogais 11y ago

Agreed. The real title should be "Refactor loops with map, filter, and reduce"

nilliams 11y ago
There are languages that have map, filter and reduce as built-ins but aren't very accommodating of -- or discourage 'fluid interfaces' that allow the piping style in his examples. E.g. in python, you can't really 'pipe' to compose its filter and map built-ins, you have to do: map(lambda x: x * 2, filter(lambda x: x % 2 != 0, [1, 2, 3])) (Or use list comprehensions which also are not pipe-like). He's also specifically talking about 'collections' as in 'lists of dicts' in Python parlance, or 'arrays of objects' in other languages. The idea that you pass this 'collection' through a 'pipe' of operations is pretty powerful and obviously maps naturally to query-like behaviour. So to give Python back some dues, the place where you do of course see some 'collection piping' is when using the Django ORM. So yeah it may sound a bit grandiose but I think he's highlighting those terms on purpose because it is about more than just map, filter and reduce.
- mdpopescu 11y ago
  Thanks for this; I am starting Python coming from C# and I kept looking for LINQ equivalents. Apparently I can stop looking, it was driving me nuts :)
  - deleted 11y ago
    [deleted]
  - ackalker 11y ago
    How about this one? mcoll - Modern API for python collections: https://github.com/nvbn/mcoll https://github.com/nvbn/mcoll
    - nilliams 11y ago
      That looks pretty neat, thanks! I'd only previously come across underscore.py [0] (python port of underscore.js) which looked a little unmaintained though perhaps because it's 'done'. [0] https://github.com/serkanyersen/underscore.py https://github.com/serkanyersen/underscore.py
  - shill 11y ago
    It's not using pipes, but you can refactor the example with this Python list comprehension. def filter_authors(authors, company): return [author['twitter_handle'] for author in authors if author['company'] == company and author['twitter_handle']] EDIT: HN code formatting seems to be broken so you may need to scroll right to see the entire comprehension. You can do other SQL-like operations with list comprehensions too. http://code.activestate.com/recipes/159974-sql-like-set-operations-with-list-comprehension-on/ http://code.activestate.com/recipes/159974-sql-like-set-oper...

prezjordan 11y ago

The term is older than this article. http://martinfowler.com/articles/collection-pipeline/ http://martinfowler.com/articles/collection-pipeline/ I also don't think he invented the term. A quick google search shows it's connected with LINQ.

bshimmin 11y ago
I don't wish to be argumentative, but I googled the term before I posted, too, and I don't see many references to it that predate Fowler's first article on the subject, or, indeed, many articles that don't actually credit Fowler for the term. Maybe he didn't invent the term, I don't know. It just feels to me like it's two words which, when used in combination, don't really add a whole lot of useful meaning to something that many of us will have been using for quite some years.
- cgopalan 11y ago
  I concur. I have noticed that people that initially come into the industry as "object bigots" (Fowler's phrase to describe himself, not mine) end up discovering patterns inherent in functional programming or other paradigms and declaring them as novel ways to do stuff in the object world or otherwise. Its probably not optimal to keep adding terms to what is essentially one single concept.

skybrian 11y ago

This is what design patterns are all about. You take some common practice, give it a name (or pick a name from commonly used names), and write documentation clearly describing it. Then we can talk about about more easily. Maybe you don't like the name, but there's nothing unusual about this.