5 ms·

A single, non-nested list comprehension or generator exp is basically map(filter). You need nesting to get filter(map). e.g. map(expensive_call, filter(co

by riobard 17y ago

A single, non-nested list comprehension or generator exp is basically map(filter). You need nesting to get filter(map). e.g. map(expensive_call, filter(cond, seq)) equals [expensive_call(each) for each in seq if cond(each)] but filter(cond, map(expensive_call, seq)) equals [each for each in [expensive_call(x) for x in seq] if cond(each)] note because of "expensive_call", it's inefficient (and silly) to do [expensive_call(each) for each in seq if cond(expensive_call(each))] So map/filter combination gives more flexibility than list comprehension, and for functional-thinking minds, it's just so natural to think in abstract terms of passing functions around. List comprehension is pretty syntactical sugar to do similar things, but it forces you think about the "how to do" instead of "what to do". That said, it's not really "compelling" though -- there is no real "compelling" reason to switch from one Turing-complete language to another given that you can do the same thing eventually. But hey, it's the itches that drive us nuts, isn't it? :) Just my 2 cents

amix 17y ago
Most times you are interested in doing the simple, e.g.: filtered = [x for x in seq if x>10] Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers :) Anyhow, I really like Guido's decision on dropping these - it creates a cleaner language and forces people to think Pythonic when programming in Python.
- riobard 17y ago
  > Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers For simple cases, yes. But I wouldn't say [each for each in [expensive_call(x) for x in seq] if cond(each)] is more readable than filter(cond, map(expensive_call, seq)) at least for functional-thinking minds. The level of thinking in abstract is different here. Now the problem is, some people see Python as a very functional language (with first-class functions etc) and want to use it that way (like Lisp), but BDFL and some core Python devs believe it is better to keep it Pythonic, thus those functional people are kinda pissed off by this and switch away from Python. Personally I don't think it will make Python a lot cleaner to remove two auxiliary functions and force people to use list comprehension when it is completely trivial to add these missing pair back (two lines of code). (disclosure: I prefer FP, but I also think keeping things Pythonic is fine most of the time. It's just that in this case, I think map/filter is pretty "Pythonic" according to me. :)
  - ramen 17y ago
    I know you all know this, but I feel like it bears mentioning that nobody is forcing you to use list comprehensions whether map/filter stay in Python's built-ins or not. They can be defined in around 3-4 lines of code each. Lisp aficionados, already accustomed to the bottom-up style of programming, ought to have no problem writing functions like these as necessary.
    - riobard 17y ago
      Yeah I know. Maybe "force" is not the right word ... probably "discourage"? Actually only one line of code is enough for each of map/filter: def map(fn, seq): return [fn(each) for each in seq] def filter(cond: seq): return [each for each in seq if cond(each)] But seriously, what do you really gain by removing these two? Isn't that too ideological? I don't really see how un-Pythonic it would be to use map/filter instead of list comprehensions. The problem is BDFL's attitude seems to drive many FP-ers away, like the guy in the original post.
      - kingkilr 17y ago
        They ultimately weren't removed. reduce was removed from the builtins, but it's still just one import away. From itertools import reduce.
    - ricree 17y ago
      If I remember correctly, "removing" these functions mostly just meant dumping them into the functools module rather than including them as a built in function.
    - holygoat 17y ago
      There's a difference between things being possible and being encouraged. De-emphasizing functional operations makes it more likely for libraries to work in a non-functional style, for tutorials to do so, etc. It's tiring fighting against a community and a (benevolent) dictator that disagree with you.
      - 17y ago
- azanar 17y ago
  [It] creates a cleaner language and forces people to think Pythonic when programming in Python. The question is whether "Pythonic," as the community defines it today, is optimal in all cases. As an analogue, there have been enough talks about things the Java community considers to be stylistically optimal that look really horrible compared to implementations in other languages -- like Python. :-) Pythonic style should be a guide, not an edict, and should be deviated from or redefined when it makes sense. The examples riobard provided already show the syntax weighing things down, a situation where syntax should give way to a more functional style approach. As for it creating a cleaner language, I try to approach this, as with all things, from the perspective of being an language-agnostic programmer. From that perspective, I do like the python syntax for simple things like what you defined, but under heavier weight mapping and filtering operations, the map/filter function call syntax seems a lot cleaner. There's nothing wrong with syntactic sugar, but I would assert that it will tend to suck when it is all you have.
- holygoat 17y ago
  Um... (filter #(> x 10) seq) sure is readable for me, and I'm fluent in Python and various Lisps. (That example is Clojure.) I would like to point out, however, that CL allows you to write: (loop for x in seq when (> x 10) collect x) which you might think is verbose ("why do I need that 'collect'?")... except that loop allows you to write things like (loop for i in *random* counting (evenp i) into evens counting (oddp i) into odds summing i into total maximizing i into max minimizing i into min finally (return (list min max total evens odds))) Loop knocks Python's trivial list comprehensions into a cocked hat. I switch between map/filter and loop depending on whether I'm working with predefined functions (e.g., (filter 'less-than-ten seq)), handling multiple sequences, doing side-effects, etc.
  - foldr 17y ago
    >Loop knocks Python's trivial list comprehensions into a cocked hat. CL could really use something like Python's generators though. The loop macro is an ugly hack in comparison.
yummyfajitas 17y ago
I don't get how map/filter is more flexible than list comprehensions. Map/filter require nesting: (filter (map expensive_call X) cond) So do list comprehensions: [y for y in [expensive_call(x) for x in X] if cond(y)] Near as I can tell, the only difference is that list comprehensions also provide a syntactic sugar for the convenience function filter_then_map. Sometimes this saves you a level of nesting, sometimes not. Incidentally, this isn't even an issue in a pure functional language with sufficiently smart compiler.
- riobard 17y ago
  Well, FP is mostly about nesting, but it's uniform and people get used to thinking that way. List comprehensions, on the other hand, are more like procedure code (conceptually), and it sucks to nest them. Nesting aside, it's the different level of abstraction that matters for FP-ers. Plus, you get 5 mentions (3 y's and 2 x's) of some intermediate variables instead of 0 in your code, so both token-wise and char-wise, map/filter alternative is shorter, and less a mental burden (think about the "succinct" idea by PG).
  - frig 17y ago
    I have a longer comment here that argues the opposite: list-comprehensions are only a syntactic pun or two away from set-builder notation, which is a higher level-of-abstraction (it declaratively states what it is) than map+filter (which specify a procedure to generate it, albeit at higher level of abstraction than a for-loop). If you can put together a nontrivial usage of map+filter with at least three source collections that's more concise than the equivalent list comprehension I'll (figuratively) eat my hat.
    - riobard 17y ago
      I'm not so sure if set notation is higher level, but for the example in your longer comment, map/filter/product is not that bad if you use it wisely. Here is my version: map(lambda (w,s,l): {'widget': w, 'sprocket': s, 'location': l} filter(lambda (w,s,l): l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s), product(widgets, sprockets, locations))) Well, I agree it is not any conciser that its list comprehension (about the same I guess?). Nothing is perfect, like you said, know you tool :)
frig 17y ago
It depends slightly on how you came to functional programming. You can arguably trace Python's list comprehension syntax all the way back to setl, a "set-theoretic programming language", and the resemblance to mathematical set-builder notation is intentional; compare - let B = { g(a) s.t. | a \in A and f(a) holds} - B = [g(a) for a in A if f(a)] If you're used to thinking in sets having to decompose into "maps" and "filters" is a speedbump; easy to do but nice to avoid. Where list comprehensions really start to shine is making it comparatively trivial to pull from multiple source collections without a lot of ugly machinery: [{'widget':w,'sprocket':s,'location':l} for w in widgets for s in sprockets for l in locations if l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s)] ...which is about where explicit map + filter start to become annoying. You can use: map(lambda i: {'widget':i[0], 'sprocket':i[1], 'location':i[2]}, filter(lambda i: i[2].hasInStock(i[0]) and i[2].hasInStock(i[1]) and i[0].isUsableWith(i[1]), itertools.product(widgets,sprockets,locations))) ...but to my eyes that is not only very ugly but just going by character count the # of characters given over to keywords (map, lambda, filter) instead of "what i'm doing here" is huge. Additionally use of itertools forces use of tuples for your intermediate values and thus the lambdas are gobbledegook until you get to the tail end of the statement and see that i[0] == widget, i[1] == sprocket, and i[2] == location. I could define some constants (WIDGET = 0, SPROCKET = 1, LOCATION = 2, etc) but now it's even longer. It gets even worse if you try to be clever with the sequence of operations. You might look at that definition and say lo! I can pre-filter out stuff not in stock at each location and make things more efficient. Naively you'd wind up with: map(lambda d: {'location':d[0], 'widget':d[1], 'sprocket':d[2]}, flatten(map(lambda l: list(filter(lambda p: p[1].isUsableWith(p[2]), itertools.product([l],filter(lambda w: l.hasInStock(w), widgets), filter(lambda s: l.hasInStock(s), sprockets)))), locations))) #nb must-supply-you-own-flatten-method Under some circumstances that might be substantially faster than the previous approach. But compare the equivalent but you could have instead gone with: [{'widget':w,'sprocket':s,'location':l} for l in locations for w in [widget for widget in widgets if l.hasInStock(widget)] for s in [sprocket for sprocket in sprockets if l.hasInStock(sprocket)] if w.isUsableWith(s)] So you lose a little flexibility in simple cases at in exchange for increasing the scope of what you can get away with as "readable" one-liners.
- riobard 17y ago
  Python's lambda sucks, but that does not necessarily mean map/filter sucks with it too. Of course there are cases where list comprehensions are more convenient and more powerful (esp. if they are more like their "real" counterparts in Haskell and friends). But in cases when map/filter are more convenient, I like the option of using them.

by riobard 17y ago

amix 17y ago

Most times you are interested in doing the simple, e.g.: filtered = [x for x in seq if x>10] Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers :) Anyhow, I really like Guido's decision on dropping these - it creates a cleaner language and forces people to think Pythonic when programming in Python.

riobard 17y ago
> Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers For simple cases, yes. But I wouldn't say [each for each in [expensive_call(x) for x in seq] if cond(each)] is more readable than filter(cond, map(expensive_call, seq)) at least for functional-thinking minds. The level of thinking in abstract is different here. Now the problem is, some people see Python as a very functional language (with first-class functions etc) and want to use it that way (like Lisp), but BDFL and some core Python devs believe it is better to keep it Pythonic, thus those functional people are kinda pissed off by this and switch away from Python. Personally I don't think it will make Python a lot cleaner to remove two auxiliary functions and force people to use list comprehension when it is completely trivial to add these missing pair back (two lines of code). (disclosure: I prefer FP, but I also think keeping things Pythonic is fine most of the time. It's just that in this case, I think map/filter is pretty "Pythonic" according to me. :)
- ramen 17y ago
  I know you all know this, but I feel like it bears mentioning that nobody is forcing you to use list comprehensions whether map/filter stay in Python's built-ins or not. They can be defined in around 3-4 lines of code each. Lisp aficionados, already accustomed to the bottom-up style of programming, ought to have no problem writing functions like these as necessary.
  - riobard 17y ago
    Yeah I know. Maybe "force" is not the right word ... probably "discourage"? Actually only one line of code is enough for each of map/filter: def map(fn, seq): return [fn(each) for each in seq] def filter(cond: seq): return [each for each in seq if cond(each)] But seriously, what do you really gain by removing these two? Isn't that too ideological? I don't really see how un-Pythonic it would be to use map/filter instead of list comprehensions. The problem is BDFL's attitude seems to drive many FP-ers away, like the guy in the original post.
    - kingkilr 17y ago
      They ultimately weren't removed. reduce was removed from the builtins, but it's still just one import away. From itertools import reduce.
  - ricree 17y ago
    If I remember correctly, "removing" these functions mostly just meant dumping them into the functools module rather than including them as a built in function.
  - holygoat 17y ago
    There's a difference between things being possible and being encouraged. De-emphasizing functional operations makes it more likely for libraries to work in a non-functional style, for tutorials to do so, etc. It's tiring fighting against a community and a (benevolent) dictator that disagree with you.
    - 17y ago
azanar 17y ago
[It] creates a cleaner language and forces people to think Pythonic when programming in Python. The question is whether "Pythonic," as the community defines it today, is optimal in all cases. As an analogue, there have been enough talks about things the Java community considers to be stylistically optimal that look really horrible compared to implementations in other languages -- like Python. :-) Pythonic style should be a guide, not an edict, and should be deviated from or redefined when it makes sense. The examples riobard provided already show the syntax weighing things down, a situation where syntax should give way to a more functional style approach. As for it creating a cleaner language, I try to approach this, as with all things, from the perspective of being an language-agnostic programmer. From that perspective, I do like the python syntax for simple things like what you defined, but under heavier weight mapping and filtering operations, the map/filter function call syntax seems a lot cleaner. There's nothing wrong with syntactic sugar, but I would assert that it will tend to suck when it is all you have.
holygoat 17y ago
Um... (filter #(> x 10) seq) sure is readable for me, and I'm fluent in Python and various Lisps. (That example is Clojure.) I would like to point out, however, that CL allows you to write: (loop for x in seq when (> x 10) collect x) which you might think is verbose ("why do I need that 'collect'?")... except that loop allows you to write things like (loop for i in *random* counting (evenp i) into evens counting (oddp i) into odds summing i into total maximizing i into max minimizing i into min finally (return (list min max total evens odds))) Loop knocks Python's trivial list comprehensions into a cocked hat. I switch between map/filter and loop depending on whether I'm working with predefined functions (e.g., (filter 'less-than-ten seq)), handling multiple sequences, doing side-effects, etc.
- foldr 17y ago
  >Loop knocks Python's trivial list comprehensions into a cocked hat. CL could really use something like Python's generators though. The loop macro is an ugly hack in comparison.

yummyfajitas 17y ago

I don't get how map/filter is more flexible than list comprehensions. Map/filter require nesting: (filter (map expensive_call X) cond) So do list comprehensions: [y for y in [expensive_call(x) for x in X] if cond(y)] Near as I can tell, the only difference is that list comprehensions also provide a syntactic sugar for the convenience function filter_then_map. Sometimes this saves you a level of nesting, sometimes not. Incidentally, this isn't even an issue in a pure functional language with sufficiently smart compiler.

riobard 17y ago
Well, FP is mostly about nesting, but it's uniform and people get used to thinking that way. List comprehensions, on the other hand, are more like procedure code (conceptually), and it sucks to nest them. Nesting aside, it's the different level of abstraction that matters for FP-ers. Plus, you get 5 mentions (3 y's and 2 x's) of some intermediate variables instead of 0 in your code, so both token-wise and char-wise, map/filter alternative is shorter, and less a mental burden (think about the "succinct" idea by PG).
- frig 17y ago
  I have a longer comment here that argues the opposite: list-comprehensions are only a syntactic pun or two away from set-builder notation, which is a higher level-of-abstraction (it declaratively states what it is) than map+filter (which specify a procedure to generate it, albeit at higher level of abstraction than a for-loop). If you can put together a nontrivial usage of map+filter with at least three source collections that's more concise than the equivalent list comprehension I'll (figuratively) eat my hat.
  - riobard 17y ago
    I'm not so sure if set notation is higher level, but for the example in your longer comment, map/filter/product is not that bad if you use it wisely. Here is my version: map(lambda (w,s,l): {'widget': w, 'sprocket': s, 'location': l} filter(lambda (w,s,l): l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s), product(widgets, sprockets, locations))) Well, I agree it is not any conciser that its list comprehension (about the same I guess?). Nothing is perfect, like you said, know you tool :)

frig 17y ago

It depends slightly on how you came to functional programming. You can arguably trace Python's list comprehension syntax all the way back to setl, a "set-theoretic programming language", and the resemblance to mathematical set-builder notation is intentional; compare - let B = { g(a) s.t. | a \in A and f(a) holds} - B = [g(a) for a in A if f(a)] If you're used to thinking in sets having to decompose into "maps" and "filters" is a speedbump; easy to do but nice to avoid. Where list comprehensions really start to shine is making it comparatively trivial to pull from multiple source collections without a lot of ugly machinery: [{'widget':w,'sprocket':s,'location':l} for w in widgets for s in sprockets for l in locations if l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s)] ...which is about where explicit map + filter start to become annoying. You can use: map(lambda i: {'widget':i[0], 'sprocket':i[1], 'location':i[2]}, filter(lambda i: i[2].hasInStock(i[0]) and i[2].hasInStock(i[1]) and i[0].isUsableWith(i[1]), itertools.product(widgets,sprockets,locations))) ...but to my eyes that is not only very ugly but just going by character count the # of characters given over to keywords (map, lambda, filter) instead of "what i'm doing here" is huge. Additionally use of itertools forces use of tuples for your intermediate values and thus the lambdas are gobbledegook until you get to the tail end of the statement and see that i[0] == widget, i[1] == sprocket, and i[2] == location. I could define some constants (WIDGET = 0, SPROCKET = 1, LOCATION = 2, etc) but now it's even longer. It gets even worse if you try to be clever with the sequence of operations. You might look at that definition and say lo! I can pre-filter out stuff not in stock at each location and make things more efficient. Naively you'd wind up with: map(lambda d: {'location':d[0], 'widget':d[1], 'sprocket':d[2]}, flatten(map(lambda l: list(filter(lambda p: p[1].isUsableWith(p[2]), itertools.product([l],filter(lambda w: l.hasInStock(w), widgets), filter(lambda s: l.hasInStock(s), sprockets)))), locations))) #nb must-supply-you-own-flatten-method Under some circumstances that might be substantially faster than the previous approach. But compare the equivalent but you could have instead gone with: [{'widget':w,'sprocket':s,'location':l} for l in locations for w in [widget for widget in widgets if l.hasInStock(widget)] for s in [sprocket for sprocket in sprockets if l.hasInStock(sprocket)] if w.isUsableWith(s)] So you lose a little flexibility in simple cases at in exchange for increasing the scope of what you can get away with as "readable" one-liners.

riobard 17y ago
Python's lambda sucks, but that does not necessarily mean map/filter sucks with it too. Of course there are cases where list comprehensions are more convenient and more powerful (esp. if they are more like their "real" counterparts in Haskell and friends). But in cases when map/filter are more convenient, I like the option of using them.