6 ms·

Guido van Rossum has vocally[1] expressed a distaste for functional programming idioms. His loss, I think. List comprehensions have always struck me as a very

Guido van Rossum has vocally[1] expressed a distaste for functional programming idioms. His loss, I think. List comprehensions have always struck me as a very ugly mechanism which is convenient in simple instances and which composes together poorly. This kind of special-casing is all over the place in Python's design. Consider, for example, all the unintuitive syntactic and semantic contortions introduced to support interval comparisons[2]. The overall result is a language which is apparently simple but in practice surprisingly complex. [1] http://www.artima.com/weblogs/viewpost.jsp?thread=98196 http://www.artima.com/weblogs/viewpost.jsp?thread=98196 [2] https://docs.python.org/2/reference/expressions.html#not-in https://docs.python.org/2/reference/expressions.html#not-in

pekk 10y ago
If your ideal is Lisp, it's not surprising if you find every other language lacking.
- Pxtl 10y ago
  Lisp represents the ultimate sacrifice of legibility, type-safety, and familiarity in exchange for expressiveness. But it's reasonable to say "I like other languages than Lisp for other reasons, but I wish they had Lisp feature X in a form that fits the philosophy of the language". I mean, Python could offer an alternate sexpr syntax that makes it a lisp, and that would provide nice higher-order-function support for map and filter and foldr, but that wouldn't be a "pythonic" solution to the problem. I want a "pythonic" answer for map and filter and reduce. List Comprehensions are supposed to provide that, but imho they're a failed attempt. They're ugly and restrictive.
  - zardeh 10y ago
    That's because there's an argument to be made here that the blog poster here is misusing the comprehensions syntax. While what he does works, it can just as easily be written as list(itertools.chain(*[[x for x in sublist if x%3 == 0] for sublist in nested_list])) Which uses a bit of magic to massage the end result, but it is much clearer for the inner portions, and much similar to your 'lispy' style. (`itertools.chain(` is equivalent to a 1-level flatten) You can do the same thing in the bigger case, and you end up nesting your structures such that you get something like results = [[[[[[[[[[[[[[[(a, b, g, d, e, z, et, th, i, k, th, mu, n, xi, o, pi, r, s) for th in range(1, 27) if (2*(th**2) + o**2 + r**2 == 1000) and ((th**2 * z**2 * (s - z) + s**2) == 7225)] for o in range(1, 27) if (b**3 + z**3 + n**3 + o**9 == 1997)] for r in range(1, 27)] for z in range(1, 27)] for n in range(1, 27) if F(a, k)**2 + F(b, n)**2 + F(d, xi)**2 + F(e, pi)**2 + F(et, mu)**2 + F(i, s)**2 == 6] for b in range(1, 27)] for e in range(1, 27)] for s in range(1, 27)] for (a, mu) in ((4, 1), (2, 2), (1, 4)) if a*(a+pi) == 4 * g] for pi in range(1, 27)] for g in range(1, 27) if (g-1)**2 + d**2 + xi**2 - xi == 600] for d in range(1, 27)] for (et, i) in ((xi-7, xi-11), (xi-11,xi-7), (xi+7, xi+11), (xi+11, xi+7)) if 0 < et <= 26 and 0 < i <= 26] for xi in range(1, 27)] for k in (3,4)] Which admittedly isn't better*, but you can build up over time (ie. you can start with [f(k) for k in range(3, 4)], and then provide f(k) in a closure, and repeat, to build this up over time.
  - sedachv 10y ago
    Do you know about the Common Lisp loop macro? You can write the same list comprehension as in the article with almost exactly the same syntax using CL loop for, if and collect clauses. It is hard to take criticism like "<Language X> represents the ultimate sacrifice of legibility, type-safety, and familiarity in exchange for expressiveness" seriously if you do not even have basic proficiency with <Language X>.
- draw_down 10y ago
  Plenty of non-Lisps with decent lambdas and map/filter.
  - Pxtl 10y ago
    Exactly. I'm primarily a C# coder and I love C#'s implementation of map/filter using a terse Lambda syntax and higher-order functions named after the SQL counterparts of map/filter: Select and Where.
Pxtl 10y ago
I get that this may have been a reasonable attitude back in the '90s when most languages doing higher order functions were a bit cryptic-looking about it, but in 2017 there's enough variations on the theme that he should be able to come up with something clean and pythonic.
YeGoblynQueenne 10y ago
>> List comprehensions have always struck me as a very ugly mechanism I've always thought they are basically an implementation of set builder notation, e.g.: X = {a | a ∈ Y} Equivalent to: X = [a for a in Y] And so on? [1] https://en.wikipedia.org/wiki/Set-builder_notation https://en.wikipedia.org/wiki/Set-builder_notation
- Pxtl 10y ago
  While normally following math notation is good, the lack of composability shown in the article is a real drawback of that syntax.
  - YeGoblynQueenne 10y ago
    I think that's more of an implementation issue, rather than a problem with the idea of comprehensions. In set builder notation you can order elements as you like without changing the semantics. So you can denote the following "nested membership" statement in either order: X = {a | A ∈ Y, a ∈ A } X = {a | a ∈ A, A ∈ Y} But in Python, changing the order changes the meaning: >>> Y = (('a','b','c'), ('d','e','f')) >>> X = [a for A in Y for a in A] >>> X ['a', 'b', 'c', 'd', 'e', 'f'] >>> X = [a for a in A for A in Y] >>> X [('a', 'b', 'c'), ('a', 'b', 'c'), ('d', 'e', 'f'), ('d', 'e', 'f')] It's the difference between declarative and imperative semantics- a kind of declarative-imperative impedance mismatch, if you like. The set-builder notation tells you how things are, timeless and unchanging. The imperative notation tells you how to get there. Consequently, you can't just throw any set of comprehension elements together and hope that you'll get something that makes sense. This messes up the elegance of the notation considerably, and it also makes it very, very fiddly to write some things that you might conceivably want to write, so much so that it ends up being much more readable to write a traditional for-loop, than use a comprehension. There might even be orderings that are not allowed by the Python compiler, even though they might make sense in a declarative formalism. I can't think of any right now, but I'm pretty sure there are plenty- and lambdas are not going to be a get-out-of-jail-free card either.
hcrisp 10y ago
Here's a Python example in "map/filter" functional programming style, adapted from the generator version I posted elsewhere. from fractions import Fraction as F def thor(): for th in range(1, 27): for o in range(1, 27): for r in range(1, 27): if 2*(th**2) + o**2 + r**2 == 1000: yield th, o, r def sz(thors): th, o, r = thors for s in range(1, 27): for z in range(1, 27): if (th**2 * z**2 * (s - z) + s**2) == 7225: yield th, o, r, s, z def gdxi(szs): th, o, r, s, z = szs for g in range(1, 27): for d in range(1, 27): for xi in range(1, 27): if (g-1)**2 + d**2 + xi**2 - xi == 600: yield th, o, r, s, z, g, d, xi def eti(gdxis): th, o, r, s, z, g, d, xi = gdxis for (et, i) in ( (xi-7, xi-11), (xi-11,xi-7), (xi+7, xi+11), (xi+11, xi+7)): if 0 < et <= 26 and 0 < i <= 26: yield th, o, r, s, z, g, d, xi, et, i def amupi(etis): th, o, r, s, z, g, d, xi, et, i = etis for (a, mu) in ((4, 1), (2, 2), (1, 4)): for pi in range(1, 27): if a*(a+pi) == 4 * g: yield th, o, r, s, z, g, d, xi, et, i, a, mu, pi def kben(amupis): th, o, r, s, z, g, d, xi, et, i, a, mu, pi = amupis for k in (3,4): for b in range(1, 27): for e in range(1, 27): for n in range(1, 27): if b**3 + z**3 + n**3 + o**9 == 1997: if (F(a, k)**2 + F(b, n)**2 + F(d, xi)**2 + F(e, pi)**2 + F(et, mu)**2 + F(i, s)**2 == 6): yield (a, b, g, d, e, z, et, th, i, k, th, mu, n, xi, o, pi, r, s) flatten = lambda l: [item for sublist in l for item in sublist] thors = thor() szs = flatten(filter(None, map(lambda x: list(sz(x)), thors))) gdxis = flatten(filter(None, map(lambda x: list(gdxi(x)), szs))) etis = flatten(filter(None, map(lambda x: list(eti(x)), gdxis))) amupis = flatten(filter(None, map(lambda x: list(amupi(x)), etis))) results = flatten(filter(None, map(lambda x: list(kben(x)), amupis))) print('\t'.join(('a', 'b', 'g', 'd', 'e', 'z', 'et', 'th', 'i', 'k', 'l', 'mu', 'n', 'xi', 'o', 'pi', 'r', 's'))) for r in results: print('\t'.join(map(str, r)))
- hcrisp 10y ago
  The last part can be simplified to this: thors = thor() szs = flatten(map(list, map(sz, thors))) gdxis = flatten(map(list, map(gdxi, szs))) etis = flatten(map(list, map(eti, gdxis))) amupis = flatten(map(list, map(amupi, etis))) results = flatten(map(list, map(kben, amupis))) # print results

pekk 10y ago

If your ideal is Lisp, it's not surprising if you find every other language lacking.

Pxtl 10y ago
Lisp represents the ultimate sacrifice of legibility, type-safety, and familiarity in exchange for expressiveness. But it's reasonable to say "I like other languages than Lisp for other reasons, but I wish they had Lisp feature X in a form that fits the philosophy of the language". I mean, Python could offer an alternate sexpr syntax that makes it a lisp, and that would provide nice higher-order-function support for map and filter and foldr, but that wouldn't be a "pythonic" solution to the problem. I want a "pythonic" answer for map and filter and reduce. List Comprehensions are supposed to provide that, but imho they're a failed attempt. They're ugly and restrictive.
- zardeh 10y ago
  That's because there's an argument to be made here that the blog poster here is misusing the comprehensions syntax. While what he does works, it can just as easily be written as list(itertools.chain(*[[x for x in sublist if x%3 == 0] for sublist in nested_list])) Which uses a bit of magic to massage the end result, but it is much clearer for the inner portions, and much similar to your 'lispy' style. (`itertools.chain(` is equivalent to a 1-level flatten) You can do the same thing in the bigger case, and you end up nesting your structures such that you get something like results = [[[[[[[[[[[[[[[(a, b, g, d, e, z, et, th, i, k, th, mu, n, xi, o, pi, r, s) for th in range(1, 27) if (2*(th**2) + o**2 + r**2 == 1000) and ((th**2 * z**2 * (s - z) + s**2) == 7225)] for o in range(1, 27) if (b**3 + z**3 + n**3 + o**9 == 1997)] for r in range(1, 27)] for z in range(1, 27)] for n in range(1, 27) if F(a, k)**2 + F(b, n)**2 + F(d, xi)**2 + F(e, pi)**2 + F(et, mu)**2 + F(i, s)**2 == 6] for b in range(1, 27)] for e in range(1, 27)] for s in range(1, 27)] for (a, mu) in ((4, 1), (2, 2), (1, 4)) if a*(a+pi) == 4 * g] for pi in range(1, 27)] for g in range(1, 27) if (g-1)**2 + d**2 + xi**2 - xi == 600] for d in range(1, 27)] for (et, i) in ((xi-7, xi-11), (xi-11,xi-7), (xi+7, xi+11), (xi+11, xi+7)) if 0 < et <= 26 and 0 < i <= 26] for xi in range(1, 27)] for k in (3,4)] Which admittedly isn't better*, but you can build up over time (ie. you can start with [f(k) for k in range(3, 4)], and then provide f(k) in a closure, and repeat, to build this up over time.
- sedachv 10y ago
  Do you know about the Common Lisp loop macro? You can write the same list comprehension as in the article with almost exactly the same syntax using CL loop for, if and collect clauses. It is hard to take criticism like "<Language X> represents the ultimate sacrifice of legibility, type-safety, and familiarity in exchange for expressiveness" seriously if you do not even have basic proficiency with <Language X>.
draw_down 10y ago
Plenty of non-Lisps with decent lambdas and map/filter.
- Pxtl 10y ago
  Exactly. I'm primarily a C# coder and I love C#'s implementation of map/filter using a terse Lambda syntax and higher-order functions named after the SQL counterparts of map/filter: Select and Where.

Pxtl 10y ago

I get that this may have been a reasonable attitude back in the '90s when most languages doing higher order functions were a bit cryptic-looking about it, but in 2017 there's enough variations on the theme that he should be able to come up with something clean and pythonic.

YeGoblynQueenne 10y ago

>> List comprehensions have always struck me as a very ugly mechanism I've always thought they are basically an implementation of set builder notation, e.g.: X = {a | a ∈ Y} Equivalent to: X = [a for a in Y] And so on? [1] https://en.wikipedia.org/wiki/Set-builder_notation https://en.wikipedia.org/wiki/Set-builder_notation

Pxtl 10y ago
While normally following math notation is good, the lack of composability shown in the article is a real drawback of that syntax.
- YeGoblynQueenne 10y ago
  I think that's more of an implementation issue, rather than a problem with the idea of comprehensions. In set builder notation you can order elements as you like without changing the semantics. So you can denote the following "nested membership" statement in either order: X = {a | A ∈ Y, a ∈ A } X = {a | a ∈ A, A ∈ Y} But in Python, changing the order changes the meaning: >>> Y = (('a','b','c'), ('d','e','f')) >>> X = [a for A in Y for a in A] >>> X ['a', 'b', 'c', 'd', 'e', 'f'] >>> X = [a for a in A for A in Y] >>> X [('a', 'b', 'c'), ('a', 'b', 'c'), ('d', 'e', 'f'), ('d', 'e', 'f')] It's the difference between declarative and imperative semantics- a kind of declarative-imperative impedance mismatch, if you like. The set-builder notation tells you how things are, timeless and unchanging. The imperative notation tells you how to get there. Consequently, you can't just throw any set of comprehension elements together and hope that you'll get something that makes sense. This messes up the elegance of the notation considerably, and it also makes it very, very fiddly to write some things that you might conceivably want to write, so much so that it ends up being much more readable to write a traditional for-loop, than use a comprehension. There might even be orderings that are not allowed by the Python compiler, even though they might make sense in a declarative formalism. I can't think of any right now, but I'm pretty sure there are plenty- and lambdas are not going to be a get-out-of-jail-free card either.

hcrisp 10y ago

Here's a Python example in "map/filter" functional programming style, adapted from the generator version I posted elsewhere. from fractions import Fraction as F def thor(): for th in range(1, 27): for o in range(1, 27): for r in range(1, 27): if 2*(th**2) + o**2 + r**2 == 1000: yield th, o, r def sz(thors): th, o, r = thors for s in range(1, 27): for z in range(1, 27): if (th**2 * z**2 * (s - z) + s**2) == 7225: yield th, o, r, s, z def gdxi(szs): th, o, r, s, z = szs for g in range(1, 27): for d in range(1, 27): for xi in range(1, 27): if (g-1)**2 + d**2 + xi**2 - xi == 600: yield th, o, r, s, z, g, d, xi def eti(gdxis): th, o, r, s, z, g, d, xi = gdxis for (et, i) in ( (xi-7, xi-11), (xi-11,xi-7), (xi+7, xi+11), (xi+11, xi+7)): if 0 < et <= 26 and 0 < i <= 26: yield th, o, r, s, z, g, d, xi, et, i def amupi(etis): th, o, r, s, z, g, d, xi, et, i = etis for (a, mu) in ((4, 1), (2, 2), (1, 4)): for pi in range(1, 27): if a*(a+pi) == 4 * g: yield th, o, r, s, z, g, d, xi, et, i, a, mu, pi def kben(amupis): th, o, r, s, z, g, d, xi, et, i, a, mu, pi = amupis for k in (3,4): for b in range(1, 27): for e in range(1, 27): for n in range(1, 27): if b**3 + z**3 + n**3 + o**9 == 1997: if (F(a, k)**2 + F(b, n)**2 + F(d, xi)**2 + F(e, pi)**2 + F(et, mu)**2 + F(i, s)**2 == 6): yield (a, b, g, d, e, z, et, th, i, k, th, mu, n, xi, o, pi, r, s) flatten = lambda l: [item for sublist in l for item in sublist] thors = thor() szs = flatten(filter(None, map(lambda x: list(sz(x)), thors))) gdxis = flatten(filter(None, map(lambda x: list(gdxi(x)), szs))) etis = flatten(filter(None, map(lambda x: list(eti(x)), gdxis))) amupis = flatten(filter(None, map(lambda x: list(amupi(x)), etis))) results = flatten(filter(None, map(lambda x: list(kben(x)), amupis))) print('\t'.join(('a', 'b', 'g', 'd', 'e', 'z', 'et', 'th', 'i', 'k', 'l', 'mu', 'n', 'xi', 'o', 'pi', 'r', 's'))) for r in results: print('\t'.join(map(str, r)))

hcrisp 10y ago
The last part can be simplified to this: thors = thor() szs = flatten(map(list, map(sz, thors))) gdxis = flatten(map(list, map(gdxi, szs))) etis = flatten(map(list, map(eti, gdxis))) amupis = flatten(map(list, map(amupi, etis))) results = flatten(map(list, map(kben, amupis))) # print results