5 ms·

Honest question: why would anyone make a list comprehension that's multiple levels deep? Isn't the purpose of comprehensions to provide quick-and-dirty inline f

Honest question: why would anyone make a list comprehension that's multiple levels deep? Isn't the purpose of comprehensions to provide quick-and-dirty inline for loops, where a full for loop is too verbose?

jetrink 8y ago
It's useful for working with permutations. E.g, [f(a, b) for a in some_list for b in some_list] or even [f(a, b) for a in some_list for b in some_list if a is not b]
- kbp 8y ago
  itertools has functions that make that sort of nesting unnecessary (and some would say make the code clearer), eg: import itertools as it [f(a, b) for a, b in it.product(some_list, some_list) if a is not b]
  - deleted 8y ago
    [deleted]
  - kbp 8y ago
    Oh, or more directly: [f(a, b) for a, b in it.permutations(some_list, r=2)] And then you could also use itertools.starmap instead of using a comprehension at all, if you wanted.
SatvikBeri 8y ago
Comprehensions have some other benefits too, for example, they're much less likely to be accidentally stateful. My general experience is that for complex logic, a comprehension is harder to write initially, but much less likely to have bugs when I get it right. That said, I usually won't use deeply nested comprehensions, but sometimes it actually is the clearest way to parse something (e.g. extracting a field from nested JSON.)
shmolyneaux 8y ago
While supported, list comprehensions that are multiple levels deep isn't considered good style. There is an example of this in Google's python style guide: https://github.com/google/styleguide/blob/gh-pages/pyguide.md#27-comprehensions--generator-expressions https://github.com/google/styleguide/blob/gh-pages/pyguide.m...
bjourne 8y ago
People who come from math backgrounds often becomes infatuated with complicated list comprehensions when they realize that they can be used as set-builder notation. {(x,y) for x in range(10) for y in range(10) if y == 2*x}
- mike00632 8y ago
  That looks exactly like my notes from math courses and I'm overjoyed that it's meaningful in Python.
- analog31 8y ago
  I use stuff like that. But my rule of thumb is that I try and make it readable by breaking it up into multiple lines and using indentation. If I can't make it readable, then I will consider another way of writing it. This is mainly for self preservation, since I'm likely to be the person who has to read it later.
klodolph 8y ago
For me it’s very rare and used where I would write something like concatMap in Haskell. citiesByState = { 'WA': ['Seattle', 'Spokane', 'Olympia'], 'OR': ['Portland', 'Salem'], 'CA': ['San Francisco', 'Los Angeles', 'San Diego'], } cityToState = {city: state for state, cities in citiesByState.items() for city in cities} I would generally avoid multiple levels in a comprehension but there are some very simple two-level cases like this that I use occasionally. I feel that the code is easy to read, at least.
- evincarofautumn 8y ago
  In Haskell, these direct equivalents aren’t bad at all: cityToState = Map.fromList [ (city, state) | (state, cities) <- Map.toList citiesByState , city <- cities ] cityToState = Map.fromList $ do (state, cities) <- Map.toList citiesByState city <- cities pure (city, state) For multiple “nested loops”, I generally prefer do-notation over both list comprehensions and combinators such as concatMap, unless the structure is simple enough that the combinator version is much shorter.
  - klodolph 8y ago
    And for those following who don’t know Haskell, both of these are basically sugar for concatMap. (>>=) = flip concatMap
quietbritishjim 8y ago
Multiple for loops in list comprehensions (if that's what's meant by "multi-level") are pretty straightforward once you realise the simple rule that translates them into loops: write everything to the right of the expression in the same order with nesting. For example this: l = [f(x, y) for x in X for y in Y if x > y] is the same as this: l = [] for x in X: for y in Y: if x > y: l.append(f(x,y)) If a list comprehension doesn't fit on one line I try to highlight this to any future reader by using one line per thing that would get nested: l = [ f(x, y) for x in X for y in Y if x > y ] Needless to say, there are still list comprehensions that are complex enough that they ought to be broken out into loops. But being nested isn't enough by itself.
dorfsmay 8y ago
It used to be for performance reasons, the way python runs comprehension is faster than plain loops, so the more you could cram in a single comprehension, the faster it ran. These days though, you can split it in generator expressions and make it run at the end inside a comprehension, and you get best of both worlds: splitting helps readability, running it all at once in a single comprehension at the end keeps it performing.

jetrink 8y ago

It's useful for working with permutations. E.g, [f(a, b) for a in some_list for b in some_list] or even [f(a, b) for a in some_list for b in some_list if a is not b]

kbp 8y ago
itertools has functions that make that sort of nesting unnecessary (and some would say make the code clearer), eg: import itertools as it [f(a, b) for a, b in it.product(some_list, some_list) if a is not b]
- deleted 8y ago
  [deleted]
- kbp 8y ago
  Oh, or more directly: [f(a, b) for a, b in it.permutations(some_list, r=2)] And then you could also use itertools.starmap instead of using a comprehension at all, if you wanted.

SatvikBeri 8y ago

Comprehensions have some other benefits too, for example, they're much less likely to be accidentally stateful. My general experience is that for complex logic, a comprehension is harder to write initially, but much less likely to have bugs when I get it right. That said, I usually won't use deeply nested comprehensions, but sometimes it actually is the clearest way to parse something (e.g. extracting a field from nested JSON.)

shmolyneaux 8y ago

While supported, list comprehensions that are multiple levels deep isn't considered good style. There is an example of this in Google's python style guide: https://github.com/google/styleguide/blob/gh-pages/pyguide.md#27-comprehensions--generator-expressions https://github.com/google/styleguide/blob/gh-pages/pyguide.m...

bjourne 8y ago

People who come from math backgrounds often becomes infatuated with complicated list comprehensions when they realize that they can be used as set-builder notation. {(x,y) for x in range(10) for y in range(10) if y == 2*x}

mike00632 8y ago
That looks exactly like my notes from math courses and I'm overjoyed that it's meaningful in Python.
analog31 8y ago
I use stuff like that. But my rule of thumb is that I try and make it readable by breaking it up into multiple lines and using indentation. If I can't make it readable, then I will consider another way of writing it. This is mainly for self preservation, since I'm likely to be the person who has to read it later.

klodolph 8y ago

For me it’s very rare and used where I would write something like concatMap in Haskell. citiesByState = { 'WA': ['Seattle', 'Spokane', 'Olympia'], 'OR': ['Portland', 'Salem'], 'CA': ['San Francisco', 'Los Angeles', 'San Diego'], } cityToState = {city: state for state, cities in citiesByState.items() for city in cities} I would generally avoid multiple levels in a comprehension but there are some very simple two-level cases like this that I use occasionally. I feel that the code is easy to read, at least.

evincarofautumn 8y ago
In Haskell, these direct equivalents aren’t bad at all: cityToState = Map.fromList [ (city, state) | (state, cities) <- Map.toList citiesByState , city <- cities ] cityToState = Map.fromList $ do (state, cities) <- Map.toList citiesByState city <- cities pure (city, state) For multiple “nested loops”, I generally prefer do-notation over both list comprehensions and combinators such as concatMap, unless the structure is simple enough that the combinator version is much shorter.
- klodolph 8y ago
  And for those following who don’t know Haskell, both of these are basically sugar for concatMap. (>>=) = flip concatMap

quietbritishjim 8y ago

Multiple for loops in list comprehensions (if that's what's meant by "multi-level") are pretty straightforward once you realise the simple rule that translates them into loops: write everything to the right of the expression in the same order with nesting. For example this: l = [f(x, y) for x in X for y in Y if x > y] is the same as this: l = [] for x in X: for y in Y: if x > y: l.append(f(x,y)) If a list comprehension doesn't fit on one line I try to highlight this to any future reader by using one line per thing that would get nested: l = [ f(x, y) for x in X for y in Y if x > y ] Needless to say, there are still list comprehensions that are complex enough that they ought to be broken out into loops. But being nested isn't enough by itself.

dorfsmay 8y ago

It used to be for performance reasons, the way python runs comprehension is faster than plain loops, so the more you could cram in a single comprehension, the faster it ran. These days though, you can split it in generator expressions and make it run at the end inside a comprehension, and you get best of both worlds: splitting helps readability, running it all at once in a single comprehension at the end keeps it performing.