3 ms·

I learned several languages before Python, but the one that made it the most difficult was Ruby. After having done Ruby for years and becoming really familiar w

I learned several languages before Python, but the one that made it the most difficult was Ruby. After having done Ruby for years and becoming really familiar with idioms and standard practices, coming to python feels like trying to cuddle with a porcupine. List (or set, or dict) comprehensions are really cool... one level deep. The moment you do 'in...in' my brain shuts off. It takes me something like 30 minutes to finally get it, but then ten seconds later, I've lost it again. And there are a bunch of other things that just feel really awkward and uncomfortable in Python

whytevuhuni 2y ago
There's a pretty easy trick to nested comprehensions. Just write the loop as normal: for name, values in map_of_values.items(): for value in values: yield name, value Then concatenate everything on one line: [for name, values in map_of_values.items(): for value in values: yield name, value] Then move the yield to the very start, wrap it in parentheses (if necessary), and remove colons: [(name, value) for name, values in map_of_values.items() for value in values] And that's your comprehension. Maybe this makes more sense if you're reading it on a wide screen: [ for name, values in map_of_values.items(): for value in values: yield name, value] ^^^^^^^^^^^^^^^^^ /------------------------------------------------------------------------------/ vvvvvvvvvvvvv [(name, value) for name, values in map_of_values.items() for value in values ]
- JodieBenitez 2y ago
  I write Python every day and I would have just write the "normal" loop which is the only one that is really readable in your examples, at least to me.
  - whytevuhuni 2y ago
    I wrote it on one line to make the example make sense, but most Python comprehensions are plenty readable if they're split into many lines: new_list = [ (name, value) for name, values in map_of_values.items() for value in values ] The normal loop needs either a yield (which means you have to put it inside a generator function, and then call and list() the generator), or you have to make a list and keep appending things into it, which looks worse.
    - dijksterhuis 2y ago
      (not the parent) agree on the appending to a list. that’s an obvious code smell of “i didn’t know how to do idiomatic python iteration when i wrote this”. i disagree that the format of the list comprehension above is easier to read than a generator function personally. however, it’s definitely easier to read than most multi-loop list comprehensions i’ve seen before (even with HN formatting sticking the x.items() on a new line :/ ). so i see what you’re getting at. it is definitely better. i just find it weird that the the tuple definition comes before the thing i’m doing to get the tuple. sometimes i have to think “backwards” while reading it. i don’t like thinking while reading code. it makes it harder to read the code. and that “thinking” effect increases when the logic is complicated.
      - whytevuhuni 2y ago
        > i just find it weird that the the tuple definition comes before the thing i’m doing to get the tuple. I'll have to give you that one, after ten years of professional development it still feels backwards to me as well. I think we can blame the math people for that one: {x + 2 ∣ x ∈ N, x < 10}
    - JodieBenitez 2y ago
      That's indeed much better on many lines. I don't know why some people insist on making long lines.
- skeledrew 2y ago
  Wow, this is nice. I've been doing Python for quite a few years and whenever I needed a nested comprehension I'd always have to go to my cheatsheet. Now that I've seen how it's composed that's one less thing I'll need to lookup again. Thank you.
  - TheOtherHobbes 2y ago
    It's a fairly clunky idiom and there's no reason to use it if you prefer more explicit code. I can see the attraction for terse solutions, but even the name is questionable. ("Comprehension?" Of what? It's not comprehending anything. It's a filter/processor.)
- xk3 2y ago
  The first example is a generator. If you wanted to keep that just use `()` instead of `[]`: ((name, value) for name, values in map_of_values.items() for value in values)
dijksterhuis 2y ago
i have a rule of thumb around “one liner” [0] comprehensions that go beyond two nested for loops and/or multiple if conditions — turn it into a generator function def gen_not_one_y_from_xs(xs): for ys in xs: for y in ys: if y != 1: yield y # … later, actual call not_one_ys = list(gen_y_from_xs(my_xs)) the point of the rule being — if there are two nested for loops in a comprehension then it’s more complicated than previously thought and should be written out explicitly. list comprehensions are a convenience shortcut, so treat them as such and only use them for simple/obvious loops, basically. edit — also, now it’s possible to unit test the for loop logic and debug each step, which you cannot do with a list comprehension (unless a function is just returning the result of a list comprehension… which is… yeah… i mean you could do that…) [0]: 9 times out of 10, multiple for loop comprehensions are not one liners and end up sprawling over multiple lines and become an abject misery to debug.
mixmastamyk 2y ago
The loops are in standard order, with the yield value up front. Write the loops, surround with brackets, move the value up to the first line.

whytevuhuni 2y ago

There's a pretty easy trick to nested comprehensions. Just write the loop as normal: for name, values in map_of_values.items(): for value in values: yield name, value Then concatenate everything on one line: [for name, values in map_of_values.items(): for value in values: yield name, value] Then move the yield to the very start, wrap it in parentheses (if necessary), and remove colons: [(name, value) for name, values in map_of_values.items() for value in values] And that's your comprehension. Maybe this makes more sense if you're reading it on a wide screen: [ for name, values in map_of_values.items(): for value in values: yield name, value] ^^^^^^^^^^^^^^^^^ /------------------------------------------------------------------------------/ vvvvvvvvvvvvv [(name, value) for name, values in map_of_values.items() for value in values ]

JodieBenitez 2y ago
I write Python every day and I would have just write the "normal" loop which is the only one that is really readable in your examples, at least to me.
- whytevuhuni 2y ago
  I wrote it on one line to make the example make sense, but most Python comprehensions are plenty readable if they're split into many lines: new_list = [ (name, value) for name, values in map_of_values.items() for value in values ] The normal loop needs either a yield (which means you have to put it inside a generator function, and then call and list() the generator), or you have to make a list and keep appending things into it, which looks worse.
  - dijksterhuis 2y ago
    (not the parent) agree on the appending to a list. that’s an obvious code smell of “i didn’t know how to do idiomatic python iteration when i wrote this”. i disagree that the format of the list comprehension above is easier to read than a generator function personally. however, it’s definitely easier to read than most multi-loop list comprehensions i’ve seen before (even with HN formatting sticking the x.items() on a new line :/ ). so i see what you’re getting at. it is definitely better. i just find it weird that the the tuple definition comes before the thing i’m doing to get the tuple. sometimes i have to think “backwards” while reading it. i don’t like thinking while reading code. it makes it harder to read the code. and that “thinking” effect increases when the logic is complicated.
    - whytevuhuni 2y ago
      > i just find it weird that the the tuple definition comes before the thing i’m doing to get the tuple. I'll have to give you that one, after ten years of professional development it still feels backwards to me as well. I think we can blame the math people for that one: {x + 2 ∣ x ∈ N, x < 10}
  - JodieBenitez 2y ago
    That's indeed much better on many lines. I don't know why some people insist on making long lines.
skeledrew 2y ago
Wow, this is nice. I've been doing Python for quite a few years and whenever I needed a nested comprehension I'd always have to go to my cheatsheet. Now that I've seen how it's composed that's one less thing I'll need to lookup again. Thank you.
- TheOtherHobbes 2y ago
  It's a fairly clunky idiom and there's no reason to use it if you prefer more explicit code. I can see the attraction for terse solutions, but even the name is questionable. ("Comprehension?" Of what? It's not comprehending anything. It's a filter/processor.)
xk3 2y ago
The first example is a generator. If you wanted to keep that just use `()` instead of `[]`: ((name, value) for name, values in map_of_values.items() for value in values)

dijksterhuis 2y ago

i have a rule of thumb around “one liner” [0] comprehensions that go beyond two nested for loops and/or multiple if conditions — turn it into a generator function def gen_not_one_y_from_xs(xs): for ys in xs: for y in ys: if y != 1: yield y # … later, actual call not_one_ys = list(gen_y_from_xs(my_xs)) the point of the rule being — if there are two nested for loops in a comprehension then it’s more complicated than previously thought and should be written out explicitly. list comprehensions are a convenience shortcut, so treat them as such and only use them for simple/obvious loops, basically. edit — also, now it’s possible to unit test the for loop logic and debug each step, which you cannot do with a list comprehension (unless a function is just returning the result of a list comprehension… which is… yeah… i mean you could do that…) [0]: 9 times out of 10, multiple for loop comprehensions are not one liners and end up sprawling over multiple lines and become an abject misery to debug.

mixmastamyk 2y ago

The loops are in standard order, with the yield value up front. Write the loops, surround with brackets, move the value up to the first line.