5 ms·

I spent some time comparing Python's [0] map/filter and comprehensions against Snigl [1]. Comprehensions are just plain faster, especially if you perform say f

I spent some time comparing Python's [0] map/filter and comprehensions against Snigl [1]. Comprehensions are just plain faster, especially if you perform say filter and map at the same time. That and a lambda implementation that only a mother could love, lack of tail call optimization etc. What bothers me about comprehensions is they come with a separate set of rules, like LOOP in Common Lisp or templates in C++; but I'm pretty sure the restrictions are part of what makes them fast. [0] https://gitlab.com/sifoo/snigl/blob/master/bench/seq.py https://gitlab.com/sifoo/snigl/blob/master/bench/seq.py [1] https://gitlab.com/sifoo/snigl/blob/master/bench/seq.sl https://gitlab.com/sifoo/snigl/blob/master/bench/seq.sl

klyrs 8y ago
Agreed, this made my eye twitch: > filter(P, S) is almost always written clearer as [x for x in S if P(x)] sure, Guido, if you think it's clear that the variable x is in the enclosing scope...
- joshuamorton 8y ago
  Python 3 changed this, iirc. Comprehension no longer leak.
  - jks 8y ago
    It's still an assignment operation, and you can still get weird effects in the enclosing scope. I did something like this by accident: Python 3.7.1 (default, Nov 28 2018, 21:47:25) Type 'copyright', 'credits' or 'license' for more information IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: lookup = {} In [2]: [x for (x, lookup[x]) in ('ab', 'cd', 'ef') if x in lookup] Out[2]: ['a', 'c', 'e'] In [3]: lookup Out[3]: {'a': 'b', 'c': 'd', 'e': 'f'}
    - joshuamorton 8y ago
      The whacky thing there isn't the list comprehension though, it's the implicit assignment of `lookup[x]`. If you desugar it some (which isn't really possible, but [(x, lookup.setdefault(x, y)) for (x, y) in ('ab', 'cd', 'ef')] comes close), what's happening makes a lot more sense. It works exactly the same way as any other scope: first checks locally, then nonlocally or globally. You're modifying a nonlocal. The syntax you're using to modify the nonlocal is surprising, but that's the surprising part, not the ability to modify a nonlocal.
      - jks 8y ago
        I think I was personally surprised by two assumptions that I implicitly held, both of which disappear when you write them down: 1. The syntax of comprehensions led me to think that they introduce some kind of a variable binding, such as `let` in many Lisps, and that binding `lookup[x]` would have been a syntax error. Of course Python didn't introduce anything like that for comprehensions, and it's just a normal assignment operation, with an extra scope inserted in Python 3 to stop `x` leaking into the enclosing scope. 2. Because of the swap idiom `x, y = y, x` I thought tuple assignment happens somehow in parallel, so my code should have caused an undefined variable error, even if it is not a syntax error. But of course it just evaluates the right hand side first and then assigns to the left hand side in left-to-right order, which the language reference https://docs.python.org/3/reference/simple_stmts.html#assignment-statements https://docs.python.org/3/reference/simple_stmts.html#assign... notes "sometimes result[s] in confusion" (near the end of the section).
      - theamk 8y ago
        I'd desugar it like this: >>> lookup = {} >>> for (x, lookup[x]) in ('ab', 'cd', 'ef'): ... pass ... >>> lookup {'a': 'b', 'c': 'd', 'e': 'f'} This makes is much more clear that there is just a double assignment, and that comprehension are pretty close to regular `for` statement. (this was a great example btw -- I write python3 a lot, and I was staring at it for a long time before Icould figure it out)
    - zephyrfalcon 8y ago
      It appears to work without list comprehensions as well: Python 3.7.1 (default, Nov 6 2018, 18:49:54) Type 'copyright', 'credits' or 'license' for more information IPython 7.1.1 -- An enhanced Interactive Python. Type '?' for help. In [1]: lookup = {} In [2]: a, lookup[a] = 'xy' In [3]: a Out[3]: 'x' In [4]: lookup Out[4]: {'x': 'y'} I wasn't previously aware that this was possible, but I guess it's no different from a = 'x' lookup[a] = 'y' # i.e. lookup['x'] = y
    - klyrs 8y ago
      Okay that's pretty sweet actually. I write a lot of legacy-compatible code, so I'm kinda behind on Py3. Maybe that's my 2019 project.
masklinn 8y ago
> Comprehensions are just plain faster, especially if you perform say filter and map at the same time. Comprehensions are often slower if you're applying a function regardless, definitely so if that function is a builtin.

klyrs 8y ago

Agreed, this made my eye twitch: > filter(P, S) is almost always written clearer as [x for x in S if P(x)] sure, Guido, if you think it's clear that the variable x is in the enclosing scope...

joshuamorton 8y ago
Python 3 changed this, iirc. Comprehension no longer leak.
- jks 8y ago
  It's still an assignment operation, and you can still get weird effects in the enclosing scope. I did something like this by accident: Python 3.7.1 (default, Nov 28 2018, 21:47:25) Type 'copyright', 'credits' or 'license' for more information IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: lookup = {} In [2]: [x for (x, lookup[x]) in ('ab', 'cd', 'ef') if x in lookup] Out[2]: ['a', 'c', 'e'] In [3]: lookup Out[3]: {'a': 'b', 'c': 'd', 'e': 'f'}
  - joshuamorton 8y ago
    The whacky thing there isn't the list comprehension though, it's the implicit assignment of `lookup[x]`. If you desugar it some (which isn't really possible, but [(x, lookup.setdefault(x, y)) for (x, y) in ('ab', 'cd', 'ef')] comes close), what's happening makes a lot more sense. It works exactly the same way as any other scope: first checks locally, then nonlocally or globally. You're modifying a nonlocal. The syntax you're using to modify the nonlocal is surprising, but that's the surprising part, not the ability to modify a nonlocal.
    - jks 8y ago
      I think I was personally surprised by two assumptions that I implicitly held, both of which disappear when you write them down: 1. The syntax of comprehensions led me to think that they introduce some kind of a variable binding, such as `let` in many Lisps, and that binding `lookup[x]` would have been a syntax error. Of course Python didn't introduce anything like that for comprehensions, and it's just a normal assignment operation, with an extra scope inserted in Python 3 to stop `x` leaking into the enclosing scope. 2. Because of the swap idiom `x, y = y, x` I thought tuple assignment happens somehow in parallel, so my code should have caused an undefined variable error, even if it is not a syntax error. But of course it just evaluates the right hand side first and then assigns to the left hand side in left-to-right order, which the language reference https://docs.python.org/3/reference/simple_stmts.html#assignment-statements https://docs.python.org/3/reference/simple_stmts.html#assign... notes "sometimes result[s] in confusion" (near the end of the section).
    - theamk 8y ago
      I'd desugar it like this: >>> lookup = {} >>> for (x, lookup[x]) in ('ab', 'cd', 'ef'): ... pass ... >>> lookup {'a': 'b', 'c': 'd', 'e': 'f'} This makes is much more clear that there is just a double assignment, and that comprehension are pretty close to regular `for` statement. (this was a great example btw -- I write python3 a lot, and I was staring at it for a long time before Icould figure it out)
  - zephyrfalcon 8y ago
    It appears to work without list comprehensions as well: Python 3.7.1 (default, Nov 6 2018, 18:49:54) Type 'copyright', 'credits' or 'license' for more information IPython 7.1.1 -- An enhanced Interactive Python. Type '?' for help. In [1]: lookup = {} In [2]: a, lookup[a] = 'xy' In [3]: a Out[3]: 'x' In [4]: lookup Out[4]: {'x': 'y'} I wasn't previously aware that this was possible, but I guess it's no different from a = 'x' lookup[a] = 'y' # i.e. lookup['x'] = y
  - klyrs 8y ago
    Okay that's pretty sweet actually. I write a lot of legacy-compatible code, so I'm kinda behind on Py3. Maybe that's my 2019 project.

masklinn 8y ago

> Comprehensions are just plain faster, especially if you perform say filter and map at the same time. Comprehensions are often slower if you're applying a function regardless, definitely so if that function is a builtin.