3 ms·
Python's lambda also has an interesting design flaw that anyone using it will come up against sooner or later [1]. The way it interacts with Python's scoping ru
by roenxi 8y ago
Python's lambda also has an interesting design flaw that anyone using it will come up against sooner or later [1]. The way it interacts with Python's scoping rules is quite broken. This hinders using it as freely as is possible in, eg, Clojure.
>>> fs = [(lambda n: i + n) for i in range(10)]
>>> [f(4) for f in fs]
[13, 13, 13, 13, 13, 13, 13, 13, 13, 13]
[1] http://math.andrej.com/2009/04/09/pythons-lambda-is-broken/ http://math.andrej.com/2009/04/09/pythons-lambda-is-broken/
- UncleEntity 8y agoPython stores the captured variables in __globals__ which is shared with all the different lambdas for some odd reason: >>> fs[0] == fs[1] False >>> fs[0].__globals__ == fs[1].__globals__ True >>> fs[0].__globals__['i'], fs[1].__globals__['i'] (9, 9) Kind of weird? --edit-- >>> def f(i): ... return lambda n: i + n >>> fs = [f(i) for i in range(10)] >>> [f(4) for f in fs] [4, 5, 6, 7, 8, 9, 10, 11, 12, 13] It works if it has somewhere to store the captured variables so just don't pollute the global environment and all is fine...
- abecedarius 8y agoI think you're looking at Python 2, because in py3 the list comprehensions create a nested scope for the loop variable to be local in. -- i.e. 'i' would be in a local environment, not in globals. This doesn't change the behavior of these particular closures, since i is still mutated in the loop.
- UncleEntity 8y agoPython 3.6.5 (default, Apr 4 2018, 15:01:18) ...even weirder: >>> fs[0].__closure__ == fs[1].__closure__ True
- abecedarius 8y agoSince I don't have py3 installed at the moment, I can't check anything, but I'd guess this is two closures with distinct ids but the same environments, and == is doing a structural comparison rather than an identity comparison. Wish I could look into it right now.
- UncleEntity 8y ago>>> fs[0].__closure__, fs[1].__closure__ ((<cell at 0x7f4e9111b048: int object at 0x7f4e98e99500>,), (<cell at 0x7f4e9111b048: int object at 0x7f4e98e99500>,)) So lambda isn't broken it's the list comprehension going all wonky with its local variables methinks.
- abecedarius 8y agoAll right, I looked into this using repl.it. The __closure__ field is not actually a closure in the usual sense of the word -- it's the environment of the closure, and fs[0] and fs[1] are expected to share the same environment. If you look at the identities of fs[0] and fs[1] themselves, they are different. It's as I said originally, just with a confusing name for the environment field.
- xapata 8y agoA lambda is no different in scoping rules than a regular function def. If you put a def in a loop, you'll find the same behavior. Closures are only created for functions defined inside other functions. Thus the term non-local. They're variables that are neither local nor global. Don't call it broken when it works as intended.
- _sdegutis 8y ago“Works as intended” but also “unintuitive to most users” is still broken. Disclaimer: I have hated python ever since falling out of love with it 8 years ago for reasons just like this one so I’m definitely biased.
- xapata 8y agoSince programming is a rapidly growing field and Python is the first language for many, I suspect that Python's scoping rules are in fact intuitive to most of its users. The ones who stumble are those with intuition built by some other language.
- _sdegutis 8y agoI'd argue that's the opposite. Python was the first dynamic language I learned. The scoping rules in Python didn't bother me because I didn't know any better, not because they're intuitive or well designed.
- xapata 8y agoFair anecdote. I'll offer that in my experience teaching hundreds of students, most don't have trouble with the cascade from locals to globals to builtins, and quickly grasp that globals are looked up for each call. That closures work at all confuses the ones who are thinking carefully about local scope. I may have a biased sample of students, but I think it's decent evidence.
- fulafel 8y agoAs another comment said, this works the same way in def and lambda. This same thing happens in Javascript: http://markheath.net/post/javascript-closure-loop-gotcha http://markheath.net/post/javascript-closure-loop-gotcha