6 ms·
My biggest complaint about Python is that it somehow doesn’t get flak for having the same (if not worse) scoping as JS, which gets endless hate for its function
by jakear 7y ago
My biggest complaint about Python is that it somehow doesn’t get flak for having the same (if not worse) scoping as JS, which gets endless hate for its function-scoped variables. (So much so that block scoped variables are the new normal in JS, but not in Py!)
Take for instance:
>>> powers_of_x = [lambda x: x^i for i in range(10)]
>>> [f(2) for f in powers_of_x]
[512, 512, 512, 512, 512, 512, 512, 512, 512, 512]
To me this is more absurd than any of the JS “wat”s I’ve seen.
In JS:
powersOfX = Array.from({length: 10}).map((_,i) => x => x^i)
powersOfX.map(f => f(2))
Not a particular fan of the Array.from over list comprehensions in terms of syntax, but I much prefer consistency of semantics JS provides by not adding new syntax.
Here Python has special syntax for both list comprehensions and for anonymous single line functions, and they interact in a highly unexpected way.
Funny that Py doesn’t need semicolons, as I’m reminded of GJS telling me of adding special forms to languages: “beware the allure of syntactic sugar, lest you bring upon yourself the curse of the semicolon” (paraphrased of course, he said it much better). It seems Python has fallen into the trap of syntactic sugar, with the curse manifesting itself not in the form of a semicolon, but in confusing and unexpected interactions between its various special forms. Another example: the walrus operator and all its oddities listed in TFA.
- varelaz 7y agoThere is nothing bad in general with list comprehension there. Problem is lambda which uses variable from outer scope. In Python, comparing to JS, using variables from outer scope is not so common. I prefer more obvious constructions and in case of problems with scope I can even create class to protect value that I need to manage.
- BiteCode_dev 7y agoI agree totally with that, it's an ugly trap in python, although mitigated by the fact the language doesn't encourage this kind of paradigm so you rarely encounter it. But more absurd than creating a global variable if you forget "var/let/const" ? More absurd than "this" being schizophrenic ? More absurd than having no namespace for 20 years ? That's pushing it.
- aidos 7y agoAfter a long time using both languages, I’d argue that non-sane equality checks in js is the biggest source of bugs.
- tyri_kai_psomi 7y agoUnfortunately, this is not true. Although the equality operators are a favorite bikeshedding topic, the largest source of errors in JS code bases are type errors. Using a "non-sane" equality check, such as those with type coercion would actually mask or alleviate these source of bugs you mention.
- aidos 7y agoI was thinking more of the case where the types match but objects and arrays with the same contents are considered different. I’ve watched every member of my team get stung by it again and again - and then have to create workarounds to get past it.
- CallocRoast 7y agoI don’t agree and this behavior shouldn’t be surprising. The alternative would be to walk the container and compare the value of each element, which could be horrible.
- BiteCode_dev 7y agoQuite the contrary, it's very beautiful and useful: >>> (1, 2) == (1, 2) True >>> (2, 1) == (1, 2) False If you need identity for perf: >>> (1, 2) is (1, 2) False If you just need the type check: >>> isinstance((1, 2), type((1, 2))) True It's very explicit, practical, and you can set the scale of practicality vs performances where you want. Plus: no implicit weird type conversion, only one equality comparison operator, and no hidden rules. I think it's sane.
- aidos 7y ago
- aidos 7y agoMaybe I’m misunderstanding, but I’d argue that arrow syntax in js is equally special syntax as lambda function. They’re both native ways of making anonymous functions. Python’s version is more clunky, but no more special
- verttii 7y agoAren't Python's lambdas expressions while JS's arrow functions are not?
- jakear 7y agoInteresting, I don’t use python day-to-day, so I had assumed that the delayed capturing of the iteration variable was a lambda thing. Looks like this is still broken: for i in range(10): def mul(x): return i^x powers_of_x.append(mul) That to me coming from a JS background is totally wild
- aidos 7y agoIt’s actually due to python late binding. The variables are looked up at call time, rather than function define time. One could argue that your example shouldn’t work anymore in python3 since variables in list comprehensions go out of scope when they finish now. I haven’t tried it myself. I fill it under, “things I never need to do” :-)
- masklinn 7y agoIt's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change. for(var i=0; i<10; ++i) { powersOfX.push(x => x^i); } will behave the same way because it does the same thing, so will e.g. for i := 0; i<10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return x ^ i }) } in Go. One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible). The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over. Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions. In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas: for i in range(10): powers_of_x.append(lambda x, i=i: x^i) [0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings) [1] and remains a regular issue in async code fighting over closed-over context
- mschuetz 7y agoYes! So much that! I can't believe that it's okay in python to declare/initialize a variable within a loop or if statement, and that it goes to live on afterwards. JS fixed that issue years ago with let and const.
- edflsafoiewq 7y agoI don't find this at all unexpected. In an imperative language I expect a loop to be implemented by mutating a common variable.
- jakear 7y agoDo you perhaps think that because you come from a background in low-level languages?
- deleted 7y ago[deleted]
- owl57 7y agoImperative loop, maybe. But list comprehensions are a sublanguage imported from Haskell and looking like Haskell, so it's somewhat natural to be surprised by mutable variables here.
- deleted 7y ago[deleted]
- deleted 7y ago[deleted]
- fanf2 7y agoLua and Perl use fresh variables for each iteration, instead of mutating a shared copy.
- beagle3 7y agoThe real WTF here is that you are getting a power output from thee ^ operator, when it actually does xor. Indeed, I've never seen JavaScript mix up operators :) /s
- xscott 7y agoEven more fundamental is the nonlocal/global stuff required in order to avoid declaring your variables. Many people are surprised by what this snippet does: a = 0 def f(): print(a) a = 1 f() f()
- cutler 7y agoAll it does is produce an error: "UnboundLocalError: local variable 'a' referenced before assignment". The second f() call makes no difference.
- xscott 7y agoIs that what you expected from such a simple piece of code?
- Scarblac 7y agoWell it's f that has the problem, rather obviously, and the other three lines are unrelated. So I don't see how this is some especially simple way of triggering that error.
- xscott 7y agoThe other three lines are not unrelated. If you just take the first three lies (and the last one), it's fine: a = 0 def f(): print(a) f() Maybe this is all obvious to you, but I'll bet you can't name any other language which behaves this way. Compare the original Python to JavaScript (or Lua, Perl, Tcl, Ruby, C, Scheme, Rust, Clojure, or ...): a = 0 function f() { console.log(a) a = 1 } f() f() This one behaves how I think most people would initially expect the Python snippet to behave. First it prints 0, then it prints 1. (Of course JavaScript has it's flaws too...) If you don't believe me, ask your coworkers and friends what they think it does before running it. My only real point is that Python conflates variable declaration and variable assignment in a way which initially seems like a friendly time-saver, but which ends up being pretty subtle and confusing until you've learned its quirks. All of that just to avoid declaring your variables (in some fictional version of Python): var a = 0 def f(): print(a) a = 1 f() f() Here it would be clear which are declarations + initializations, and which are only assignments. And for just the cost of typing the word "var", the compiler could tell you when you've made typos in your variable names. As an added bonus, you could get rid of the "global" and "nonlocal" keywords.
- deleted 7y ago[deleted]
- Kwantuum 7y agoYou can achieve the same thing in python by using two lambdas (which is actually what you're doing in JS with map): >>> powers_of_x = [(lambda i: (lambda x: x**i))(i) for i in range(10)] >>> [f(2) for f in powers_of_x] [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] Or you can use default parameter values to use a single lambda (though this means it can be overridden, it's not semantically equivalent to the js implementation) >>> powers_of_x = [lambda x, i=i: x**i for i in range(10)] >>> [f(2) for f in powers_of_x] [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] your python snippet is equivalent to the following JS: >>> function* comprehension(iterable){ >>> let i; >>> for(let j of iterable){ >>> i = j; >>> yield x => x**i; >>> } >>> } >>> powersOfX = [...comprehension([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])]; >>> powersOfX.map(f => f(2)) [512, 512, 512, 512, 512, 512, 512, 512, 512, 512] And Python's comprehensions are actually a shorthand for writing the above generator with this syntax: yieldExpression for i in iterable. The semantics are consistent with the regular for..in >>> def comprehension(iterable): >>> for i in iterable: >>> yield lambda x: x**i >>> [*comprehension(range(10))] is equivalent to >>> [lambda x: x**i for i in range(10)] This is all a consequence of python's late binding.
- Izkata 7y agoAmusingly, the "default parameter" version relies on a feature that also gives rise to another WTPython entry: https://github.com/satwikkansal/wtfpython/blob/master/README.md#-beware-of-default-mutable-arguments https://github.com/satwikkansal/wtfpython/blob/master/README...
- sjagoe 7y agoThat's a different issue around default arguments that are mutable, rather than default arguments generally. The lambda case here is more like a default argument of a constant integer (immutable) than a default that is a list, which can be altered directly by the function it's a default of.
- Izkata 7y agoThe underlying feature that both rely on is that default arguments are set on function declaration instead of function call. Mutable/immutable only affects whether or not you hit the bug.
- deleted 7y ago[deleted]
- fxj 7y agoYou forgot the i=i in your lambda definition. Then there exists a local variable i and everything works as you wanted.
- faizshah 7y agoThis situation is mentioned in the docs: https://docs.python.org/3.8/faq/programming.html#why-do-lambdas-defined-in-a-loop-with-different-values-all-return-the-same-result https://docs.python.org/3.8/faq/programming.html#why-do-lamb... "This happens because i is not local to the lambdas, but is defined in the outer scope, and it is accessed when the lambda is called — not when it is defined. At the end of the loop, the value of i is 4, so all the functions now return 42, i.e. 16." These sort of scoping gotchas are pretty common across all programming languages and are a great argument for unit testing. As usual the answer is be more explicit what you're asking the language to do: >>> powers_of_x = [lambda x, i=i: x*i for i in range(10)] >>> [f(2) for f in powers_of_x] [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] Note the positional argument with default "i=i." More discussion: https://stackoverflow.com/questions/452610/how-do-i-create-a-list-of-python-lambdas-in-a-list-comprehension-for-loop https://stackoverflow.com/questions/452610/how-do-i-create-a...
- b0rsuk 7y ago>My biggest complaint about Python is that it somehow doesn’t get flak for having the same (if not worse) scoping as JS, which gets endless hate for its function-scoped variables. (So much so that block scoped variables are the new normal in JS, but not in Py!) It's less of a problem in Python because it has built-in module support. A global variable is only global in the module, unless you import it with "from foo import *", which lazy programmers do but is discouraged by style guides.
- intrepidhero 7y agoIn a decade of python use I can probably count on one hand the number of times lambda has been a good solution to a problem I had. Many times I end up re-writing them as regular functions for clarity sake.
- xscott 7y agoWhat's the symptom and what's the cause? Python's lambdas aren't very good, so they're frequently not a very good solution...
- watersb 7y ago"Excess syntactic sugar causes cancer of the semicolon." GJS: Gerry Sussman attributes this quote to Alan Perlis. _Structure and Interpretation of Computer Programs_: https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-10.html#footnote_Temp_17 https://mitpress.mit.edu/sites/default/files/sicp/full-text/...