5 ms·

It’s actually due to python late binding. The variables are looked up at call time, rather than function define time. One could argue that your example shouldn’

by aidos 7y ago

It’s actually due to python late binding. The variables are looked up at call time, rather than function define time. One could argue that your example shouldn’t work anymore in python3 since variables in list comprehensions go out of scope when they finish now. I haven’t tried it myself. I fill it under, “things I never need to do” :-)

masklinn 7y ago
It's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change. for(var i=0; i<10; ++i) { powersOfX.push(x => x^i); } will behave the same way because it does the same thing, so will e.g. for i := 0; i<10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return x ^ i }) } in Go. One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible). The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over. Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions. In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas: for i in range(10): powers_of_x.append(lambda x, i=i: x^i) [0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings) [1] and remains a regular issue in async code fighting over closed-over context
- gpderetta 7y ago
  C++: std::vector<std::function<int(int)>> powers_of_x; for (int i = 0; i < 10; ++1) power_of_x.push_back([=](int x) { return std::pow(x, i); }); works as one would intuitively expect (a different i is captured for each iteration). The issue is with those languages that conflate values with references.
  - masklinn 7y ago
    … > excluding the special case of low-level languages with capture clauses Of course it works, you're specifically capturing `i` by value. It's not exactly surprising that doing things completely differently yields a different result.
    - gpderetta 7y ago
      To be fair I missed the note at the end of your commend on my first read. Still, in the specific C++ example, no other capture clause is valid. Also, python could have chosen a slightly different closure semantics and preserved sanity: instead of closing over the binding itself, it could close over each object reference separately (exactly in the same way the default parameter hack works).
- aidos 7y ago
  You're right. I remembered the reason after reading your sibling comment and almost deleted my comment to avoid the confusion. Thanks for going into the details.
- loopz 7y ago
  Works intuitively in golang: https://play.golang.org/p/pn2jBTaTNS8 https://play.golang.org/p/pn2jBTaTNS8 package main import ( "fmt" ) // return a^n func Power(a, n int) int { var i, result int result = 1 for i = 0; i < n; i++ { result *= a } return result } func main() { fmt.Println("Hello, playground") x := 2 var powers_of_x []int for i := 0; i < 10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return Power(x, i) }(x)) } fmt.Println(powers_of_x) } //Output: //Hello, playground //[1 2 4 8 16 32 64 128 256 512]
  - gpderetta 7y ago
    I know nothing about go, but it seems to me in your example powers_of_x is a list of n integers, while in the example being discussed is an list of n functions (which can be used to compute the n'th power).
  - masklinn 7y ago
    I provided a Go example demonstrating that it does not. Your version 1. uses a slice of integers which completely misses the point 2. uses an IIFE which completely misses the point The point is to look at what happens when you close over the iteration variable.

by aidos 7y ago

masklinn 7y ago

It's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change. for(var i=0; i<10; ++i) { powersOfX.push(x => x^i); } will behave the same way because it does the same thing, so will e.g. for i := 0; i<10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return x ^ i }) } in Go. One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible). The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over. Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions. In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas: for i in range(10): powers_of_x.append(lambda x, i=i: x^i) [0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings) [1] and remains a regular issue in async code fighting over closed-over context

gpderetta 7y ago
C++: std::vector<std::function<int(int)>> powers_of_x; for (int i = 0; i < 10; ++1) power_of_x.push_back([=](int x) { return std::pow(x, i); }); works as one would intuitively expect (a different i is captured for each iteration). The issue is with those languages that conflate values with references.
- masklinn 7y ago
  … > excluding the special case of low-level languages with capture clauses Of course it works, you're specifically capturing `i` by value. It's not exactly surprising that doing things completely differently yields a different result.
  - gpderetta 7y ago
    To be fair I missed the note at the end of your commend on my first read. Still, in the specific C++ example, no other capture clause is valid. Also, python could have chosen a slightly different closure semantics and preserved sanity: instead of closing over the binding itself, it could close over each object reference separately (exactly in the same way the default parameter hack works).
aidos 7y ago
You're right. I remembered the reason after reading your sibling comment and almost deleted my comment to avoid the confusion. Thanks for going into the details.
loopz 7y ago
Works intuitively in golang: https://play.golang.org/p/pn2jBTaTNS8 https://play.golang.org/p/pn2jBTaTNS8 package main import ( "fmt" ) // return a^n func Power(a, n int) int { var i, result int result = 1 for i = 0; i < n; i++ { result *= a } return result } func main() { fmt.Println("Hello, playground") x := 2 var powers_of_x []int for i := 0; i < 10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return Power(x, i) }(x)) } fmt.Println(powers_of_x) } //Output: //Hello, playground //[1 2 4 8 16 32 64 128 256 512]
- gpderetta 7y ago
  I know nothing about go, but it seems to me in your example powers_of_x is a list of n integers, while in the example being discussed is an list of n functions (which can be used to compute the n'th power).
- masklinn 7y ago
  I provided a Go example demonstrating that it does not. Your version 1. uses a slice of integers which completely misses the point 2. uses an IIFE which completely misses the point The point is to look at what happens when you close over the iteration variable.