4 ms·

Is it a bug? I've always depended on late-binding closures and I think even recently in a for loop, not that I'm going to go digging. You can do neat things wit

Is it a bug? I've always depended on late-binding closures and I think even recently in a for loop, not that I'm going to go digging. You can do neat things with multiple functions sharing the same closure. If you don't want the behavior bind the variable to a new name in a new scope. From the post I get the sense that this is more problematic for languages with pointers.

lmm 3y ago
IMO it's a misdesign in the same way as e.g. JavaScript's "this". Most languages figured out 40 or so years ago that scoping should be lexical.
- orbisvicis 3y ago
  The scope is lexical, the lookup is dynamic. What you want is for each loop iteration to create a new scope, which I would categorize as "not lexical".
  - lmm 3y ago
    By that argument a recursive function shouldn't create a new scope every time it recurses, and a language that fails Knuth's 1964 benchmark of reasonable scoping (the "man or boy test") would be fine. The loop body is lexically a block and like any other block it should have its own scope every time it runs.
    - Dylan16807 3y ago
      Also, loop bodies already did have their own scope each iteration. I wouldn't say either behavior is non-lexical. The only thing changing is which lexical scope these variables go into.
      - lmm 3y ago
        If the loop "variable" (and IMO thinking of it as a variable is halfway to making the mistake) is in a single scope whose lifetime is all passes through the loop body, that's literally non-lexical; there is no block in the program text that corresponds to that scope. Lexically there's the containing function and the loop body, there's no intermediate scope nestled between them.
        Dylan16807 3y ago
        > and IMO thinking of it as a variable is halfway to making the mistake I used plural for a reason. > there is no block in the program text that corresponds to that scope. The scope starts at the for. There is a bunch of state that is tied to the loop, and if you rewrote it as a less magic kind of loop you'd need to explicitly mark a scope here. What's non-lexical about it? You could replace "for" with "{ for" to see that a scope of "all passes through the loop body" does not require anything dynamic. And surely whether a scope is implicit or explicit doesn't change whether a scope is lexical. In C I can write "if (1) int x=2;" and that x is scoped to an implicit block that ends at the semicolon. Would you say an if with a declaration in it is non-lexical, because both the true block and the else block can access the variable? I would just say the if has a scope, and there are two scopes inside it, all lexical. And the same of a for loop having an outer and inner scope.
    - orbisvicis 3y ago
      Except that the for loop does not create a new scope and is not a block: https://docs.python.org/3/reference/executionmodel.html#structure-of-a-program https://docs.python.org/3/reference/executionmodel.html#stru...
simiones 3y ago
The problem isn't with closures, the closure semantics are perfectly fine. The problem is in the implementation of for-range loops, where the clear expectation is that the loop variable is scoped to each loop iteration, not to the whole loop scope (otherwise said, that the loop variable is re-bound to a new value in each loop iteration). The mental mode approximately everyone has for a loop like this: for _, v := range values { //do stuff with v } is that it is equivalent to the following loop: for i := range values { v := values[i] //do stuff with v } In Go 1.22 and later, that is exactly what the semantics will be. In Go 1.21 or earlier, the semantics are closer to this (ignoring the empty list case for brevity): for i := 0, v := values[0]; i < len(values); i++, v=values[i] { //do stuff with v } And note that this mis-design has appeared in virtually every language that has loops and closures, and has been either fixed (C# 5.0, Go 1.22) or it keeps being a footgun that people complain about (Python, Common Lisp, C++).
- 4bpp 3y ago
  I don't know, my feeling is that the issue really is with how closure capture was interpreted when imperative languages started implementing lambdas. What was happening in Go seems to either amount to default capture by reference rather than value, or to the loop counters in question being unmarked reference types. The former strikes me as unintuitive given that before lambdas, reference-taking in imperative languages was universally marked (ex. &a); the latter strikes me as unintuitive because with some ugly exceptions (Java), reference types should be marked in usage (ex. *a + *b instead of a+b). Compare to C++ lambdas, where reference captures must be announced in the [] preamble with the & sigil associated with reference-taking. (In functional languages, this problem did not arise, since most variables are immutable and those that are not are syntactically marked and decidedly second-class. In particular, you would probably not implement a loop using a mutable counter or iterator.)
  - simiones 3y ago
    Even if Go allowed both capture-by-value and capture-by-reference, this issue would have arisen when using capture-by-reference. For example, in the following C++: auto v = std::vector<int>{1, 2, 3}; auto prints = std::vector<std::function<void()>>(); auto incrs = std::vector<std::function<void()>>(); for (auto x : v) { prints.push_back([&x]()->void {std::cout<<x<<", "; }) incrs.push_back([&x]()->void {++x;}); } for (auto f : incrs) { f(); } for (auto f : prints) { f(); } //expected to print 2, 3, 4; actually prints 6, 6, 6 I would also note that this problem very much arises in functional languages - it exists in the same way in Common Lisp and Scheme, and I believe it very much applies to OCaml as well (though I'm not sure how their loops work). Tried it out, OCaml does the expected thing: open List let funs = ref [ ] ;; for i = 1 to 3 do funs := (fun () -> print_int i) :: !funs done ;; List.iter (fun f -> f()) !funs ;; //prints 321

lmm 3y ago

IMO it's a misdesign in the same way as e.g. JavaScript's "this". Most languages figured out 40 or so years ago that scoping should be lexical.

orbisvicis 3y ago
The scope is lexical, the lookup is dynamic. What you want is for each loop iteration to create a new scope, which I would categorize as "not lexical".
- lmm 3y ago
  By that argument a recursive function shouldn't create a new scope every time it recurses, and a language that fails Knuth's 1964 benchmark of reasonable scoping (the "man or boy test") would be fine. The loop body is lexically a block and like any other block it should have its own scope every time it runs.
  - Dylan16807 3y ago
    Also, loop bodies already did have their own scope each iteration. I wouldn't say either behavior is non-lexical. The only thing changing is which lexical scope these variables go into.
    - lmm 3y ago
      If the loop "variable" (and IMO thinking of it as a variable is halfway to making the mistake) is in a single scope whose lifetime is all passes through the loop body, that's literally non-lexical; there is no block in the program text that corresponds to that scope. Lexically there's the containing function and the loop body, there's no intermediate scope nestled between them.
      - Dylan16807 3y ago
        > and IMO thinking of it as a variable is halfway to making the mistake I used plural for a reason. > there is no block in the program text that corresponds to that scope. The scope starts at the for. There is a bunch of state that is tied to the loop, and if you rewrote it as a less magic kind of loop you'd need to explicitly mark a scope here. What's non-lexical about it? You could replace "for" with "{ for" to see that a scope of "all passes through the loop body" does not require anything dynamic. And surely whether a scope is implicit or explicit doesn't change whether a scope is lexical. In C I can write "if (1) int x=2;" and that x is scoped to an implicit block that ends at the semicolon. Would you say an if with a declaration in it is non-lexical, because both the true block and the else block can access the variable? I would just say the if has a scope, and there are two scopes inside it, all lexical. And the same of a for loop having an outer and inner scope.
  - orbisvicis 3y ago
    Except that the for loop does not create a new scope and is not a block: https://docs.python.org/3/reference/executionmodel.html#structure-of-a-program https://docs.python.org/3/reference/executionmodel.html#stru...

simiones 3y ago

The problem isn't with closures, the closure semantics are perfectly fine. The problem is in the implementation of for-range loops, where the clear expectation is that the loop variable is scoped to each loop iteration, not to the whole loop scope (otherwise said, that the loop variable is re-bound to a new value in each loop iteration). The mental mode approximately everyone has for a loop like this: for _, v := range values { //do stuff with v } is that it is equivalent to the following loop: for i := range values { v := values[i] //do stuff with v } In Go 1.22 and later, that is exactly what the semantics will be. In Go 1.21 or earlier, the semantics are closer to this (ignoring the empty list case for brevity): for i := 0, v := values[0]; i < len(values); i++, v=values[i] { //do stuff with v } And note that this mis-design has appeared in virtually every language that has loops and closures, and has been either fixed (C# 5.0, Go 1.22) or it keeps being a footgun that people complain about (Python, Common Lisp, C++).

4bpp 3y ago
I don't know, my feeling is that the issue really is with how closure capture was interpreted when imperative languages started implementing lambdas. What was happening in Go seems to either amount to default capture by reference rather than value, or to the loop counters in question being unmarked reference types. The former strikes me as unintuitive given that before lambdas, reference-taking in imperative languages was universally marked (ex. &a); the latter strikes me as unintuitive because with some ugly exceptions (Java), reference types should be marked in usage (ex. *a + *b instead of a+b). Compare to C++ lambdas, where reference captures must be announced in the [] preamble with the & sigil associated with reference-taking. (In functional languages, this problem did not arise, since most variables are immutable and those that are not are syntactically marked and decidedly second-class. In particular, you would probably not implement a loop using a mutable counter or iterator.)
- simiones 3y ago
  Even if Go allowed both capture-by-value and capture-by-reference, this issue would have arisen when using capture-by-reference. For example, in the following C++: auto v = std::vector<int>{1, 2, 3}; auto prints = std::vector<std::function<void()>>(); auto incrs = std::vector<std::function<void()>>(); for (auto x : v) { prints.push_back([&x]()->void {std::cout<<x<<", "; }) incrs.push_back([&x]()->void {++x;}); } for (auto f : incrs) { f(); } for (auto f : prints) { f(); } //expected to print 2, 3, 4; actually prints 6, 6, 6 I would also note that this problem very much arises in functional languages - it exists in the same way in Common Lisp and Scheme, and I believe it very much applies to OCaml as well (though I'm not sure how their loops work). Tried it out, OCaml does the expected thing: open List let funs = ref [ ] ;; for i = 1 to 3 do funs := (fun () -> print_int i) :: !funs done ;; List.iter (fun f -> f()) !funs ;; //prints 321