4 ms·

The problem isn't with closures, the closure semantics are perfectly fine. The problem is in the implementation of for-range loops, where the clear expectation

The problem isn't with closures, the closure semantics are perfectly fine. The problem is in the implementation of for-range loops, where the clear expectation is that the loop variable is scoped to each loop iteration, not to the whole loop scope (otherwise said, that the loop variable is re-bound to a new value in each loop iteration). The mental mode approximately everyone has for a loop like this: for _, v := range values { //do stuff with v } is that it is equivalent to the following loop: for i := range values { v := values[i] //do stuff with v } In Go 1.22 and later, that is exactly what the semantics will be. In Go 1.21 or earlier, the semantics are closer to this (ignoring the empty list case for brevity): for i := 0, v := values[0]; i < len(values); i++, v=values[i] { //do stuff with v } And note that this mis-design has appeared in virtually every language that has loops and closures, and has been either fixed (C# 5.0, Go 1.22) or it keeps being a footgun that people complain about (Python, Common Lisp, C++).

4bpp 3y ago
I don't know, my feeling is that the issue really is with how closure capture was interpreted when imperative languages started implementing lambdas. What was happening in Go seems to either amount to default capture by reference rather than value, or to the loop counters in question being unmarked reference types. The former strikes me as unintuitive given that before lambdas, reference-taking in imperative languages was universally marked (ex. &a); the latter strikes me as unintuitive because with some ugly exceptions (Java), reference types should be marked in usage (ex. *a + *b instead of a+b). Compare to C++ lambdas, where reference captures must be announced in the [] preamble with the & sigil associated with reference-taking. (In functional languages, this problem did not arise, since most variables are immutable and those that are not are syntactically marked and decidedly second-class. In particular, you would probably not implement a loop using a mutable counter or iterator.)
- simiones 3y ago
  Even if Go allowed both capture-by-value and capture-by-reference, this issue would have arisen when using capture-by-reference. For example, in the following C++: auto v = std::vector<int>{1, 2, 3}; auto prints = std::vector<std::function<void()>>(); auto incrs = std::vector<std::function<void()>>(); for (auto x : v) { prints.push_back([&x]()->void {std::cout<<x<<", "; }) incrs.push_back([&x]()->void {++x;}); } for (auto f : incrs) { f(); } for (auto f : prints) { f(); } //expected to print 2, 3, 4; actually prints 6, 6, 6 I would also note that this problem very much arises in functional languages - it exists in the same way in Common Lisp and Scheme, and I believe it very much applies to OCaml as well (though I'm not sure how their loops work). Tried it out, OCaml does the expected thing: open List let funs = ref [ ] ;; for i = 1 to 3 do funs := (fun () -> print_int i) :: !funs done ;; List.iter (fun f -> f()) !funs ;; //prints 321
  - 4bpp 3y ago
    > this issue would have arisen when using capture-by-reference I understand - but in those languages capture-by-reference has to be an explicit choice (by writing the &) rather than the default, which highlights the actual behaviour. The problem with the old Go solution was that it would apparently behave as capture by reference without any explicit syntactic marker that it is so, and without a more natural alternative that captures by value, in a context where from other languages you would expect that the capture would happen by value. > Common Lisp and Scheme I have to admit I haven't worked in either outside of a tutorial setting, but my understanding is that they are quite well-known for having design choices in variable scoping that are unusual and frowned upon in modern language design > Ocaml Your example shows that it captures by value as I said, right? For it to work as the old Go examples, i would have to be a ref cell whose contents are updated between iterations, which is not how the semantics of for work. If it did, you'd have to use the loop counter as !i.
    - tsimionescu 3y ago
      In Go 1.22 as well, closures still capture-by-reference. The change is that there is now a new loop variable in each loop iteration, just like in OCaml. But two closures that refer the same loop variable (that are created in the same iteration, that is) will still see the changes each makes to that variable. And what I was trying to show with my example was that this kind of behavior would be observable in OCaml as well, if it were to be implemented like that.
orbisvicis 3y ago
I think that's a C-centric assumption which is moot as Python's "for" does not create any new scopes. Just reading Knuth's man-or-boy test I was struck by the alien nature of the ALGOL 60 execution model, even though to Python it can be considered a distant ancestor. https://en.m.wikipedia.org/wiki/Man_or_boy_test https://en.m.wikipedia.org/wiki/Man_or_boy_test

4bpp 3y ago

I don't know, my feeling is that the issue really is with how closure capture was interpreted when imperative languages started implementing lambdas. What was happening in Go seems to either amount to default capture by reference rather than value, or to the loop counters in question being unmarked reference types. The former strikes me as unintuitive given that before lambdas, reference-taking in imperative languages was universally marked (ex. &a); the latter strikes me as unintuitive because with some ugly exceptions (Java), reference types should be marked in usage (ex. *a + *b instead of a+b). Compare to C++ lambdas, where reference captures must be announced in the [] preamble with the & sigil associated with reference-taking. (In functional languages, this problem did not arise, since most variables are immutable and those that are not are syntactically marked and decidedly second-class. In particular, you would probably not implement a loop using a mutable counter or iterator.)

simiones 3y ago
Even if Go allowed both capture-by-value and capture-by-reference, this issue would have arisen when using capture-by-reference. For example, in the following C++: auto v = std::vector<int>{1, 2, 3}; auto prints = std::vector<std::function<void()>>(); auto incrs = std::vector<std::function<void()>>(); for (auto x : v) { prints.push_back([&x]()->void {std::cout<<x<<", "; }) incrs.push_back([&x]()->void {++x;}); } for (auto f : incrs) { f(); } for (auto f : prints) { f(); } //expected to print 2, 3, 4; actually prints 6, 6, 6 I would also note that this problem very much arises in functional languages - it exists in the same way in Common Lisp and Scheme, and I believe it very much applies to OCaml as well (though I'm not sure how their loops work). Tried it out, OCaml does the expected thing: open List let funs = ref [ ] ;; for i = 1 to 3 do funs := (fun () -> print_int i) :: !funs done ;; List.iter (fun f -> f()) !funs ;; //prints 321
- 4bpp 3y ago
  > this issue would have arisen when using capture-by-reference I understand - but in those languages capture-by-reference has to be an explicit choice (by writing the &) rather than the default, which highlights the actual behaviour. The problem with the old Go solution was that it would apparently behave as capture by reference without any explicit syntactic marker that it is so, and without a more natural alternative that captures by value, in a context where from other languages you would expect that the capture would happen by value. > Common Lisp and Scheme I have to admit I haven't worked in either outside of a tutorial setting, but my understanding is that they are quite well-known for having design choices in variable scoping that are unusual and frowned upon in modern language design > Ocaml Your example shows that it captures by value as I said, right? For it to work as the old Go examples, i would have to be a ref cell whose contents are updated between iterations, which is not how the semantics of for work. If it did, you'd have to use the loop counter as !i.
  - tsimionescu 3y ago
    In Go 1.22 as well, closures still capture-by-reference. The change is that there is now a new loop variable in each loop iteration, just like in OCaml. But two closures that refer the same loop variable (that are created in the same iteration, that is) will still see the changes each makes to that variable. And what I was trying to show with my example was that this kind of behavior would be observable in OCaml as well, if it were to be implemented like that.

orbisvicis 3y ago

I think that's a C-centric assumption which is moot as Python's "for" does not create any new scopes. Just reading Knuth's man-or-boy test I was struck by the alien nature of the ALGOL 60 execution model, even though to Python it can be considered a distant ancestor. https://en.m.wikipedia.org/wiki/Man_or_boy_test https://en.m.wikipedia.org/wiki/Man_or_boy_test