4 ms·

I'm an experienced Go programmer and still accidentally do this every so often. It certainly bit me and my team several times when we were newish to Go. And par

I'm an experienced Go programmer and still accidentally do this every so often. It certainly bit me and my team several times when we were newish to Go. And particularly for newbies, it's hard to debug and understand when you do run into it. All of which to say I'm really glad they're trying to fix this.

tezza 4y ago
Yep, bitten by this one on repeat myself. definitely a WTF moment each time.
- kubb 4y ago
  same here, happened several times when launching a goroutine inside the loop the fix was usually adding x := x before go func() { do something with x }
  - masklinn 4y ago
    A common alternative fix is to use IIFE, since the expression must be a call. `go` evaluates everything but the final function call in the context of the caller. So go func (x int) { … } (x) Will do the same, and is easier to extract to a named function.
    - Joker_vD 4y ago
      The latter is also easier to re-inline accidentally because go func (x int) { do_work(x) } (x) seems like a very roundabout way to say go do_work(x) The "x := x" solution too suffers from this problem but slightly less: both idioms look like they are no-ops (while they are actually not) but at least "x := x" is weird enough to look like it was a deliberate choice, not some vestige from refactoring.
      - masklinn 4y ago
        Er… it makes no difference? Your first version is in fact a worse way to write the second one.
        Joker_vD 4y ago
        Huh. Now the closure-capturing in Go makes even less to me: instead of capturing the variable's current value it captures the variable itself i.e. puts &x into the closure instead of x — and no other piece of Go does that, although I was sure "go fun(args)" passed args by-ref but apparently not. What's even the point of capturing the variable itself? To allow for writing inline callbacks that could sneakily mutate loop-local variables?
        masklinn 4y ago
        More generally allow the closure to write to its environment. That is the normal behaviour of closures in imperative langages, Java being the major exception because it rejects closing over non-final variable (and obviously that only blocks assignment, if the object is mutable you can do what you want to it).
        gpderetta 4y ago
        In c++ close by value/reference is under programmer control and there is no default
        deleted 4y ago
        [deleted]
        randomswede 4y ago
        What would be the use of closing over the value rather than the variable? That would stop a lot of interesting use of closed-over variables (like persisting values from call to call).
        masklinn 4y ago
        In fairness while that would be a lot less convenient in reference-based langages in Go you could just close over a pointer to the variable, making the relationship explicit. That’s how you’d do it using a [=] lambda in c++ or a move closure in rust.
fweimer 4y ago
I've been annoyed more than once by Java's “local variables referenced from a lambda expression must be final or effectively final” error. I used to suspect that this was just an excuse to simplify the javac implementation, but I'm not so sure anymore. It is interesting to see the issue from the other side. The issue is not restricted to variables declared in loop headers, so the proposed loop change for Go might only be the start.
- masklinn 4y ago
  > The issue is not restricted to variables declared in loop headers, so the proposed loop change for Go might only be the start. Technically it’s not but practically it’s by far the most common way for this to unexpectedly arise. The other cases like closing over a variable and then modifying it before the closure is invoked are a lot less common to hit unexpectedly, and a lot harder to fix nicely (short of Java’s big hammer).
- titzer 4y ago
  > I used to suspect that this was just an excuse to simplify the javac implementation, but I'm not so sure anymore. AFAICT it doesn't simplify javac much, if at all. It still needs to synthesize closure objects with fields to store the closed-over values. It's just that those fields can be final. I think Java did this to avoid programmer confusion. I think it was the right choice.
  - twic 4y ago
    It simplifies the implementation immensely, because the fields in those synthetic objects can be populated with copies of the variables from the stack frame. This: void foo() { int a = 1; Runnable r = () -> System.out.println(a); r.run(); // prints 1 } Can get turned into something like this: class r_closure implements Runnable { final int a; r_closure(int a) { this.a = a; } @Override public void run() { System.out.println(a); } } void foo() { int a = 1; Runnable r = new r_closure(a); r.run(); // prints 1 } The local a is a perfectly normal local, and the field a is a perfectly normal field. What would have to happen if the variable was mutable? For example, if you wanted to write this: void foo() { int a = 1; Runnable r = () -> System.out.println(a); a = 2; r.run(); // prints 2 } You have to transform it to something like this: class r_closure implements Runnable { int a; r_closure(int a) { this.a = a; } @Override public void run() { System.out.println(a); } } void foo() { int _a = 1; r_closure r = new r_closure(_a); r.a = 2; r.run(); } Where there is no local, and where the method looks like it's accessing a local, it's actually reaching into the closure and mutating its field! Now think about doing this if you've captured a variable in two closures, or a variable number of closures in a list. The wheels come off this approach. Instead, you would have to promote the shared mutable variable to its own object, like this: class int_box { int i; int_box(int i) { this.i = i; } } class q_closure implements Runnable { final int_box a; q_closure(int_box a) { this.a = a; } @Override public void run() { System.out.println("q = " + a.i); } } class r_closure implements Runnable { final int_box a; r_closure(int_box a) { this.a = a; } @Override public void run() { System.out.println("r = " + a.i); } } void foo() { int_box a = new int_box(1); Runnable q = new q_closure(a); Runnable r = new r_closure(a); a.i = 2; q.run(); // prints 2 r.run(); // prints 2 } Now you've taken a simple local variable which just need to be copied, and turned into its own thing on the heap!
Blikkentrekker 4y ago
It's even worse in python where loops do not even create a new scope and assign on each iteration. From reading this Go creates a scope for the entire loop, but assigns rather than initializes on each iteration. Then again, Python has the same syntax for assignment and initialization.
Cthulhu_ 4y ago
rsc himself mentioned having been bitten by it on quite a few occasions, too.

tezza 4y ago

Yep, bitten by this one on repeat myself. definitely a WTF moment each time.

kubb 4y ago
same here, happened several times when launching a goroutine inside the loop the fix was usually adding x := x before go func() { do something with x }
- masklinn 4y ago
  A common alternative fix is to use IIFE, since the expression must be a call. `go` evaluates everything but the final function call in the context of the caller. So go func (x int) { … } (x) Will do the same, and is easier to extract to a named function.
  - Joker_vD 4y ago
    The latter is also easier to re-inline accidentally because go func (x int) { do_work(x) } (x) seems like a very roundabout way to say go do_work(x) The "x := x" solution too suffers from this problem but slightly less: both idioms look like they are no-ops (while they are actually not) but at least "x := x" is weird enough to look like it was a deliberate choice, not some vestige from refactoring.
    - masklinn 4y ago
      Er… it makes no difference? Your first version is in fact a worse way to write the second one.
      - Joker_vD 4y ago
        Huh. Now the closure-capturing in Go makes even less to me: instead of capturing the variable's current value it captures the variable itself i.e. puts &x into the closure instead of x — and no other piece of Go does that, although I was sure "go fun(args)" passed args by-ref but apparently not. What's even the point of capturing the variable itself? To allow for writing inline callbacks that could sneakily mutate loop-local variables?
        masklinn 4y ago
        More generally allow the closure to write to its environment. That is the normal behaviour of closures in imperative langages, Java being the major exception because it rejects closing over non-final variable (and obviously that only blocks assignment, if the object is mutable you can do what you want to it).
        gpderetta 4y ago
        In c++ close by value/reference is under programmer control and there is no default
        deleted 4y ago
        [deleted]
        randomswede 4y ago
        What would be the use of closing over the value rather than the variable? That would stop a lot of interesting use of closed-over variables (like persisting values from call to call).
        masklinn 4y ago
        In fairness while that would be a lot less convenient in reference-based langages in Go you could just close over a pointer to the variable, making the relationship explicit. That’s how you’d do it using a [=] lambda in c++ or a move closure in rust.

fweimer 4y ago

I've been annoyed more than once by Java's “local variables referenced from a lambda expression must be final or effectively final” error. I used to suspect that this was just an excuse to simplify the javac implementation, but I'm not so sure anymore. It is interesting to see the issue from the other side. The issue is not restricted to variables declared in loop headers, so the proposed loop change for Go might only be the start.

masklinn 4y ago
> The issue is not restricted to variables declared in loop headers, so the proposed loop change for Go might only be the start. Technically it’s not but practically it’s by far the most common way for this to unexpectedly arise. The other cases like closing over a variable and then modifying it before the closure is invoked are a lot less common to hit unexpectedly, and a lot harder to fix nicely (short of Java’s big hammer).
titzer 4y ago
> I used to suspect that this was just an excuse to simplify the javac implementation, but I'm not so sure anymore. AFAICT it doesn't simplify javac much, if at all. It still needs to synthesize closure objects with fields to store the closed-over values. It's just that those fields can be final. I think Java did this to avoid programmer confusion. I think it was the right choice.
- twic 4y ago
  It simplifies the implementation immensely, because the fields in those synthetic objects can be populated with copies of the variables from the stack frame. This: void foo() { int a = 1; Runnable r = () -> System.out.println(a); r.run(); // prints 1 } Can get turned into something like this: class r_closure implements Runnable { final int a; r_closure(int a) { this.a = a; } @Override public void run() { System.out.println(a); } } void foo() { int a = 1; Runnable r = new r_closure(a); r.run(); // prints 1 } The local a is a perfectly normal local, and the field a is a perfectly normal field. What would have to happen if the variable was mutable? For example, if you wanted to write this: void foo() { int a = 1; Runnable r = () -> System.out.println(a); a = 2; r.run(); // prints 2 } You have to transform it to something like this: class r_closure implements Runnable { int a; r_closure(int a) { this.a = a; } @Override public void run() { System.out.println(a); } } void foo() { int _a = 1; r_closure r = new r_closure(_a); r.a = 2; r.run(); } Where there is no local, and where the method looks like it's accessing a local, it's actually reaching into the closure and mutating its field! Now think about doing this if you've captured a variable in two closures, or a variable number of closures in a list. The wheels come off this approach. Instead, you would have to promote the shared mutable variable to its own object, like this: class int_box { int i; int_box(int i) { this.i = i; } } class q_closure implements Runnable { final int_box a; q_closure(int_box a) { this.a = a; } @Override public void run() { System.out.println("q = " + a.i); } } class r_closure implements Runnable { final int_box a; r_closure(int_box a) { this.a = a; } @Override public void run() { System.out.println("r = " + a.i); } } void foo() { int_box a = new int_box(1); Runnable q = new q_closure(a); Runnable r = new r_closure(a); a.i = 2; q.run(); // prints 2 r.run(); // prints 2 } Now you've taken a simple local variable which just need to be copied, and turned into its own thing on the heap!

Blikkentrekker 4y ago

It's even worse in python where loops do not even create a new scope and assign on each iteration. From reading this Go creates a scope for the entire loop, but assigns rather than initializes on each iteration. Then again, Python has the same syntax for assignment and initialization.

Cthulhu_ 4y ago

rsc himself mentioned having been bitten by it on quite a few occasions, too.