5 ms·

How did they even end up with such a crazy behaviour?

mpartel 4y ago
Several languages have or have had this behaviour. C# made a similar change to its 'foreach' loops a long time ago (but not 'for' loops for some reason). I suppose it's the natural way to do it when implementing a language and not thinking about it too much. It makes a for-loop equivalent to a simple while loop with the loop vvariable initialized outside of the loop.
- hardware2win 4y ago
  In c# you mean closure capture?
  - masklinn 4y ago
    Almost certainly, that is the vehicle for the issue in most languages.
- Blikkentrekker 4y ago
  To be fair it is no more difficult to implement it with the variable initialized inside of it. In fact, it's easier to.
- int_19h 4y ago
  The C# change is described in detail in the linked GitHub issue - one of C# devs left a comment explaining it. TL;DR is that both "for" and "foreach" scoping fixes would be breaking changes, but "foreach" was easier to justify because it was already a C#-specific construct syntactically, unlike "for" which uses the same exact syntax as C, Java etc, and they were very sensitive to backwards compatibility at the time (esp. since the tooling didn't have the ability to target various language versions within the same project easily). At the same time, "foreach" represented the vast majority of breakage when they looked at existing code, perhaps because the scoping in classic "for" is more obvious due to the fact that variable mutation is explicit there.
foldr 4y ago
It’s just what happens if you interpret the for loop as modifying a single variable binding on every loop rather than rebinding on every loop. That’s how loops have normally been conceived in imperative languages, but it doesn’t make much of a difference until you add closures (which can capture the binding) or garbage collection (allowing you to capture the address of the ‘local’ variable). The same issue existed with ‘var’ for loops in JS and bit me several times: https://stackoverflow.com/questions/750486/javascript-closure-inside-loops-simple-practical-example https://stackoverflow.com/questions/750486/javascript-closur...
the_mitsuhiko 4y ago
If only this was a go specific problem: >>> def foo(): ... return [lambda: x for x in range(10)] ... >>> [x() for x in foo()] [9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
- randfur 4y ago
  I remember learning about this gotcha around the time that Go was coming out. I was very disappointed that they were sticking to this behaviour around the same time that JavaScript was fixing it. As a new language they get less sympathy from me.
  - lawn 4y ago
    It does feel like Go is relearning lessons of other languages quite often. It's good that this and generics could be fixed later on, but it's unfortunate that they couldn't fix the null pointers (and going further always defaulting to some default value) and error handling.
    - masklinn 4y ago
      > It does feel like Go is relearning lessons of other languages quite often. A common sentiment when Go was unveiled was that its designers had ignored the 35-odd years of language research and experience since C.
- int_19h 4y ago
  Python actually does this right for sequence comprehensions: >>> [f() for f in (lambda: x for x in range(10))] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] List comprehensions have been around for longer, and so behave the way they do for backwards compatibility reasons.
masklinn 4y ago
Technically there is no crazy behaviour, it’s a natural consequence of scoping loop variables outside the loop, which historically was common. It became an issue as lambdas and other lambda-type constructs (which implicitly keep a reference on the loop variable) became more common, and a bunch of languages got caught in it. Later languages switched to the “inner scoping” mechanism to avoid it. Go did not follow the switch because Go.
- nemetroid 4y ago
  This is not really what's happening. The variable is scoped to the loop. The issue is that even if you take the address of the variable, each iteration of the loop will have the same variable with the same address. nums := []int{1, 2, 3} for _, num := range nums { fmt.Printf("%p\n", &num) } This will print the same address three times. If you add "num := num" as the first line in the loop, it will print three different addresses. The proposal is to make this the default behaviour.
  - masklinn 4y ago
    > This is not really what's happening. Yes it is. > The variable is scoped to the loop. Maybe “loop body” would make the comment clearer? > The issue is that even if you take the address of the variable, each iteration of the loop will have the same variable with the same address. Hence the variable not being scoped to the loop (body), one variable is shared between all iterations of the loop.
    - nemetroid 4y ago
      > Maybe “loop body” would make the comment clearer? I think "loop iteration" might.
      - foldr 4y ago
        Arguably not, because an 'iteration' is a unit of execution, not a lexical unit. If the variables were really scoped to loop iterations, that would be a form of dynamic scope, which would have a different semantics. So for example, say the loop calls a function foo. This function executes inside every iteration of the loop, but within foo, one cannot access the loop iteration variable (as it is in a different lexical context).
mmis1000 4y ago
I think it's quite normal. Because the common three part for semantic is just a suger of initVariable() while checkVariable(): doSomethong… updateVariable() So the variable is shared by all iterations at first place. And that isn't a issue until we have lambda or something similar that can capture reference of loop variable. Later languages found it is problematic when use with reference capturing features and changed to something else (one variable per iteration)
beeforpork 4y ago
If you hide pointers and destructively update variables, this happens easily. I'd say the Go problem is a bit worse because there are also exposed pointers via that '&' operator. But the problem should have been known when the language was designed. In CommonLisp, i.e., quite a mighty but old language, it is the same: the capture is on a loop variable that is destructively updated: (setf refs (loop for i in '(1 2 3) collect #'(lambda () i))) -> (#<function) #<function> #<function>) (mapcar #'(lambda (f) (funcall f)) refs) -> (3 3 3) The same done with a fresh function parameter works. And since this is the usual style in Lisp, I suppose the problem will not be that obvious like in Go: (setf refs (mapcar #'(lambda (i) #'(lambda () i)) '(1 2 3))) -> (#<function> #<function> #<function>) (mapcar #'(lambda (f) (funcall f)) refs) -> (1 2 3)
jlouis 4y ago
It is a combination of scoping, which other people already wrote about, and also because the language support mutable data by default. Had the language been immutable/persistent by default, this particular semantic problem cannot occur in the first place. Complexity increases rapidly when you combine constructs in a programming language. You get some feature interactions which are hard to get right, and also to predict.

Several languages have or have had this behaviour. C# made a similar change to its 'foreach' loops a long time ago (but not 'for' loops for some reason). I suppose it's the natural way to do it when implementing a language and not thinking about it too much. It makes a for-loop equivalent to a simple while loop with the loop vvariable initialized outside of the loop.

hardware2win 4y ago
In c# you mean closure capture?
- masklinn 4y ago
  Almost certainly, that is the vehicle for the issue in most languages.
Blikkentrekker 4y ago
To be fair it is no more difficult to implement it with the variable initialized inside of it. In fact, it's easier to.
int_19h 4y ago
The C# change is described in detail in the linked GitHub issue - one of C# devs left a comment explaining it. TL;DR is that both "for" and "foreach" scoping fixes would be breaking changes, but "foreach" was easier to justify because it was already a C#-specific construct syntactically, unlike "for" which uses the same exact syntax as C, Java etc, and they were very sensitive to backwards compatibility at the time (esp. since the tooling didn't have the ability to target various language versions within the same project easily). At the same time, "foreach" represented the vast majority of breakage when they looked at existing code, perhaps because the scoping in classic "for" is more obvious due to the fact that variable mutation is explicit there.

foldr 4y ago

It’s just what happens if you interpret the for loop as modifying a single variable binding on every loop rather than rebinding on every loop. That’s how loops have normally been conceived in imperative languages, but it doesn’t make much of a difference until you add closures (which can capture the binding) or garbage collection (allowing you to capture the address of the ‘local’ variable). The same issue existed with ‘var’ for loops in JS and bit me several times: https://stackoverflow.com/questions/750486/javascript-closure-inside-loops-simple-practical-example https://stackoverflow.com/questions/750486/javascript-closur...

the_mitsuhiko 4y ago

If only this was a go specific problem: >>> def foo(): ... return [lambda: x for x in range(10)] ... >>> [x() for x in foo()] [9, 9, 9, 9, 9, 9, 9, 9, 9, 9]

randfur 4y ago
I remember learning about this gotcha around the time that Go was coming out. I was very disappointed that they were sticking to this behaviour around the same time that JavaScript was fixing it. As a new language they get less sympathy from me.
- lawn 4y ago
  It does feel like Go is relearning lessons of other languages quite often. It's good that this and generics could be fixed later on, but it's unfortunate that they couldn't fix the null pointers (and going further always defaulting to some default value) and error handling.
  - masklinn 4y ago
    > It does feel like Go is relearning lessons of other languages quite often. A common sentiment when Go was unveiled was that its designers had ignored the 35-odd years of language research and experience since C.
int_19h 4y ago
Python actually does this right for sequence comprehensions: >>> [f() for f in (lambda: x for x in range(10))] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] List comprehensions have been around for longer, and so behave the way they do for backwards compatibility reasons.

masklinn 4y ago

Technically there is no crazy behaviour, it’s a natural consequence of scoping loop variables outside the loop, which historically was common. It became an issue as lambdas and other lambda-type constructs (which implicitly keep a reference on the loop variable) became more common, and a bunch of languages got caught in it. Later languages switched to the “inner scoping” mechanism to avoid it. Go did not follow the switch because Go.

nemetroid 4y ago
This is not really what's happening. The variable is scoped to the loop. The issue is that even if you take the address of the variable, each iteration of the loop will have the same variable with the same address. nums := []int{1, 2, 3} for _, num := range nums { fmt.Printf("%p\n", &num) } This will print the same address three times. If you add "num := num" as the first line in the loop, it will print three different addresses. The proposal is to make this the default behaviour.
- masklinn 4y ago
  > This is not really what's happening. Yes it is. > The variable is scoped to the loop. Maybe “loop body” would make the comment clearer? > The issue is that even if you take the address of the variable, each iteration of the loop will have the same variable with the same address. Hence the variable not being scoped to the loop (body), one variable is shared between all iterations of the loop.
  - nemetroid 4y ago
    > Maybe “loop body” would make the comment clearer? I think "loop iteration" might.
    - foldr 4y ago
      Arguably not, because an 'iteration' is a unit of execution, not a lexical unit. If the variables were really scoped to loop iterations, that would be a form of dynamic scope, which would have a different semantics. So for example, say the loop calls a function foo. This function executes inside every iteration of the loop, but within foo, one cannot access the loop iteration variable (as it is in a different lexical context).

mmis1000 4y ago

I think it's quite normal. Because the common three part for semantic is just a suger of initVariable() while checkVariable(): doSomethong… updateVariable() So the variable is shared by all iterations at first place. And that isn't a issue until we have lambda or something similar that can capture reference of loop variable. Later languages found it is problematic when use with reference capturing features and changed to something else (one variable per iteration)

beeforpork 4y ago

If you hide pointers and destructively update variables, this happens easily. I'd say the Go problem is a bit worse because there are also exposed pointers via that '&' operator. But the problem should have been known when the language was designed. In CommonLisp, i.e., quite a mighty but old language, it is the same: the capture is on a loop variable that is destructively updated: (setf refs (loop for i in '(1 2 3) collect #'(lambda () i))) -> (#<function) #<function> #<function>) (mapcar #'(lambda (f) (funcall f)) refs) -> (3 3 3) The same done with a fresh function parameter works. And since this is the usual style in Lisp, I suppose the problem will not be that obvious like in Go: (setf refs (mapcar #'(lambda (i) #'(lambda () i)) '(1 2 3))) -> (#<function> #<function> #<function>) (mapcar #'(lambda (f) (funcall f)) refs) -> (1 2 3)

jlouis 4y ago

It is a combination of scoping, which other people already wrote about, and also because the language support mutable data by default. Had the language been immutable/persistent by default, this particular semantic problem cannot occur in the first place. Complexity increases rapidly when you combine constructs in a programming language. You get some feature interactions which are hard to get right, and also to predict.