5 ms·

Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback

by kubb 3y ago

Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback. This could have been prevented by having one person on the team with actual language design experience, who could point this issue out in the design process. In this case, after 10 or so years, and thousands of production bugs, they backpedaled. How many other badly designed features exist in the language, and are simply not being acknowledged? If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost? What if the flaw is more subtle, and explaining why it's bad is harder than in this very simple case, that can be illustrated with 5 lines of code? What if the link between it and its consequences isn't that clear, but the consequences are just as grave? Will it ever get fixed?

badrequest 3y ago
Are you suggesting Rob Pike has no experience designing a programming language?
- ziyao_w 3y ago
  I think the parent was trying to imply that Ken Thompson had no experience in designing a programming language :-) Seriously though, "having experience" and "getting things right" are two different things, although Golang got a lot of things right, and the parent is being unnecessarily harsh.
- ranting-moth 3y ago
  Or Ken Thompson?
- kubb 3y ago
  I should say language design knowledge.
  - Cthulhu_ 3y ago
    Are you aware who these people are?
Patrickmi 3y ago
So whats your point ?, old ideas never die ?, language design is not language purpose and goal ?, they made a mistake creating Go ?, refusing to find something suitable or just break compatibility?
scottlamb 3y ago
Others have suggested that Rob Pike and Ken Thompson have some language design experience, to state it mildly. I also want to point out... > Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback. I think "user feedback" isn't the whole story. It's not just the Go team passively listening as users point out obvious flaws. I've noticed in other changes (e.g. the monotonic time change [1]) the Go team has done a pretty disciplined study of user code in Google's monorepo and on github. That's mentioned in this case too. This is a good practice, not evidence of failure. [1] https://go.googlesource.com/proposal/+/master/design/12914-monotonic.md#appendix_time_now-usage https://go.googlesource.com/proposal/+/master/design/12914-m...
- kaba0 3y ago
  > Rob Pike and Ken Thompson They are huge names in the field, but honestly, they just suck at language design itself.
  - Cthulhu_ 3y ago
    Can you substantiate that or is this just flaming?
    - scottlamb 3y ago
      I'm sure they could come up with a list of language decisions they disagree with. I'm equally sure that if you asked kubb, kaba0, and three other strongly opinionated folks for a list of good language designers, each of the <5 lists you get back would be very short, and there'd be no overlap between them.
AaronFriel 3y ago
Many languages have made this mistake, despite having engineers and teams with many decades or centuries of total experience working on programming languages. Almost all languages have the loop variable semantics Go chose: C/C++, Java, C# (until 5.0), JavaScript (when using `var`), Python. Honestly: are there any C-like, imperative languages with for loops, that _don't_ behave like this? That decision only becomes painful when capturing variables by reference becomes cheap and common; that is, when languages introduce lightweight closures (aka lambdas, anonymous functions, ...). Then the semantics of a for loop subtly change. Language designers have frequently implemented lightweight closures before realizing the risk, and then must make a difficult choice of whether to take a painful breaking change. The Go team can be persuaded, it's just a tall order. And give them credit where credit is due: this is genuinely a significant, breaking change. It's the right change, but it's not an easy decision to make more than a decade into a language's usage. That said, there may be a kernel of truth to what you're alluding to: that the Go team can be hard to persuade and has taken some principled (I would argue, wrong) positions. I'm tracking several Go bugs myself where I believe the Go standard library behaves incorrectly. But I don't think this situation is the right one to make this argument.
- adra 3y ago
  This isn't a bug in java. Java has the idea of "effectively final" variables, and only final or effectively final values are allowed to be passed into lambdas seemingly to avoid this specific defect. Ironically, I just had a review the other day that touched on this go "interaction". The outcome of this go code in java would be as you'd expect, each lambda generated uses a unique copy of the loop reference value.
  - AaronFriel 3y ago
    Oh, today I learned. I think this was an issue in Scala (with `var`), but this seems like a great compromise for Java core. I suppose Java had many years after C#'s introduction of closures to reflect on what went well and what did not. Go, created in 2007, predates both languages having lightweight closures. Not surprising that they made the decision they did. Your comment inspired me to ask what Rust does in this situation, but of course, they've opted for both a different "for" loop construct, but even if they hadn't, the borrow checker enforces a similar requirement as Java's effectively final lambda limitation.
randomdata 3y ago
> Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback. Since "Go 1" was deemed complete and the "Go 2" project began in 2018, the direction of the language was given to the community. It is no longer the Go team's place to shove whatever they think into the language. It has to be what the community wants, and that feedback showed it is what they want.
- Cthulhu_ 3y ago
  The Go team never shoved anything into the language without good reason, and they will not allow the community to shove anything into the language; that's how we got half baked classes in Javascript and half baked functional programming in Java, or the overall trend of languages taking features from other languages because community members say "this language would be better if it had features from this other language" often enough.
  - randomdata 3y ago
    The "Go 1" project was centred around the Go Team. They built what they wanted and needed with little regard for the rest of the world. If they felt loop variable capturing was important, they would have added it. Of course, they didn't find it necessary, so it wasn't added. When "Go 1" reached its natural stopping point and closed down, the "Go 2" project emerged to continue development of Go under the wants and needs of the community. That capture is being added now because the community has shown a desire to have it. The Go Team may use their expertise to guide the community in the right direction, but we are here because the "Go 2" project is community driven. The original commenter seems unaware that the project changed hands.
jacquesm 3y ago
> This could have been prevented by having one person on the team with actual language design experience I thought you were serious, right up to that bit. Well played. I hope.
alecbz 3y ago
> This could have been prevented by having one person on the team with actual language design experience, who could point this issue out in the design process. Instead of making a mistake, they could have simply not. See also RFC 9225: Software Defects Considered Harmful https://www.rfc-editor.org/rfc/rfc9225.html https://www.rfc-editor.org/rfc/rfc9225.html
akira2501 3y ago
> How many other badly designed features exist in the language, and are simply not being acknowledged? Very few. > If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost? If you're right yet don't have a better idea then what do you expect to occur? > What if the link between it and its consequences isn't that clear, but the consequences are just as grave? The consequence is your developers must be careful with loop variables or they will introduce bugs. That's not particularly "grave" nor even especially novel. I'll admit, it's not a good ivory tower language, but then again, that's probably why I use it so often. It gets the job done and it doesn't waste my time with useless hypothetical features.
rsc 3y ago
I answered some of this above: https://news.ycombinator.com/item?id=37577312 https://news.ycombinator.com/item?id=37577312
amomchilov 3y ago
I can’t find it now, but I remember some joke about “it’s an interesting language, but why did you ignore the last 50 years of programming language design?” I find Go quite frustrating in how it decries how over-complicated some features are, and slowly comes around to realize that oh, maybe people designed them for a reason (who woulda thunk it?).
deleted 3y ago
[deleted]
mrkeen 3y ago
> If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost? Is this a subtle nod to the billion dollar mistake? Because they deliberately included the billion dollar mistake as part of the language. Even if they knew better than to include the billion dollar mistake, they were probably aware that they couldn't make a popular language without including it. Because most users demand it.

by kubb 3y ago

badrequest 3y ago

Are you suggesting Rob Pike has no experience designing a programming language?

ziyao_w 3y ago
I think the parent was trying to imply that Ken Thompson had no experience in designing a programming language :-) Seriously though, "having experience" and "getting things right" are two different things, although Golang got a lot of things right, and the parent is being unnecessarily harsh.
ranting-moth 3y ago
Or Ken Thompson?
kubb 3y ago
I should say language design knowledge.
- Cthulhu_ 3y ago
  Are you aware who these people are?

Patrickmi 3y ago

So whats your point ?, old ideas never die ?, language design is not language purpose and goal ?, they made a mistake creating Go ?, refusing to find something suitable or just break compatibility?

scottlamb 3y ago

Others have suggested that Rob Pike and Ken Thompson have some language design experience, to state it mildly. I also want to point out... > Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback. I think "user feedback" isn't the whole story. It's not just the Go team passively listening as users point out obvious flaws. I've noticed in other changes (e.g. the monotonic time change [1]) the Go team has done a pretty disciplined study of user code in Google's monorepo and on github. That's mentioned in this case too. This is a good practice, not evidence of failure. [1] https://go.googlesource.com/proposal/+/master/design/12914-monotonic.md#appendix_time_now-usage https://go.googlesource.com/proposal/+/master/design/12914-m...

kaba0 3y ago
> Rob Pike and Ken Thompson They are huge names in the field, but honestly, they just suck at language design itself.
- Cthulhu_ 3y ago
  Can you substantiate that or is this just flaming?
  - scottlamb 3y ago
    I'm sure they could come up with a list of language decisions they disagree with. I'm equally sure that if you asked kubb, kaba0, and three other strongly opinionated folks for a list of good language designers, each of the <5 lists you get back would be very short, and there'd be no overlap between them.

AaronFriel 3y ago

Many languages have made this mistake, despite having engineers and teams with many decades or centuries of total experience working on programming languages. Almost all languages have the loop variable semantics Go chose: C/C++, Java, C# (until 5.0), JavaScript (when using `var`), Python. Honestly: are there any C-like, imperative languages with for loops, that _don't_ behave like this? That decision only becomes painful when capturing variables by reference becomes cheap and common; that is, when languages introduce lightweight closures (aka lambdas, anonymous functions, ...). Then the semantics of a for loop subtly change. Language designers have frequently implemented lightweight closures before realizing the risk, and then must make a difficult choice of whether to take a painful breaking change. The Go team can be persuaded, it's just a tall order. And give them credit where credit is due: this is genuinely a significant, breaking change. It's the right change, but it's not an easy decision to make more than a decade into a language's usage. That said, there may be a kernel of truth to what you're alluding to: that the Go team can be hard to persuade and has taken some principled (I would argue, wrong) positions. I'm tracking several Go bugs myself where I believe the Go standard library behaves incorrectly. But I don't think this situation is the right one to make this argument.

adra 3y ago
This isn't a bug in java. Java has the idea of "effectively final" variables, and only final or effectively final values are allowed to be passed into lambdas seemingly to avoid this specific defect. Ironically, I just had a review the other day that touched on this go "interaction". The outcome of this go code in java would be as you'd expect, each lambda generated uses a unique copy of the loop reference value.
- AaronFriel 3y ago
  Oh, today I learned. I think this was an issue in Scala (with `var`), but this seems like a great compromise for Java core. I suppose Java had many years after C#'s introduction of closures to reflect on what went well and what did not. Go, created in 2007, predates both languages having lightweight closures. Not surprising that they made the decision they did. Your comment inspired me to ask what Rust does in this situation, but of course, they've opted for both a different "for" loop construct, but even if they hadn't, the borrow checker enforces a similar requirement as Java's effectively final lambda limitation.

randomdata 3y ago

> Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback. Since "Go 1" was deemed complete and the "Go 2" project began in 2018, the direction of the language was given to the community. It is no longer the Go team's place to shove whatever they think into the language. It has to be what the community wants, and that feedback showed it is what they want.

Cthulhu_ 3y ago
The Go team never shoved anything into the language without good reason, and they will not allow the community to shove anything into the language; that's how we got half baked classes in Javascript and half baked functional programming in Java, or the overall trend of languages taking features from other languages because community members say "this language would be better if it had features from this other language" often enough.
- randomdata 3y ago
  The "Go 1" project was centred around the Go Team. They built what they wanted and needed with little regard for the rest of the world. If they felt loop variable capturing was important, they would have added it. Of course, they didn't find it necessary, so it wasn't added. When "Go 1" reached its natural stopping point and closed down, the "Go 2" project emerged to continue development of Go under the wants and needs of the community. That capture is being added now because the community has shown a desire to have it. The Go Team may use their expertise to guide the community in the right direction, but we are here because the "Go 2" project is community driven. The original commenter seems unaware that the project changed hands.

jacquesm 3y ago

> This could have been prevented by having one person on the team with actual language design experience I thought you were serious, right up to that bit. Well played. I hope.

alecbz 3y ago

> This could have been prevented by having one person on the team with actual language design experience, who could point this issue out in the design process. Instead of making a mistake, they could have simply not. See also RFC 9225: Software Defects Considered Harmful https://www.rfc-editor.org/rfc/rfc9225.html https://www.rfc-editor.org/rfc/rfc9225.html

akira2501 3y ago

> How many other badly designed features exist in the language, and are simply not being acknowledged? Very few. > If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost? If you're right yet don't have a better idea then what do you expect to occur? > What if the link between it and its consequences isn't that clear, but the consequences are just as grave? The consequence is your developers must be careful with loop variables or they will introduce bugs. That's not particularly "grave" nor even especially novel. I'll admit, it's not a good ivory tower language, but then again, that's probably why I use it so often. It gets the job done and it doesn't waste my time with useless hypothetical features.

rsc 3y ago

I answered some of this above: https://news.ycombinator.com/item?id=37577312 https://news.ycombinator.com/item?id=37577312

amomchilov 3y ago

I can’t find it now, but I remember some joke about “it’s an interesting language, but why did you ignore the last 50 years of programming language design?” I find Go quite frustrating in how it decries how over-complicated some features are, and slowly comes around to realize that oh, maybe people designed them for a reason (who woulda thunk it?).

deleted 3y ago

[deleted]

mrkeen 3y ago

> If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost? Is this a subtle nod to the billion dollar mistake? Because they deliberately included the billion dollar mistake as part of the language. Even if they knew better than to include the billion dollar mistake, they were probably aware that they couldn't make a popular language without including it. Because most users demand it.