11 ms·

The upcoming iterator design for Go 1.23

134 points by toprerules 2y ago

mikemitchelldev 2y ago
I’m happy about the inclusion of Iterators in Go. Even if I don’t write Iterators, it makes reading code more challenging and it prepares me to read other languages with Iterators
- asp_hornet 2y ago
  That is a terrible reason for a feature in a language.
  - mikemitchelldev 2y ago
    It's a terrible reason to add a feature in a language. But I'm still happy it's there for that reason.
vundercind 2y ago
I get iterators of this yield-sort, in other languages. I still grumble every time I have to implement one. Something about the interface just bothers me. Feels sloppy.
- golergka 2y ago
  That's the nature of the craft. If you go down the road of eradicating all the things that are a little bit sloppy and inelegant, you eventually end up with something like Haskell (with a lot of extensions) and lose most of other developers along the way.
  - vundercind 2y ago
    See, iterators [edit: and “yield” in general] feel Haskelly to me. Then again I can only understand key Haskell concepts when they’re explained using any of several other languages (then they’re typically very easy to understand) so maybe I’m not the best judge of what is or is not Haskell-like.
    - remexre 2y ago
      Kinda funny, given that the Haskell iterator solution is "convert your data to be a list, and the optimizer and RTS will do the right thing as if it were an iterator."
  - xiao0 2y ago
    that sounds like a good thing
    - golergka 2y ago
      Right until you have to hire.
38 2y ago
jesus. both of those examples are awful. I dont mean that as a statement against the author, I mean it as a statement to the language designers. people are gonna start writing code like this, or worse, and Go is gonna end up like many other unreadable languages. shame.
- runarberg 2y ago
  Maybe it is just me, but I have a very hard time reading Go in general. I have a hard time with e.g. type annotations without a colon or an arrow. The := operator does not read fluently, and I always have to think about what it does to the variable binding (in python the := reads just fine). Also, whenever I encounter multiple returns it feels very cluttered and I have to stop and think what the heck is going on. And finally the the capitalization of variable does horrors to my understanding of the code that I’m reading, I know what it is for, but as I’m reading code, if a function being used is an imported from somewhere else is not a meaningful information, and just confuses me. I bet there is more, but these are some of the examples I’m consciously aware of
  - dullcrisp 2y ago
    They just sounds like ways it’s different from the language you’re using currently?
  - xyzzy_plugh 2y ago
    I could have written this post, except for exactly everything would be inverted.
  - CharlieDigital 2y ago
    It's not just you. There is something about Go that feels "off" for me as well. I started picking it up but couldn't get past some of the syntax.
- duskwuff 2y ago
  TBH, a lot of the complications in Backward() are because it's dealing with two iterators (one as an input, one as its output) and the input/output are both generics. This isn't a typical case.
adeptima 2y ago
Nice write-up! At my personal level, I feel nothing about Iterators or even Generic. I can live with or without it, and kind of dont mind it to be used by library authors. I didnt had a single line of code broken by golang version up updates in 7+ years, scanning through golang source code is always a pleasure. No hidden magic. I dont read documention either - just surfing inside of source code by clicking with CMD key. My main KPI for Golang is the ammount of clicks required to understand internals ... very few compare to other languages. If its stamped by guys like Russ Cox (rsc), I can sleep well too.
- 38 2y ago
  > No hidden magic thats the whole point of the article. this change trashes that. now many loops are going to have hidden magic
  - adeptima 2y ago
    I guess my system got immuned to such things after TS/JS coding, but if 20% of community will say it has "hidden magic" I'm not excited too. Fast learning curve for new hires is a top golang feature for me too.
  - unscaled 2y ago
    Go always had hidden magic: - Lowercase symbols are private to the package they are declared at. - A function called "init" will get executed implicitly on startup. - You can have multiple functions called "init" in the same package or even in the same file. - Files ending with "_unix.go", "_linux.go", "_windows.go" etc. will only be compiled when compiling for the specified platform. The exact list of platform is very hard to find in documentation. - There are a handful of magical built-in function that have lowercase names. - Some built-ins were generic long before generics were introduced to the language. - The copy function is defined as "copy(dst, src []Type) int" (i.e. both src and dst have to be slices of the same type), but "As a special case, it also will copy bytes from a string to a slice of bytes". There are many cases of magical behavior in classic, pre-Generic Go. Sure, if you read the documentation you can learn about these thing and you wouldn't be surprised (although good luck figuring out the order of calling init() or finding out what happened when Go introduces a new compilation target platform that happens to have the same name as the ending of one of your source files). The thing is, if you read the documentation about iterators and see that your iterating over the results of a function that returns a another function (rather than a slice, string, map or channel), you also wouldn't be surprised. And unlike the old magic, the new magic is always transparent. You can always tell what an iterator function by looking at its source code. But you can't exactly tell what suffixes trigger platform-specific compilation without looking at the Go compiler source code, and knowing where to look.
klabb3 2y ago
I’m using Go daily. Usually new features in Go are addressing some real need or pain point, or standardizing on something that’s become fragmented in the 3p ecosystem. So I’m a little confused with the proposal – maybe I’m living in a parallel Go universe, but I don’t think I have ever needed or even wanted iterators. Can someone point to a real world use case where these iterators really make things better?
- MarkMarine 2y ago
  It’s called out in the proposal, but generics allow custom container types to be created like ordered map. Iterators allow nice methods to be created by designers of these containers types, and used in a way that is similar to the go collections like map and slice. Mentioned earlier in the comments, someone brought up in order tree traversal as well. I don’t see an issue having the for range syntax that works magically for go std lib containers extended to custom containers, especially since go std lib omits so many valuable containers
- skldj28d2 2y ago
  Basically every custom data structure right now has some custom implementation of iterators. This will set a standard and make them usable with range loops. Even simple library methods like scanner.Scan, strings.Split, regex.FindAll or sql.Query should return iterators.
  - 38 2y ago
    > scanner.Scan, strings.Split, regex.FindAll or sql.Query should return iterators Will never happen. Best you can hope for is new functions
    - skldj28d2 2y ago
      I know. There are already proposals to add new functions to many of these packages that return iterators.
      - 2y ago
fsmv 2y ago
Why do people focus so much on that random side comment from Pike during one talk he gave a long time ago? It's hardly the language philosophy and it's really hard to believe that Pike actually thinks Google programmers are bad or even average given that he has seen the hiring process.
- gaganyaan 2y ago
  Because it's straight from the horse's mouth, and it does in fact capture Go's philosophy perfectly. Go is, and was designed to be, a blub language.
- Anon4Now 2y ago
  > The key point here is our programmers are Googlers, they’re not researchers. I take that to mean that they're searchers, not researchers. I.e., "Googlers" here means they use Google to search for answers, not that they're Google employees.
  - eru 2y ago
    Inside of Google the term 'Googler' unambiguously refers to co-workers; not to people who just happen to use Google products. (Source: I used to work there.)
  - jkrejcha 2y ago
    In this context, Googlers does in fact mean Google employees. The context from the surrounding talk[1] talks about Google's use cases such as concurrency being a key part of the language, etc. [1]: https://www.youtube.com/watch?v=iTrP_EmGNmw https://www.youtube.com/watch?v=iTrP_EmGNmw (quote is at 20:30)
- elgenie 2y ago
  Pike's comment gets a lot of play because it happens to have great explanatory power for observed language design choices. It not "hardly the language philosophy", it's the language philosophy stripped of the ego-fluffing layer of marketing to language end-users.
elzbardico 2y ago
It's kind of funny how the church of absolute simplicity above all things have became so dominant. As someone who has seen the cost of C++ templates metaprogramming wizardry gone wild I understand part of this reaction. I reckon we would have a real talent shortage bigger by orders of magnitude if we required familiarity with Alexandrescus' Modern C++ Programming and Herb Sutter's Exceptional C++ to onboard new programmers in a project. Yeah, definitelly there's such a thing in programming as "too clever". But should we so radically into the opposite direction? Should we sacrifice any attempt at expressiveness into the altar of simplicity? Aren't we going too far into the opposite direction?
- illusive4080 2y ago
  Yes we are going too far. Go treats us like we are inept.
  - doctor_eval 2y ago
    I dunno, I think go simply treats us as if someone else will be reading the code we write.
    - euroderf 2y ago
      Where "someone else" includes values of "oneself, but in the future".
    - illusive4080 2y ago
      I’m capable of reading someone else’s Node, Python, Rust, C++, and Java. I’ve seen good code and bad code. I’ve also seen good Go code and bad Go code. I know this is what the designers of Go intended, but they accomplished it by dumbing down the language far too much.
- GrantMoyer 2y ago
  In my view, aiming for simplicity is good, but only simplicity of the whole system, and superficial simplicity is often at odds with total simplicity. The following code is composed from a small set of simple operations, transformed = [] for (i = 0; i < len(list); i = i + 1) { el = list[i] temp = transform(el) transformed.append(temp) } but together the simple operations form a complex operation. The effect of each individual line is immediately clear, but the effect of the whole procedure is (marginally, in this toy example) obscured. However, If we extend our small set of simple operations with complex but common operations, we can express complex operations with conciceness that, overall, makes them simpler to recognize and reason about: transformed = list.map(transform) The effect of the line now requires more background knowledge, but in return the effect of the entire procedure becomes more readily apparent.
teeray 2y ago
I guess this means we need to add a “no iterators” linter to golangci-lint.
gyanreyer 2y ago
I like iterators in JavaScript well enough. I understand the syntax of this just fine, it doesn't seem that ridiculous. It's definitely a little less simple than Go usually tends to be, but it's certainly not impenetrable. And you don't have to use it. I personally probably won't use it but I don't care that it's there.
- skldj28d2 2y ago
  They'll be used primarily by library authors but most people will end up consuming them a lot. They'll eliminate a lot of unnecessary intermediate slice allocations.
- frizlab 2y ago
  You don’t have to use it is probably the most common worst argument ever. The mere existence of the construction means it _will_ be used and you will have to know about it and eventually use it.
thomastay 2y ago
Whenever iterator design comes up, I always have to refer to munificent's excellent article: https://journal.stuffwithstuff.com/2013/01/13/iteration-inside-and-out/ https://journal.stuffwithstuff.com/2013/01/13/iteration-insi... tldr: The gold standard is if your iterator syntax / syntaxes can support these two types of iteration: 1. Interleaving two sequences 2. In-order tree traversal
binary132 2y ago
I very much feel that both generics and deps were against the “spirit of Go” (trivial simplicity, stability, manageability) and a reaction to the perceived demands of “The Community” taking precedence over the opinionated, orthodox and even antisocial original design of the language. This just looks like another fluffy, friendly concession to “usability” and “what the crowd wants” at the expense of what set Go apart from the crowd. But hey, maybe I’m just being a curmudgeon and it’ll be great. As the article mentions, there was nothing stopping you from doing this sort of thing (and worse; channels as “iterators”, anyone?) in older versions of Go. But it would be silly to do, and you’d get (rightly) scolded for doing it. Now I guess that kind of thing is encouraged. :)
- MarkMarine 2y ago
  The way "generics" were implemented previously by code generation was just gross, new generics just accept that the standard library didn't make enough containers and probably shouldn't have to make every container type part of the stdlib. Next step to that logic is these iterators, why should the compiler magically support "for range" syntax on slice and map, but not on a map or tree you made. I don't understand the aversion to this, if you don't like it don't use it, but this cleans up the multiple different loop patterns and collapses them into a for range syntax, now you have 1 way to loop instead of multiple ways. The actual implementation of this, making an iterator, is more of a fix for the people developing custom container libraries, which the majority of the go community probably doesn't do. You can go on being a productive go dev and literally never learn this syntax, it won't change your use.
w10-1 2y ago
Go co-routines are the same: a bit of magic you just use and don't think much about. I'd argue the primary magic in large-scale languages should mainly be in those two places: inversion of control and concurrency.
- eru 2y ago
  Why do those two places have a special need for 'magic'? Especially concurrency is hard to get right, so it's probably better to make it as clear as possible as to what's happening? Btw, would garbage collection count as 'magic' in your book?
silisili 2y ago
This design is terrible. It feels extremely bolted on and poorly thought out. Either give us magic or make it supremely readable. This accomplishes neither - it's a barely legible mess of questionable value. To be clear, I'm not against features. But I am against anything that makes code this ugly by default.
phplovesong 2y ago
I dont feel strongly about the PR. I thinks having a common way IS a GOOD thing, as i currently very much dislike the for row.Next() loop (that does not work with a range). Having the option to have ALL iteration work with the range IS a HUGE positive IMHO, no matter of implementation.
kortex 2y ago
Is this just a form of continuation passing style? I get that it's a bit functional-y, and more complex that typical go code (admittedly a very low bar), but I don't think it's too bad. It's no worse than async generators in python, and it's downright elementary compared to anything template metaprogramming. I don't mind the pace that Go has been moving at when it comes to new features. It still feels very conservative while still evolving.
tgv 2y ago
Sure, the syntax looks more terse than the usual Go syntax. That's unfortunate. But the author of the article does point at an important reason to do it like this: > Allow for clean-up with defer When the range syntax would be syntactic sugar for an iterator created as a local variable which handles the state somewhat like this: iter := Iterator{ Next: func() bool {...}, Current: func() T {...}, } for iter.Next() { loopvar := iter.Current() ... } then there's no way (in the current language) to call its clean-up function on panic, whereas it comes naturally in the current proposal. Since these are functions we rarely write, I don't think the syntax will be a problem in practice. Perhaps they could add Yield[T] type to the standard lib to reduce some of the func-iness. The only gripe people may have is when it turns out to be much less efficient than a regular range loop.
neonsunset 2y ago
For touted Go simplicity, this looks messy and hard to read. Look at Rust iterators which are just simple and readable ‘next() -> Option<Item>’ that can be targeted with ‘for item in iter’ and chained iterator expressions or C#’s yield return or IEnumerator<T> (which support cleanup on panic too) which works with foreach and the same kind of chaining. Both are faster, easier to read and write too.
Zababa 2y ago
All of this is specifically about the "Go's Apparent Philosophy" paragraph. Instead of pulling the same quote from Rob Pike over and over and over again, people should watch/read is recent talk about Go titled "What We Got Right, What We Got Wrong": https://commandcenter.blogspot.com/2024/01/what-we-got-right-what-we-got-wrong.html https://commandcenter.blogspot.com/2024/01/what-we-got-right.... A few excerpts: > Given the title of this talk, many people might expect I'm going to be analyzing good and bad things in the language. Of course I'll do some of that, but much more besides, for several reasons. > But the real reason I'm going to talk about more than the language is that that's not what the whole project was about. Our original goal was not to create a new programming language, it was to create a better way to write software. We had issues with the languages we were using—everyone does, whatever the language—but the fundamental problems we had were not central to the features of those languages, but rather to the process that had been created for using them to build software at Google. > The creation of a new language provided a new path to explore other ideas, but it was only an enabler, not the real point. If it didn't take 45 minutes to build the binary I was working on at the time, Go would not have happened, but those 45 minutes were not because the compiler was slow, because it wasn't, or because the language it was written in was bad, because it wasn't. The slowness arose from other factors. > And those factors were what we wanted to address: The complexities of building modern server software: controlling dependencies, programming with large teams with changing personnel, ease of maintainability, efficient testing, effective use of multicore CPUs and networking, and so on. > In short, Go is not just a programming language. Of course it is a programming language, that's its definition, but its purpose was to help provide a better way to develop high-quality software, at least compared to our environment 14 plus years ago. > And that's still what it's about today. Go is a project to make building production software easier and more productive. I will repeat it because I think it is very important: > And that's still what it's about today. Go is a project to make building production software easier and more productive. Another quote about Go being more than the language: > A few weeks back, when starting to prepare this talk, I had a title but little else. To get me going, I asked people on Mastodon for input. A fair few responded, and I noticed a trend in the replies: people thought the things we got wrong were all in the language, but those we got right were in the larger story, the stuff around the language like gofmt and deployment and testing. I find that encouraging, actually. What we were trying to do seems to have had an effect. And finally: > Perhaps the most interesting consequence of these matters is that Go code looks and works the same regardless of who's writing it, is largely free of factions using different subsets of the language, and is guaranteed to continue to compile and run as time goes on. That may be a first for a major programming language. > We definitely got that right. So now that we know better what Go is about, we can try to see why the iterators are added. From https://github.com/golang/go/issues/61405#issuecomment-1638896606 https://github.com/golang/go/issues/61405#issuecomment-16388.... I'll quote some parts: > Can you provide more motivation for range over functions? > The most recent motivation is the addition of generics, which we expect will lead to custom containers such as ordered maps, and it would be good for those custom containers to work well with range loops. > Another equally good motivation is to provide a better answer for the many functions in the standard library that collect a sequence of results and return the whole thing as a slice. If the results can be generated one at a time, then a representation that allows iterating over them scales better than returning an entire slice. We do not have a standard signature for functions that represent this iteration. Adding support for functions in range would both define a standard signature and provide a real benefit that would encourage its use. > There are also functions we were reluctant to provide in slices form that probably deserve to be added in iterator form. For example, there should be a strings.Lines(text) that iterates over the lines in a text. To me this provides enough justification for adding the iterators. But that's not the important part. The important part is listening to what people say about Go, not just "Go the language", but Go as a whole. And listening to what people said recently, not what they said a decade ago. This is how you genuinely engage with other people, I think.

134 points by toprerules 2y ago

I’m happy about the inclusion of Iterators in Go. Even if I don’t write Iterators, it makes reading code more challenging and it prepares me to read other languages with Iterators

asp_hornet 2y ago
That is a terrible reason for a feature in a language.
- mikemitchelldev 2y ago
  It's a terrible reason to add a feature in a language. But I'm still happy it's there for that reason.

vundercind 2y ago

I get iterators of this yield-sort, in other languages. I still grumble every time I have to implement one. Something about the interface just bothers me. Feels sloppy.

golergka 2y ago
That's the nature of the craft. If you go down the road of eradicating all the things that are a little bit sloppy and inelegant, you eventually end up with something like Haskell (with a lot of extensions) and lose most of other developers along the way.
- vundercind 2y ago
  See, iterators [edit: and “yield” in general] feel Haskelly to me. Then again I can only understand key Haskell concepts when they’re explained using any of several other languages (then they’re typically very easy to understand) so maybe I’m not the best judge of what is or is not Haskell-like.
  - remexre 2y ago
    Kinda funny, given that the Haskell iterator solution is "convert your data to be a list, and the optimizer and RTS will do the right thing as if it were an iterator."
- xiao0 2y ago
  that sounds like a good thing
  - golergka 2y ago
    Right until you have to hire.

38 2y ago

jesus. both of those examples are awful. I dont mean that as a statement against the author, I mean it as a statement to the language designers. people are gonna start writing code like this, or worse, and Go is gonna end up like many other unreadable languages. shame.

runarberg 2y ago
Maybe it is just me, but I have a very hard time reading Go in general. I have a hard time with e.g. type annotations without a colon or an arrow. The := operator does not read fluently, and I always have to think about what it does to the variable binding (in python the := reads just fine). Also, whenever I encounter multiple returns it feels very cluttered and I have to stop and think what the heck is going on. And finally the the capitalization of variable does horrors to my understanding of the code that I’m reading, I know what it is for, but as I’m reading code, if a function being used is an imported from somewhere else is not a meaningful information, and just confuses me. I bet there is more, but these are some of the examples I’m consciously aware of
- dullcrisp 2y ago
  They just sounds like ways it’s different from the language you’re using currently?
- xyzzy_plugh 2y ago
  I could have written this post, except for exactly everything would be inverted.
- CharlieDigital 2y ago
  It's not just you. There is something about Go that feels "off" for me as well. I started picking it up but couldn't get past some of the syntax.
duskwuff 2y ago
TBH, a lot of the complications in Backward() are because it's dealing with two iterators (one as an input, one as its output) and the input/output are both generics. This isn't a typical case.

adeptima 2y ago

Nice write-up! At my personal level, I feel nothing about Iterators or even Generic. I can live with or without it, and kind of dont mind it to be used by library authors. I didnt had a single line of code broken by golang version up updates in 7+ years, scanning through golang source code is always a pleasure. No hidden magic. I dont read documention either - just surfing inside of source code by clicking with CMD key. My main KPI for Golang is the ammount of clicks required to understand internals ... very few compare to other languages. If its stamped by guys like Russ Cox (rsc), I can sleep well too.

38 2y ago
> No hidden magic thats the whole point of the article. this change trashes that. now many loops are going to have hidden magic
- adeptima 2y ago
  I guess my system got immuned to such things after TS/JS coding, but if 20% of community will say it has "hidden magic" I'm not excited too. Fast learning curve for new hires is a top golang feature for me too.
- unscaled 2y ago
  Go always had hidden magic: - Lowercase symbols are private to the package they are declared at. - A function called "init" will get executed implicitly on startup. - You can have multiple functions called "init" in the same package or even in the same file. - Files ending with "_unix.go", "_linux.go", "_windows.go" etc. will only be compiled when compiling for the specified platform. The exact list of platform is very hard to find in documentation. - There are a handful of magical built-in function that have lowercase names. - Some built-ins were generic long before generics were introduced to the language. - The copy function is defined as "copy(dst, src []Type) int" (i.e. both src and dst have to be slices of the same type), but "As a special case, it also will copy bytes from a string to a slice of bytes". There are many cases of magical behavior in classic, pre-Generic Go. Sure, if you read the documentation you can learn about these thing and you wouldn't be surprised (although good luck figuring out the order of calling init() or finding out what happened when Go introduces a new compilation target platform that happens to have the same name as the ending of one of your source files). The thing is, if you read the documentation about iterators and see that your iterating over the results of a function that returns a another function (rather than a slice, string, map or channel), you also wouldn't be surprised. And unlike the old magic, the new magic is always transparent. You can always tell what an iterator function by looking at its source code. But you can't exactly tell what suffixes trigger platform-specific compilation without looking at the Go compiler source code, and knowing where to look.

klabb3 2y ago

I’m using Go daily. Usually new features in Go are addressing some real need or pain point, or standardizing on something that’s become fragmented in the 3p ecosystem. So I’m a little confused with the proposal – maybe I’m living in a parallel Go universe, but I don’t think I have ever needed or even wanted iterators. Can someone point to a real world use case where these iterators really make things better?

MarkMarine 2y ago
It’s called out in the proposal, but generics allow custom container types to be created like ordered map. Iterators allow nice methods to be created by designers of these containers types, and used in a way that is similar to the go collections like map and slice. Mentioned earlier in the comments, someone brought up in order tree traversal as well. I don’t see an issue having the for range syntax that works magically for go std lib containers extended to custom containers, especially since go std lib omits so many valuable containers
skldj28d2 2y ago
Basically every custom data structure right now has some custom implementation of iterators. This will set a standard and make them usable with range loops. Even simple library methods like scanner.Scan, strings.Split, regex.FindAll or sql.Query should return iterators.
- 38 2y ago
  > scanner.Scan, strings.Split, regex.FindAll or sql.Query should return iterators Will never happen. Best you can hope for is new functions
  - skldj28d2 2y ago
    I know. There are already proposals to add new functions to many of these packages that return iterators.
    - 2y ago

fsmv 2y ago

Why do people focus so much on that random side comment from Pike during one talk he gave a long time ago? It's hardly the language philosophy and it's really hard to believe that Pike actually thinks Google programmers are bad or even average given that he has seen the hiring process.

gaganyaan 2y ago
Because it's straight from the horse's mouth, and it does in fact capture Go's philosophy perfectly. Go is, and was designed to be, a blub language.
Anon4Now 2y ago
> The key point here is our programmers are Googlers, they’re not researchers. I take that to mean that they're searchers, not researchers. I.e., "Googlers" here means they use Google to search for answers, not that they're Google employees.
- eru 2y ago
  Inside of Google the term 'Googler' unambiguously refers to co-workers; not to people who just happen to use Google products. (Source: I used to work there.)
- jkrejcha 2y ago
  In this context, Googlers does in fact mean Google employees. The context from the surrounding talk[1] talks about Google's use cases such as concurrency being a key part of the language, etc. [1]: https://www.youtube.com/watch?v=iTrP_EmGNmw https://www.youtube.com/watch?v=iTrP_EmGNmw (quote is at 20:30)
elgenie 2y ago
Pike's comment gets a lot of play because it happens to have great explanatory power for observed language design choices. It not "hardly the language philosophy", it's the language philosophy stripped of the ego-fluffing layer of marketing to language end-users.

elzbardico 2y ago

It's kind of funny how the church of absolute simplicity above all things have became so dominant. As someone who has seen the cost of C++ templates metaprogramming wizardry gone wild I understand part of this reaction. I reckon we would have a real talent shortage bigger by orders of magnitude if we required familiarity with Alexandrescus' Modern C++ Programming and Herb Sutter's Exceptional C++ to onboard new programmers in a project. Yeah, definitelly there's such a thing in programming as "too clever". But should we so radically into the opposite direction? Should we sacrifice any attempt at expressiveness into the altar of simplicity? Aren't we going too far into the opposite direction?

illusive4080 2y ago
Yes we are going too far. Go treats us like we are inept.
- doctor_eval 2y ago
  I dunno, I think go simply treats us as if someone else will be reading the code we write.
  - euroderf 2y ago
    Where "someone else" includes values of "oneself, but in the future".
  - illusive4080 2y ago
    I’m capable of reading someone else’s Node, Python, Rust, C++, and Java. I’ve seen good code and bad code. I’ve also seen good Go code and bad Go code. I know this is what the designers of Go intended, but they accomplished it by dumbing down the language far too much.
GrantMoyer 2y ago
In my view, aiming for simplicity is good, but only simplicity of the whole system, and superficial simplicity is often at odds with total simplicity. The following code is composed from a small set of simple operations, transformed = [] for (i = 0; i < len(list); i = i + 1) { el = list[i] temp = transform(el) transformed.append(temp) } but together the simple operations form a complex operation. The effect of each individual line is immediately clear, but the effect of the whole procedure is (marginally, in this toy example) obscured. However, If we extend our small set of simple operations with complex but common operations, we can express complex operations with conciceness that, overall, makes them simpler to recognize and reason about: transformed = list.map(transform) The effect of the line now requires more background knowledge, but in return the effect of the entire procedure becomes more readily apparent.

teeray 2y ago

I guess this means we need to add a “no iterators” linter to golangci-lint.

gyanreyer 2y ago

I like iterators in JavaScript well enough. I understand the syntax of this just fine, it doesn't seem that ridiculous. It's definitely a little less simple than Go usually tends to be, but it's certainly not impenetrable. And you don't have to use it. I personally probably won't use it but I don't care that it's there.

skldj28d2 2y ago
They'll be used primarily by library authors but most people will end up consuming them a lot. They'll eliminate a lot of unnecessary intermediate slice allocations.
frizlab 2y ago
You don’t have to use it is probably the most common worst argument ever. The mere existence of the construction means it _will_ be used and you will have to know about it and eventually use it.

thomastay 2y ago

Whenever iterator design comes up, I always have to refer to munificent's excellent article: https://journal.stuffwithstuff.com/2013/01/13/iteration-inside-and-out/ https://journal.stuffwithstuff.com/2013/01/13/iteration-insi... tldr: The gold standard is if your iterator syntax / syntaxes can support these two types of iteration: 1. Interleaving two sequences 2. In-order tree traversal

binary132 2y ago

I very much feel that both generics and deps were against the “spirit of Go” (trivial simplicity, stability, manageability) and a reaction to the perceived demands of “The Community” taking precedence over the opinionated, orthodox and even antisocial original design of the language. This just looks like another fluffy, friendly concession to “usability” and “what the crowd wants” at the expense of what set Go apart from the crowd. But hey, maybe I’m just being a curmudgeon and it’ll be great. As the article mentions, there was nothing stopping you from doing this sort of thing (and worse; channels as “iterators”, anyone?) in older versions of Go. But it would be silly to do, and you’d get (rightly) scolded for doing it. Now I guess that kind of thing is encouraged. :)

MarkMarine 2y ago
The way "generics" were implemented previously by code generation was just gross, new generics just accept that the standard library didn't make enough containers and probably shouldn't have to make every container type part of the stdlib. Next step to that logic is these iterators, why should the compiler magically support "for range" syntax on slice and map, but not on a map or tree you made. I don't understand the aversion to this, if you don't like it don't use it, but this cleans up the multiple different loop patterns and collapses them into a for range syntax, now you have 1 way to loop instead of multiple ways. The actual implementation of this, making an iterator, is more of a fix for the people developing custom container libraries, which the majority of the go community probably doesn't do. You can go on being a productive go dev and literally never learn this syntax, it won't change your use.

w10-1 2y ago

Go co-routines are the same: a bit of magic you just use and don't think much about. I'd argue the primary magic in large-scale languages should mainly be in those two places: inversion of control and concurrency.

eru 2y ago
Why do those two places have a special need for 'magic'? Especially concurrency is hard to get right, so it's probably better to make it as clear as possible as to what's happening? Btw, would garbage collection count as 'magic' in your book?

silisili 2y ago

This design is terrible. It feels extremely bolted on and poorly thought out. Either give us magic or make it supremely readable. This accomplishes neither - it's a barely legible mess of questionable value. To be clear, I'm not against features. But I am against anything that makes code this ugly by default.

phplovesong 2y ago

I dont feel strongly about the PR. I thinks having a common way IS a GOOD thing, as i currently very much dislike the for row.Next() loop (that does not work with a range). Having the option to have ALL iteration work with the range IS a HUGE positive IMHO, no matter of implementation.

kortex 2y ago

Is this just a form of continuation passing style? I get that it's a bit functional-y, and more complex that typical go code (admittedly a very low bar), but I don't think it's too bad. It's no worse than async generators in python, and it's downright elementary compared to anything template metaprogramming. I don't mind the pace that Go has been moving at when it comes to new features. It still feels very conservative while still evolving.

tgv 2y ago

Sure, the syntax looks more terse than the usual Go syntax. That's unfortunate. But the author of the article does point at an important reason to do it like this: > Allow for clean-up with defer When the range syntax would be syntactic sugar for an iterator created as a local variable which handles the state somewhat like this: iter := Iterator{ Next: func() bool {...}, Current: func() T {...}, } for iter.Next() { loopvar := iter.Current() ... } then there's no way (in the current language) to call its clean-up function on panic, whereas it comes naturally in the current proposal. Since these are functions we rarely write, I don't think the syntax will be a problem in practice. Perhaps they could add Yield[T] type to the standard lib to reduce some of the func-iness. The only gripe people may have is when it turns out to be much less efficient than a regular range loop.

neonsunset 2y ago

For touted Go simplicity, this looks messy and hard to read. Look at Rust iterators which are just simple and readable ‘next() -> Option<Item>’ that can be targeted with ‘for item in iter’ and chained iterator expressions or C#’s yield return or IEnumerator<T> (which support cleanup on panic too) which works with foreach and the same kind of chaining. Both are faster, easier to read and write too.

Zababa 2y ago

All of this is specifically about the "Go's Apparent Philosophy" paragraph. Instead of pulling the same quote from Rob Pike over and over and over again, people should watch/read is recent talk about Go titled "What We Got Right, What We Got Wrong": https://commandcenter.blogspot.com/2024/01/what-we-got-right-what-we-got-wrong.html https://commandcenter.blogspot.com/2024/01/what-we-got-right.... A few excerpts: > Given the title of this talk, many people might expect I'm going to be analyzing good and bad things in the language. Of course I'll do some of that, but much more besides, for several reasons. > But the real reason I'm going to talk about more than the language is that that's not what the whole project was about. Our original goal was not to create a new programming language, it was to create a better way to write software. We had issues with the languages we were using—everyone does, whatever the language—but the fundamental problems we had were not central to the features of those languages, but rather to the process that had been created for using them to build software at Google. > The creation of a new language provided a new path to explore other ideas, but it was only an enabler, not the real point. If it didn't take 45 minutes to build the binary I was working on at the time, Go would not have happened, but those 45 minutes were not because the compiler was slow, because it wasn't, or because the language it was written in was bad, because it wasn't. The slowness arose from other factors. > And those factors were what we wanted to address: The complexities of building modern server software: controlling dependencies, programming with large teams with changing personnel, ease of maintainability, efficient testing, effective use of multicore CPUs and networking, and so on. > In short, Go is not just a programming language. Of course it is a programming language, that's its definition, but its purpose was to help provide a better way to develop high-quality software, at least compared to our environment 14 plus years ago. > And that's still what it's about today. Go is a project to make building production software easier and more productive. I will repeat it because I think it is very important: > And that's still what it's about today. Go is a project to make building production software easier and more productive. Another quote about Go being more than the language: > A few weeks back, when starting to prepare this talk, I had a title but little else. To get me going, I asked people on Mastodon for input. A fair few responded, and I noticed a trend in the replies: people thought the things we got wrong were all in the language, but those we got right were in the larger story, the stuff around the language like gofmt and deployment and testing. I find that encouraging, actually. What we were trying to do seems to have had an effect. And finally: > Perhaps the most interesting consequence of these matters is that Go code looks and works the same regardless of who's writing it, is largely free of factions using different subsets of the language, and is guaranteed to continue to compile and run as time goes on. That may be a first for a major programming language. > We definitely got that right. So now that we know better what Go is about, we can try to see why the iterators are added. From https://github.com/golang/go/issues/61405#issuecomment-1638896606 https://github.com/golang/go/issues/61405#issuecomment-16388.... I'll quote some parts: > Can you provide more motivation for range over functions? > The most recent motivation is the addition of generics, which we expect will lead to custom containers such as ordered maps, and it would be good for those custom containers to work well with range loops. > Another equally good motivation is to provide a better answer for the many functions in the standard library that collect a sequence of results and return the whole thing as a slice. If the results can be generated one at a time, then a representation that allows iterating over them scales better than returning an entire slice. We do not have a standard signature for functions that represent this iteration. Adding support for functions in range would both define a standard signature and provide a real benefit that would encourage its use. > There are also functions we were reluctant to provide in slices form that probably deserve to be added in iterator form. For example, there should be a strings.Lines(text) that iterates over the lines in a text. To me this provides enough justification for adding the iterators. But that's not the important part. The important part is listening to what people say about Go, not just "Go the language", but Go as a whole. And listening to what people said recently, not what they said a decade ago. This is how you genuinely engage with other people, I think.