4 ms·

What `for x in y` hides from you – From Scratch Code

16 points by rbanffy 4mo ago

WhyNotHugo 4mo ago
'for' loops in Rust do the same: they create an iterator and then iterate over that. You can write the exact same loop with `let mut iter = v.iter(); while Some(x) = iter.next()`. 'for' loops in Rust are purely syntax sugar, and I somewhat wish they didn't exist. They provide you two ways of doing the same thing, but one of them hides the details from you. Having 'for' as a keyword is nice for folks coming from other languages, but then it hides the possibility of other interesting usages, like cloning an iterator inside a loop.
- bigfishrunning 4mo ago
  Which is funny, because whenever I encounter a language for which `for` *doesn't* work this way it feels antiquated. I do however wish another keyword was used in many cases, becuase `for` in C and Go is so much different then `for` in Rust or Python. I think the higher-level case (Rust and Python) should really use a word like `foreach` or maybe something compeletely different like `itr`, although I get that they want to "look" more like C
  - rbanffy 4mo ago
    I would say Python really wants to look as much not-C as it can. In this case it's not the word "for" but the "for x in" construct.
  - b40d-48b2-979e 4mo ago
    This is what PowerShell does, it has both the archaic `for` style with a newer iterator-like `foreach`.
  - adrian_b 4mo ago
    Iterators and the loops of the form "forall X in A do ..." have been invented simultaneously with the appearance of the first version of the language C (i.e. around 1974), in the languages Alphard and CLU. From Alphard and CLU, loops of this form have passed into one of the versions of the language Algol 68 (1977), and from there into the UNIX Bourne shell (1979), which inherited a few Algol 68 syntax features (because its author had worked at writing Algol 68 compilers). Iterators and "forall" loops have begun to spread into mainstream programming languages, e.g. into C++, only a couple of decades after their invention.
- Martinussen 4mo ago
  Is "hiding" in the sense that you just need to have read the docs or know how the language works at a pretty basic level really a problem, or even a negative? I would certainly say that the readability and clarity on the form of loop being used is a bigger win, either way.
- tialaramex 4mo ago
  It's true that they're just sugar but Rust does explain how the for-in loop de-sugars and you've over-simplified considerably. Your syntax also doesn't quite work. The value is in idiom, turning everything into loop expressions (The "while" keyword is also just sugar, Rust's only fundamental loop is named loop) makes it harder to discern what's actually going on. If you want to clone the iterator in some cases rather than consuming it, that should look different so that reviewers will see what you're up to.
- kevinmgranger 4mo ago
  It should be `v.into_iter()`, and that distinction matters because of ownership / move semantics. It just so happens that for most collections, `IntoIter` is also defined for references to them, which typically gives you the same behavior that `.iter()` would give.
  - tialaramex 4mo ago
    More specifically for x in y consumes y in Rust. https://doc.rust-lang.org/std/keyword.for.html https://doc.rust-lang.org/std/keyword.for.html explains how a loop is de-sugared https://rust.godbolt.org/z/5jzhxYM51 https://rust.godbolt.org/z/5jzhxYM51 ... shows that today ranges like 0..5 aren't Copy even if that would be possible, which means if they're consumed they're gone, whereas an array of integers is Copy and so consuming it doesn't mean it's gone, you can just consume it again. The desire is that Rust 2027 edition will change the nice syntax for ranges to produce new ranges like core::ranges::Range which are Copy if possible and only IntoIterator, the original ranges are never Copy but are Iterator, we now regret this choice.
- saghm 4mo ago
  In addition to the points that sibling comments have made about how you're not quite right with the exact semantics of for loops (which also take ownership via `into_iter` and therefore are a bit different from `while let`), it's worth pointing out that if you peek a bit further in to MIR, all loops in Rust just desugar into the `loop` keyword with manual breaking, including `while`. It's not really clear to me why for loops in particular bother you.
anthonj 4mo ago
I don't really get the point of the article. Even if I knew little about python, would be it surpsing that a language with no real basic types is probably abstracting a lot? Even a simple i=0, i=i+1 is "hiding" a lot in python then.
- PaulDavisThe1st 4mo ago
  But it's not a Python thing. Rust is noted below, and there's also C++. std::container<T> container; for (std::container<T>::iterator i = container.begin(); i != container.end(); ++i) { ...} auto iter = container.begin(); while (iter != container.end()) { ...; ++iter; } for (auto const & t : container) { ... }
  - tialaramex 4mo ago
    Barry Revzin has a talk explaining about different kinds of iterator. The C++ iterator is a maximalist, very powerful and flexible idea which - as a result - also has lots of opportunities to set everything on fire. Their model was the C pointer, we can iterate a pointer, compare it against some sentinel "end" value, dereference it, but also we could index into it, decrement the pointer and go backwards, comparing against a "start" value, and we can write to a reference rather than just read from it. This is far more than most iterator designs allow, and as I said if you needed more this is very welcome, but if you didn't it's probably bad news. C++ can sort iterators for example, so long as they're suitable for indexing back and forth as the sort progresses. Rust's sort functions are defined on its slice types, which makes sense in practical terms but if what you've got isn't quite a slice but could meet the requirements in C++ that's a problem for you.
    - PaulDavisThe1st 4mo ago
      It's not the ability to use a maximalist iterator that is of interest here, it's the ability to use or not use any kind of iterator at all.
- courtcircuits 4mo ago
  To be fair, when I started learning CS, the `for x in y` syntax was cryptic to me because I was unfamiliar with concepts such as iterators & generators. `for(int i=0; i<len(y); i++)` made way more sense since there is no hidden logic (besides additions as you highlighted in your comment, but which I think is easier to have a grasp of). So I really wish I had read this article when I started my CS journey a couple of years ago.
mdemare 4mo ago
AI slop. Flagged.
tantalor 4mo ago
> Compared to i++ from C/C++ or forEach from JavaScript, Python's version just works. Comparing to forEach in JS is incorrect because forEach is an method of Array. You should compare it to `for...of` in JS. Both operate on iterators. Article is missing an important distinction between iterators and other "array like" types (including strings): Iterators don't have to stop, e.g., they can take from a generator that never ceases. Both Python and JS are happy to loop forever if the iterator never stops.
- quietbritishjim 4mo ago
  There's also the range based for loop in C++. An actual comparison of iterator-style loops might be interesting but this article is a pretty vacuous bit of basic Python. It's also a confusing title, given that Scratch is the name of a programming language.
klibertp 4mo ago
TL;DR: if you want a general looping construct that works no matter the container type, you need an iterator protocol that types can opt into. This is a standard technique adopted by most programming languages that are not explicitly low-level, and has existed for the past 60[1] years. The OP (re)discovered it and thought it was worth blogging about. Well, it is kind of interesting to see how the very basic programming building block (iteration) gets generalized without incurring syntactic costs. Whether it's worth a place on the HN front page is debatable, though. [1] Too lazy to track the actual first implementation, but I'd be astonished if the concept wasn't well-known by the 70's.
- klibertp 4mo ago
  I got curious and checked: CLU had iterators in 1975. > Another key feature of the CLU type system are iterators It also had exception handling and tuple unpacking! > For instance, writing x,y := y,x would exchange values of x and y (I'm joking, that's "just" parallel assignment, it just looks like what you do in modern Python with unpacking. And yes, it's all from 1975.) Details: https://en.wikipedia.org/wiki/CLU_(programming_language) https://en.wikipedia.org/wiki/CLU_(programming_language)
- rbanffy 4mo ago
  The best known programming language in the 1980s, BASIC, had a FOR that was limited to counting. A lot of people without formal CS education would think Python’s for can go over a finite set of things and live happily at that level of abstraction without ever needing to peek under the bonnet and learn how iterators work (in Python).
  - klibertp 4mo ago
    Sure. But you know, from personal experience, you can be a happy BASIC user writing simple games without knowing about the existence of arrays. It's a bit hard, but nothing copy & paste couldn't defeat. And while learning about `DIM` was quite a breakthrough for my 8 y.o. self, I'm not sure I'd welcome an article about it on HN's front page. The same is true for iterator protocols: as noted in my other comment, they have existed since 1975 in almost the same form (with a few closely related variants). It's not easy to find a mainstream language that doesn't support them "since forever" (in "tech years"). They're just not news, no matter how you slice it.
rbanffy 4mo ago
How was this flagged?!

16 points by rbanffy 4mo ago

'for' loops in Rust do the same: they create an iterator and then iterate over that. You can write the exact same loop with `let mut iter = v.iter(); while Some(x) = iter.next()`. 'for' loops in Rust are purely syntax sugar, and I somewhat wish they didn't exist. They provide you two ways of doing the same thing, but one of them hides the details from you. Having 'for' as a keyword is nice for folks coming from other languages, but then it hides the possibility of other interesting usages, like cloning an iterator inside a loop.

bigfishrunning 4mo ago
Which is funny, because whenever I encounter a language for which `for` *doesn't* work this way it feels antiquated. I do however wish another keyword was used in many cases, becuase `for` in C and Go is so much different then `for` in Rust or Python. I think the higher-level case (Rust and Python) should really use a word like `foreach` or maybe something compeletely different like `itr`, although I get that they want to "look" more like C
- rbanffy 4mo ago
  I would say Python really wants to look as much not-C as it can. In this case it's not the word "for" but the "for x in" construct.
- b40d-48b2-979e 4mo ago
  This is what PowerShell does, it has both the archaic `for` style with a newer iterator-like `foreach`.
- adrian_b 4mo ago
  Iterators and the loops of the form "forall X in A do ..." have been invented simultaneously with the appearance of the first version of the language C (i.e. around 1974), in the languages Alphard and CLU. From Alphard and CLU, loops of this form have passed into one of the versions of the language Algol 68 (1977), and from there into the UNIX Bourne shell (1979), which inherited a few Algol 68 syntax features (because its author had worked at writing Algol 68 compilers). Iterators and "forall" loops have begun to spread into mainstream programming languages, e.g. into C++, only a couple of decades after their invention.
Martinussen 4mo ago
Is "hiding" in the sense that you just need to have read the docs or know how the language works at a pretty basic level really a problem, or even a negative? I would certainly say that the readability and clarity on the form of loop being used is a bigger win, either way.
tialaramex 4mo ago
It's true that they're just sugar but Rust does explain how the for-in loop de-sugars and you've over-simplified considerably. Your syntax also doesn't quite work. The value is in idiom, turning everything into loop expressions (The "while" keyword is also just sugar, Rust's only fundamental loop is named loop) makes it harder to discern what's actually going on. If you want to clone the iterator in some cases rather than consuming it, that should look different so that reviewers will see what you're up to.
kevinmgranger 4mo ago
It should be `v.into_iter()`, and that distinction matters because of ownership / move semantics. It just so happens that for most collections, `IntoIter` is also defined for references to them, which typically gives you the same behavior that `.iter()` would give.
- tialaramex 4mo ago
  More specifically for x in y consumes y in Rust. https://doc.rust-lang.org/std/keyword.for.html https://doc.rust-lang.org/std/keyword.for.html explains how a loop is de-sugared https://rust.godbolt.org/z/5jzhxYM51 https://rust.godbolt.org/z/5jzhxYM51 ... shows that today ranges like 0..5 aren't Copy even if that would be possible, which means if they're consumed they're gone, whereas an array of integers is Copy and so consuming it doesn't mean it's gone, you can just consume it again. The desire is that Rust 2027 edition will change the nice syntax for ranges to produce new ranges like core::ranges::Range which are Copy if possible and only IntoIterator, the original ranges are never Copy but are Iterator, we now regret this choice.
saghm 4mo ago
In addition to the points that sibling comments have made about how you're not quite right with the exact semantics of for loops (which also take ownership via `into_iter` and therefore are a bit different from `while let`), it's worth pointing out that if you peek a bit further in to MIR, all loops in Rust just desugar into the `loop` keyword with manual breaking, including `while`. It's not really clear to me why for loops in particular bother you.

anthonj 4mo ago

I don't really get the point of the article. Even if I knew little about python, would be it surpsing that a language with no real basic types is probably abstracting a lot? Even a simple i=0, i=i+1 is "hiding" a lot in python then.

PaulDavisThe1st 4mo ago
But it's not a Python thing. Rust is noted below, and there's also C++. std::container<T> container; for (std::container<T>::iterator i = container.begin(); i != container.end(); ++i) { ...} auto iter = container.begin(); while (iter != container.end()) { ...; ++iter; } for (auto const & t : container) { ... }
- tialaramex 4mo ago
  Barry Revzin has a talk explaining about different kinds of iterator. The C++ iterator is a maximalist, very powerful and flexible idea which - as a result - also has lots of opportunities to set everything on fire. Their model was the C pointer, we can iterate a pointer, compare it against some sentinel "end" value, dereference it, but also we could index into it, decrement the pointer and go backwards, comparing against a "start" value, and we can write to a reference rather than just read from it. This is far more than most iterator designs allow, and as I said if you needed more this is very welcome, but if you didn't it's probably bad news. C++ can sort iterators for example, so long as they're suitable for indexing back and forth as the sort progresses. Rust's sort functions are defined on its slice types, which makes sense in practical terms but if what you've got isn't quite a slice but could meet the requirements in C++ that's a problem for you.
  - PaulDavisThe1st 4mo ago
    It's not the ability to use a maximalist iterator that is of interest here, it's the ability to use or not use any kind of iterator at all.
courtcircuits 4mo ago
To be fair, when I started learning CS, the `for x in y` syntax was cryptic to me because I was unfamiliar with concepts such as iterators & generators. `for(int i=0; i<len(y); i++)` made way more sense since there is no hidden logic (besides additions as you highlighted in your comment, but which I think is easier to have a grasp of). So I really wish I had read this article when I started my CS journey a couple of years ago.

mdemare 4mo ago

AI slop. Flagged.

tantalor 4mo ago

> Compared to i++ from C/C++ or forEach from JavaScript, Python's version just works. Comparing to forEach in JS is incorrect because forEach is an method of Array. You should compare it to `for...of` in JS. Both operate on iterators. Article is missing an important distinction between iterators and other "array like" types (including strings): Iterators don't have to stop, e.g., they can take from a generator that never ceases. Both Python and JS are happy to loop forever if the iterator never stops.

quietbritishjim 4mo ago
There's also the range based for loop in C++. An actual comparison of iterator-style loops might be interesting but this article is a pretty vacuous bit of basic Python. It's also a confusing title, given that Scratch is the name of a programming language.

klibertp 4mo ago

TL;DR: if you want a general looping construct that works no matter the container type, you need an iterator protocol that types can opt into. This is a standard technique adopted by most programming languages that are not explicitly low-level, and has existed for the past 60[1] years. The OP (re)discovered it and thought it was worth blogging about. Well, it is kind of interesting to see how the very basic programming building block (iteration) gets generalized without incurring syntactic costs. Whether it's worth a place on the HN front page is debatable, though. [1] Too lazy to track the actual first implementation, but I'd be astonished if the concept wasn't well-known by the 70's.

klibertp 4mo ago
I got curious and checked: CLU had iterators in 1975. > Another key feature of the CLU type system are iterators It also had exception handling and tuple unpacking! > For instance, writing x,y := y,x would exchange values of x and y (I'm joking, that's "just" parallel assignment, it just looks like what you do in modern Python with unpacking. And yes, it's all from 1975.) Details: https://en.wikipedia.org/wiki/CLU_(programming_language) https://en.wikipedia.org/wiki/CLU_(programming_language)
rbanffy 4mo ago
The best known programming language in the 1980s, BASIC, had a FOR that was limited to counting. A lot of people without formal CS education would think Python’s for can go over a finite set of things and live happily at that level of abstraction without ever needing to peek under the bonnet and learn how iterators work (in Python).
- klibertp 4mo ago
  Sure. But you know, from personal experience, you can be a happy BASIC user writing simple games without knowing about the existence of arrays. It's a bit hard, but nothing copy & paste couldn't defeat. And while learning about `DIM` was quite a breakthrough for my 8 y.o. self, I'm not sure I'd welcome an article about it on HN's front page. The same is true for iterator protocols: as noted in my other comment, they have existed since 1975 in almost the same form (with a few closely related variants). It's not easy to find a mainstream language that doesn't support them "since forever" (in "tech years"). They're just not news, no matter how you slice it.

rbanffy 4mo ago

How was this flagged?!