6 ms·

This kind of syntactic sugar used to appeal to me, but now I think it's a pretty weird feature to add to a language. Using zip / enumerate primitives feels a lo

by rcme 4y ago

This kind of syntactic sugar used to appeal to me, but now I think it's a pretty weird feature to add to a language. Using zip / enumerate primitives feels a lot more flexible.

laserbeam 4y ago
Depends on what you mean by flexible. If you want to use them outside of loops then they could cause magic data copies behind the hood. Zig really hates hidden control flow/allocations/copies. Within the for syntax it's pretty straightforward what gets assigned to what variables and how copies can be avoided. Doing things like `a = @zip(some_list, some_other_list)` can be reasoned about in multiple ways, some of which involve silently calling malloc. It's particularly unclear what could be done with `a` afterwards. Zig hates that kind of ambiguity and is happy to err away from flexibility at times.
- brundolf 4y ago
  Rust also hates hidden allocations, and its iterator system can do all of this without them Although- thinking about it, that may rely on the borrow checker (move semantics specifically)
  - masklinn 4y ago
    That's nothing special though, `zip` just takes an item from each iterator, packs them into a tuple, and yields that. It has no weird bounds or requirements or anything: https://doc.rust-lang.org/std/iter/fn.zip.html https://doc.rust-lang.org/std/iter/fn.zip.html The impl of the default `next` is: fn next(&mut self) -> Option<(A::Item, B::Item)> { let x = self.a.next()?; let y = self.b.next()?; Some((x, y)) } So completely straightforward.
    - brundolf 4y ago
      I was thinking about the fact that whatever you're iterating over has to be copied around throughout the process. Rust can guarantee that eg. deep-copies (clones) of allocated structs will never happen implicitly, if your iterator owns the values being iterated. But in languages where copying can trigger allocations, this could be a problem I don't actually know whether that applies to Zig though
    - defen 4y ago
      Can that zip more than two iterators? And does it perform a bounds check on each call to `a.next()` and `b.next()`?
      - brundolf 4y ago
        It stops when one of the two iterators ends It can't zip more than two per se, but you could zip the result of the first zip into a third and get ((item1, item2), item3). You could then map these if you wanted, to flatten them into a single tuple .map(|((item1, item2), item3)| (item1, item2, item3)) Of course there's a trade-off here between ergonomics and generality
        masklinn 4y ago
        > It can't zip more than two per se, but you could zip the result of the first zip into a third and get ((item1, item2), item3). You could then map these if you wanted, to flatten them into a single tuple .map(|((item1, item2), item3)| (item1, item2, item3)) FWIW that's more or less what `itertools::izip!` does for you, it just chains `zip`s then "splats" them using a `map`.
        defen 4y ago
        > It stops when one of the two iterators ends Right; my question is, suppose you're iterating over two slice iterators - won't each call to `a.next()` and `b.next()` have to check whether that sub-iterator is done? One of the benefits of the Zig approach is that you can iterate over an arbitrary number of slices and do one check before entering the loop, followed by the compiler emitting unchecked index access in the loop. So it basically compiles down to the equivalent of a C `for` loop.
        masklinn 4y ago
        Rust's zip has a specialisation for iterators with a trusted length. Such as slice iterators. `zip` yields exactly the same assembly as a loop over the index range with an unsafe item access: https://godbolt.org/z/7ebfxbhxc https://godbolt.org/z/7ebfxbhxc
    - laserbeam 4y ago
      You're assuming that an iterator gets created and returned by zip. In zig the for loop syntax doesn't need the concept of an iterator. All you need is variables in a capture. That's what I mean by straightforward. You'd have to argue about the memory layout for an iterator, that you call functions on it etc. Lots of small decisions. In zig those things would usually be part of the standard library, not language features. If you want to call a next() function on a thing in zig, you want to be explicit. That's what the language means by "no hidden control flow". You'd use a while loop if you wanted to iterate on a hashmap for example. In some sense, this is less elegant, but it's usually more obvious what the compiler would end up doing in any bit of code.
  - gpanders 4y ago
    >Rust also hates hidden allocations Does it? Rust seems happy to allocate silently all the time. let x = String::new(“hi”); let y = vec![]; Do either of these allocate? As the writer or reader of this code, how do I know if either of these statements result in a heap allocation, or if the data is strictly on the stack? Zig’s requirement of explicitly passing around an Allocator type removes any ambiguity completely.
    - steveklabnik 4y ago
      (You’re forgetting a “new” in the string example)
      - gpanders 4y ago
        Thanks, I fixed it :)
    - deleted 4y ago
      [deleted]
    - brundolf 4y ago
      Sure, any arbitrary function (or macro) logic can allocate. It's more a philosophy, not something that's language-enforced[0] in Rust- if you're creating a mutable, variable-size data structure like a String or a Vec or a HashMap you're not going to be very surprised that it allocates at some point (though technically zero-length Vecs don't allocate on construction, they wait until an item is added) But closures don't require allocation, iterators don't require allocation, async doesn't require allocation. Copy semantics also don't allow allocation- implicit copies can only happen for data structures that are bitwise-copyable, which is enforced by the compiler. For copy-with-allocation you have to implement the Clone trait, and then invoke it explicitly with the .clone() method But the original context was a question of philosophy, so I was only speaking to Rust's overall philosophy [0] Technically I think if you're using no_std you won't have access to any standard constructs that allocate (which obviously will prevent their use at compile-time), though I believe you're still allowed to eg. call out to foreign functions manually that would allocate. And of course, this still isn't as granular as Zig's allocation-control.
pmontra 4y ago
I don't know how common working with ranges is in Zig. Ruby would iterate on multiple ranges by converting them to arrays one = (1..3) seven = (7..10) (one.to_a + seven.to_a).each {|n| puts n} I suppose that if it was common they would have added a + method to Range. Actually I think that's possible to implement it with a refinement on the Range class. Yup, it works. First time I ever used refinements. module JoinRanges refine Range do def +(other) self.to_a + other.to_a end end end using JoinRanges one = (1..3) seven = (7..10) (one + seven).each {|n| puts n}
- kdmccormick 4y ago
  This is different. You are concatenating the arrays, whereas the article & discussion are about zipping arrays.
travisgriggs 4y ago
This is my take as well. The older and more travelled I get the more I disdain these kinds of things. Your language syntax should do whatever the "thing" is that your language model is all about. Syntactic sugar should be for the things you do LOTS. I watch language after language add sugar to maintain the appeal of their product, one niche group or application at a time. It turns into a death by a thousand cuts, or by a thousand sugar cubes. Most languages start out simple and appealing and understandable, an increasingly short amount of time later, they've layered on "helper" after "helper" to the point it takes a bit of expertese to consume the language effectively. I dream of a world where we'd measure languages by the complexity of their ASTs rather than their popularity on a TIOBE or StackOverflow index.
- AndyKelley 4y ago
  Arguably this change to the zig language is overall a simplification because the loop index capture is no longer a special case.
  - travisgriggs 4y ago
    Could be. I think you're more the expert here than me? :D To me, the followin is a bit of syntactic sugar that I think is the kind of transcendental "go big/basic with it" that I hint at. Some time ago, I worked in a language that had this idea that any composable block of code could be captured as 0-N statements between the characters [ and ]. They thought they were being clever and called it a "block of code". Which I thought was cool, because it looked like a block. Pedants called it a BlockClosure. If you wanted to pass parameters to one of these, they used a colon denoted list. So a two arg block might look like [:a :b | <code goes here> ] So yay, pass a closure to a service, and it "captures" the values be invoking said closure with arguments. And then the authors thought, okay, enough sugar for a few days, let's just use this. I mean really really really use this. You can use a two arg block like that for a zip function of course, but why limit it to iteration? Use it in the standard library to implement the "for each" function. Which when you looked at was just that "how dare they not have a for syntax" while implementation. But because it wasn't embedded in the syntax, you could copy/paste/modify to come up with a filter iteration. Or a reduce. Or a map. Or all kinds of interesting compositions "selectAndCollectAndReject" with 3 closures. And why stop there? They decided, "let's just do boolean logic with these block things too". So where as most languages has special syntax for conditionals (and once they start, they're in competition with their peers to keep adding more and more of them (do while, case, if, if with N elses, on and on). But they just wrote it like <condition> ifTrue: [trueBlock] ifFalse: [falseBlock] Sure they optimized it, but from a linguistic point of view, it was the same thing as above. No new sugar was needed. Whereas many languages have added sugar for optionals (usually involving ?s), this language, 20 years ago, was doing it with closures already. Someone noticed they could implement the following family of "functions" ifNil: [nilBlock] ifNil: [nilBlock] notNil: [:notNilValue | notNilBlock] ifNotNil: [:notNilValue | notNilBlock] Sure, not as terse as ? (which some endeavoured to deal with), but the language semantics didn't have to change each time there was a new thing to do. I'm sure there's a Lisper out there that can write their analog to the above. Because it too, is was one of these "do much with little" langauges.
hryx 4y ago
In general Zig foregoes syntactic sugar and requires implementing higher-level APIs by composing primitives. But a new language feature is a candidate when it solves a use case that can't otherwise be solved, or opens up a path to more efficient code. Loris' blog post points out that the new for loops address the latter: > In the multi-sequence for loop version it’s only necessary to test once at the beginning of the loop that the two arrays have equal size, instead of having 2 assertions run every loop iteration. The multi-sequence for loop syntax helps convey intention more clearly to the compiler, which in turn lets it generate more efficient code. It also builds on existing properties of slices/arrays, rather than adding a new "enumerate primitive".
cryptonector 4y ago
To me this looks a lot like closure syntax w/ non-local exits. Seems quite reasonable for a functional programming language.
moomin 4y ago
I think it matters what your target use cases are. This makes me think quite a few people are running ECS systems.
deleted 4y ago
[deleted]
cgh 4y ago
How cache-friendly are zip/enumerate implementations? Zig is influenced by the ideas behind Data Oriented Design, mentioned in the article (and a buried lede, if you ask me). Explicit for loops like this are generally cache-friendly and ideal for eg game programming, as shown in the structs of arrays example.
- vore 4y ago
  As cache-friendly as advancing two pointers and a bounds check.
- deleted 4y ago
  [deleted]
- steveklabnik 4y ago
  I tossed together a simple function using enumerate https://godbolt.org/z/PKsEdKvKK https://godbolt.org/z/PKsEdKvKK You get the same exact asm as the manual loop. Of course, the idiom recognition seems to kick in, in both cases, as there's no actual loop here. I tossed in a +sum, which makes that fail, so you get some loops, check it out: https://godbolt.org/z/1ddf5ded7 https://godbolt.org/z/1ddf5ded7 They are one instruction different in length, which is kind of amusing to me. Some small differences.
  - cgh 4y ago
    Thanks, that’s exactly what I was asking. I don’t write Rust so it’s informative to see this.
    - steveklabnik 4y ago
      Any time.
bjoli 4y ago
It isn't more flexible. Maybe more expressive or convenient, since you are only describing the flow in a cursory manner. Sometimes you will need to be explicit about what you want to happen, and then a proper loop facility will let you do that.

by rcme 4y ago

This kind of syntactic sugar used to appeal to me, but now I think it's a pretty weird feature to add to a language. Using zip / enumerate primitives feels a lot more flexible.

laserbeam 4y ago

Depends on what you mean by flexible. If you want to use them outside of loops then they could cause magic data copies behind the hood. Zig really hates hidden control flow/allocations/copies. Within the for syntax it's pretty straightforward what gets assigned to what variables and how copies can be avoided. Doing things like `a = @zip(some_list, some_other_list)` can be reasoned about in multiple ways, some of which involve silently calling malloc. It's particularly unclear what could be done with `a` afterwards. Zig hates that kind of ambiguity and is happy to err away from flexibility at times.

brundolf 4y ago
Rust also hates hidden allocations, and its iterator system can do all of this without them Although- thinking about it, that may rely on the borrow checker (move semantics specifically)
- masklinn 4y ago
  That's nothing special though, `zip` just takes an item from each iterator, packs them into a tuple, and yields that. It has no weird bounds or requirements or anything: https://doc.rust-lang.org/std/iter/fn.zip.html https://doc.rust-lang.org/std/iter/fn.zip.html The impl of the default `next` is: fn next(&mut self) -> Option<(A::Item, B::Item)> { let x = self.a.next()?; let y = self.b.next()?; Some((x, y)) } So completely straightforward.
  - brundolf 4y ago
    I was thinking about the fact that whatever you're iterating over has to be copied around throughout the process. Rust can guarantee that eg. deep-copies (clones) of allocated structs will never happen implicitly, if your iterator owns the values being iterated. But in languages where copying can trigger allocations, this could be a problem I don't actually know whether that applies to Zig though
  - defen 4y ago
    Can that zip more than two iterators? And does it perform a bounds check on each call to `a.next()` and `b.next()`?
    - brundolf 4y ago
      It stops when one of the two iterators ends It can't zip more than two per se, but you could zip the result of the first zip into a third and get ((item1, item2), item3). You could then map these if you wanted, to flatten them into a single tuple .map(|((item1, item2), item3)| (item1, item2, item3)) Of course there's a trade-off here between ergonomics and generality
      - masklinn 4y ago
        > It can't zip more than two per se, but you could zip the result of the first zip into a third and get ((item1, item2), item3). You could then map these if you wanted, to flatten them into a single tuple .map(|((item1, item2), item3)| (item1, item2, item3)) FWIW that's more or less what `itertools::izip!` does for you, it just chains `zip`s then "splats" them using a `map`.
      - defen 4y ago
        > It stops when one of the two iterators ends Right; my question is, suppose you're iterating over two slice iterators - won't each call to `a.next()` and `b.next()` have to check whether that sub-iterator is done? One of the benefits of the Zig approach is that you can iterate over an arbitrary number of slices and do one check before entering the loop, followed by the compiler emitting unchecked index access in the loop. So it basically compiles down to the equivalent of a C `for` loop.
        masklinn 4y ago
        Rust's zip has a specialisation for iterators with a trusted length. Such as slice iterators. `zip` yields exactly the same assembly as a loop over the index range with an unsafe item access: https://godbolt.org/z/7ebfxbhxc https://godbolt.org/z/7ebfxbhxc
  - laserbeam 4y ago
    You're assuming that an iterator gets created and returned by zip. In zig the for loop syntax doesn't need the concept of an iterator. All you need is variables in a capture. That's what I mean by straightforward. You'd have to argue about the memory layout for an iterator, that you call functions on it etc. Lots of small decisions. In zig those things would usually be part of the standard library, not language features. If you want to call a next() function on a thing in zig, you want to be explicit. That's what the language means by "no hidden control flow". You'd use a while loop if you wanted to iterate on a hashmap for example. In some sense, this is less elegant, but it's usually more obvious what the compiler would end up doing in any bit of code.
- gpanders 4y ago
  >Rust also hates hidden allocations Does it? Rust seems happy to allocate silently all the time. let x = String::new(“hi”); let y = vec![]; Do either of these allocate? As the writer or reader of this code, how do I know if either of these statements result in a heap allocation, or if the data is strictly on the stack? Zig’s requirement of explicitly passing around an Allocator type removes any ambiguity completely.
  - steveklabnik 4y ago
    (You’re forgetting a “new” in the string example)
    - gpanders 4y ago
      Thanks, I fixed it :)
  - deleted 4y ago
    [deleted]
  - brundolf 4y ago
    Sure, any arbitrary function (or macro) logic can allocate. It's more a philosophy, not something that's language-enforced[0] in Rust- if you're creating a mutable, variable-size data structure like a String or a Vec or a HashMap you're not going to be very surprised that it allocates at some point (though technically zero-length Vecs don't allocate on construction, they wait until an item is added) But closures don't require allocation, iterators don't require allocation, async doesn't require allocation. Copy semantics also don't allow allocation- implicit copies can only happen for data structures that are bitwise-copyable, which is enforced by the compiler. For copy-with-allocation you have to implement the Clone trait, and then invoke it explicitly with the .clone() method But the original context was a question of philosophy, so I was only speaking to Rust's overall philosophy [0] Technically I think if you're using no_std you won't have access to any standard constructs that allocate (which obviously will prevent their use at compile-time), though I believe you're still allowed to eg. call out to foreign functions manually that would allocate. And of course, this still isn't as granular as Zig's allocation-control.

pmontra 4y ago

I don't know how common working with ranges is in Zig. Ruby would iterate on multiple ranges by converting them to arrays one = (1..3) seven = (7..10) (one.to_a + seven.to_a).each {|n| puts n} I suppose that if it was common they would have added a + method to Range. Actually I think that's possible to implement it with a refinement on the Range class. Yup, it works. First time I ever used refinements. module JoinRanges refine Range do def +(other) self.to_a + other.to_a end end end using JoinRanges one = (1..3) seven = (7..10) (one + seven).each {|n| puts n}

kdmccormick 4y ago
This is different. You are concatenating the arrays, whereas the article & discussion are about zipping arrays.

travisgriggs 4y ago

This is my take as well. The older and more travelled I get the more I disdain these kinds of things. Your language syntax should do whatever the "thing" is that your language model is all about. Syntactic sugar should be for the things you do LOTS. I watch language after language add sugar to maintain the appeal of their product, one niche group or application at a time. It turns into a death by a thousand cuts, or by a thousand sugar cubes. Most languages start out simple and appealing and understandable, an increasingly short amount of time later, they've layered on "helper" after "helper" to the point it takes a bit of expertese to consume the language effectively. I dream of a world where we'd measure languages by the complexity of their ASTs rather than their popularity on a TIOBE or StackOverflow index.

AndyKelley 4y ago
Arguably this change to the zig language is overall a simplification because the loop index capture is no longer a special case.
- travisgriggs 4y ago
  Could be. I think you're more the expert here than me? :D To me, the followin is a bit of syntactic sugar that I think is the kind of transcendental "go big/basic with it" that I hint at. Some time ago, I worked in a language that had this idea that any composable block of code could be captured as 0-N statements between the characters [ and ]. They thought they were being clever and called it a "block of code". Which I thought was cool, because it looked like a block. Pedants called it a BlockClosure. If you wanted to pass parameters to one of these, they used a colon denoted list. So a two arg block might look like [:a :b | <code goes here> ] So yay, pass a closure to a service, and it "captures" the values be invoking said closure with arguments. And then the authors thought, okay, enough sugar for a few days, let's just use this. I mean really really really use this. You can use a two arg block like that for a zip function of course, but why limit it to iteration? Use it in the standard library to implement the "for each" function. Which when you looked at was just that "how dare they not have a for syntax" while implementation. But because it wasn't embedded in the syntax, you could copy/paste/modify to come up with a filter iteration. Or a reduce. Or a map. Or all kinds of interesting compositions "selectAndCollectAndReject" with 3 closures. And why stop there? They decided, "let's just do boolean logic with these block things too". So where as most languages has special syntax for conditionals (and once they start, they're in competition with their peers to keep adding more and more of them (do while, case, if, if with N elses, on and on). But they just wrote it like <condition> ifTrue: [trueBlock] ifFalse: [falseBlock] Sure they optimized it, but from a linguistic point of view, it was the same thing as above. No new sugar was needed. Whereas many languages have added sugar for optionals (usually involving ?s), this language, 20 years ago, was doing it with closures already. Someone noticed they could implement the following family of "functions" ifNil: [nilBlock] ifNil: [nilBlock] notNil: [:notNilValue | notNilBlock] ifNotNil: [:notNilValue | notNilBlock] Sure, not as terse as ? (which some endeavoured to deal with), but the language semantics didn't have to change each time there was a new thing to do. I'm sure there's a Lisper out there that can write their analog to the above. Because it too, is was one of these "do much with little" langauges.

hryx 4y ago

In general Zig foregoes syntactic sugar and requires implementing higher-level APIs by composing primitives. But a new language feature is a candidate when it solves a use case that can't otherwise be solved, or opens up a path to more efficient code. Loris' blog post points out that the new for loops address the latter: > In the multi-sequence for loop version it’s only necessary to test once at the beginning of the loop that the two arrays have equal size, instead of having 2 assertions run every loop iteration. The multi-sequence for loop syntax helps convey intention more clearly to the compiler, which in turn lets it generate more efficient code. It also builds on existing properties of slices/arrays, rather than adding a new "enumerate primitive".

cryptonector 4y ago

To me this looks a lot like closure syntax w/ non-local exits. Seems quite reasonable for a functional programming language.

moomin 4y ago

I think it matters what your target use cases are. This makes me think quite a few people are running ECS systems.

deleted 4y ago

[deleted]

cgh 4y ago

How cache-friendly are zip/enumerate implementations? Zig is influenced by the ideas behind Data Oriented Design, mentioned in the article (and a buried lede, if you ask me). Explicit for loops like this are generally cache-friendly and ideal for eg game programming, as shown in the structs of arrays example.

vore 4y ago
As cache-friendly as advancing two pointers and a bounds check.
deleted 4y ago
[deleted]
steveklabnik 4y ago
I tossed together a simple function using enumerate https://godbolt.org/z/PKsEdKvKK https://godbolt.org/z/PKsEdKvKK You get the same exact asm as the manual loop. Of course, the idiom recognition seems to kick in, in both cases, as there's no actual loop here. I tossed in a +sum, which makes that fail, so you get some loops, check it out: https://godbolt.org/z/1ddf5ded7 https://godbolt.org/z/1ddf5ded7 They are one instruction different in length, which is kind of amusing to me. Some small differences.
- cgh 4y ago
  Thanks, that’s exactly what I was asking. I don’t write Rust so it’s informative to see this.
  - steveklabnik 4y ago
    Any time.

bjoli 4y ago

It isn't more flexible. Maybe more expressive or convenient, since you are only describing the flow in a cursory manner. Sometimes you will need to be explicit about what you want to happen, and then a proper loop facility will let you do that.