7 ms·
The humble for loop in Rust
- mden 2y agoAnyone have examples where fold leads to easier to read code than a for loop in Rust?
- Mond_ 2y agoI find `foo.iter().map(|x| x.bar()).collect()` almost always easier to read and better at expressing intent than a for loop. The other direction is more interesting to me: Those are the awkward cases where people sometimes overdo it with the functional iterator heavy style.
- ironhaven 2y agoI have used fold for converting strings to a bitset in advent of code let string = "ewfsan"; let bitset = string.bytes().fold(0u32 |acc, ch| acc | 1 << (ch - b'a')); This is a idiom that I have used many times so this being more consice than a for loop is a plus Of course if you have never seen a syntax before it will make less sense that anything you have seen before
- bippihippi1 2y agoafair I've mostly only used fold when doing maths not covered by the standard sum or product. Fold is similar to map reduce but it's just one expression.
- kccqzy 2y agoReadability is subjective. I personally find fold almost always more readable than a for loop when the accumulator variable has a simple type. This is because merely seeing fold can already telling me several things: it will iterate over the entire collection without early exits like "break" in a loop; the data dependency between each iteration is made clear into a single variable. I find it slightly difficult to read when the accumulator variable actually has multiple parts, like a complicated tuple. It's worse when part of the accumulator is a bool indicating whether it's finished; that's just a poor emulation of "break" in a for loop.
- underdeserver 2y agoWithout looking too much at the generated assembly in https://godbolt.org/z/fKEPaTdTv https://godbolt.org/z/fKEPaTdTv, it seems that using_map uses SIMD instructions (movdqu and psubd) in the main loop, while using_loop doesn't, so that could indeed explain the performance difference.
- nemo1618 2y ago> Why is map so much faster? I am not sure. I suspect with the map() option the Rust compiler figures out it can avoid allocations altogether by simply writing over the original vector, while with the loop it can't. Or maybe it's using SIMD? I tried to look in the compiler explorer but I'm not competent enough yet to figure it out. Maybe someone else can explain! Yep, it's due to SIMD -- in the assembly for `using_map`, you can spot pcmpeqd, movdqu, and psubd, while `using_loop` doesn't have any of these.
- aidos 2y agoAh thanks! I was going to ask in here to see if anyone knew because the explanation about overwriting the same vector seemed pretty off base. Never worked in Rust though so wondered if the iterator api had some weird optional notion of size that could be utilised throughout the chain.
- c0balt 2y ago> Never worked in Rust though so wondered if the iterator api had some weird optional notion of size that could be utilised throughout the chain. Fwiw, this does exist: [Iterator::size_hint] (https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.size_hint https://doc.rust-lang.org/std/iter/trait.Iterator.html#metho...)
- hedgehog 2y agoIt seems `map` should have less restrictive semantics (specifically ordering) than `for`, does that allow more optimization? I don't know much about Rust internals.
- ironhaven 2y agoReading the godbolt it looks like for the push loop llvm is unable to remove the `grow_one` capacity check after every push. Becaue of this the Vec could possibly reallocate after every push meaning it can't auto vectorize.
- dmart 2y agoI would have liked to see a comparison in the fold() section that functions the same way as the original for loop: list_of_lists.into_iter().fold(Vec::new(), |mut accumulator, list| { accumulator.extend(list); accumulator } )
- kccqzy 2y agoThe author's fold example is unfair. They could've just called accumulator.extend() and then return the accumulator inside the fold example for a fair apple to apple comparison. Just mark the accumulator as mut. Furthermore, I'd use with_capacity in both cases: Vec::with_capacity(list_of_lists.iter().map(|l| l.len()).sum())
- IshKebab 2y agoYeah I kind of wish there was a way to still use `?` inside map/filter/etc. lambdas. Error handling with that functional style is generally way more awkward than for loops, but also it's often more elegant in other ways (e.g. Rayon). I think Ruby has some kind of feature that works like that but IIRC it looked less foot-gun more foot-bazooka. Does anyone know of any languages that solve that problem elegantly?
- zozbot234 2y ago"Ad-Hoc Effects in Rust" https://capi.hannobraun.com/daily/2024-12-12 https://capi.hannobraun.com/daily/2024-12-12 discusses the issue in detail. An early attempt at a solution (2022) was provided by https://blog.rust-lang.org/inside-rust/2022/07/27/keyword-generics.html https://blog.rust-lang.org/inside-rust/2022/07/27/keyword-ge...
- mijoharas 2y agoWhat's the ruby feature you're referring to?
- IshKebab 2y agohttps://medium.com/swlh/returning-from-a-ruby-proc-beware-of-where-you-land-a29add7ddd9d https://medium.com/swlh/returning-from-a-ruby-proc-beware-of...
- anon-3988 2y agoI think what you want is a short-circuiting behavior. I don't see a problem with the status quo, if map is able to short-circuit, that means that it runs in order and have side effects. If your "map" can cause the whole function to return, then just write a for-loop. I don't even know if it should return from the whole function or just stop the mapping?
- lilyball 2y agoWhy does fallible_flatten_fold have that accumulator.clone() in it? You're cloning the in-progress vector only to throw away the original, it's extremely wasteful and completely unnecessary. Just declare the accumulator as `mut`.
- dhosek 2y agoIt’s interesting to note that performance of a for loop versus the functional-style mechanism varies by language. On Java, there is a performance penalty (possibly shrunken since Java 8) for using the FP idioms while in Rust, they end up much faster.
- kstrauser 2y agoThat surprises me, and I'd expect the FP version to be at least as fast, with the option to be much faster. With the for loop, you're saying "run against this value, then run against the next one, then run against the next one, then...". If the compiler isn't certain that the iterations aren't free of side effects, then it would have to run each one in order before moving on to the next. `.map(...)` implies "I don't care about ordering, and therefore you don't need to, either", freeing the compiler to schedule the loops in a more optimal order, or in parallel or with SIMD, or any other optimization that lets it get the job done as fast as possible. I'm sure someone will come up with an example, but I can't personally think of any way where a for-loop's semantics would let a clever compiler write faster code than the equivalent map.
- varikin 2y agoStreams (FP in java) is slower than for loops for a couple reasons. A big one is java doesn't natively support real closures or lambdas. It does have syntax for them, but that is just syntactic sugar for an class with a single method under the hood. So streams end up doing lot of object allocation and garbage for the fake closures. Also, streams operate on objects, so they have to be on the heap. You can't use them with primitives on the stack. Though with autoboxing, the JVM may play some tricks with a list of Integer objects really being primitives on the stack, but I would never count on it. As for SIMD, Java isn't going to parallelize anything automatically. You need to tell it you run the steam in parallel which will split it into threads. Java doesn't have lightweight threads like coroutines. I know lightweight threads are on the roadmap and maybe available in Java 21 or newer. I know real closures have been considered, but I don't if it's gone anywhere. It's hard to do a quick search because we got "closures" in Java 8 so theres a lot of noise. And as a caveat, I am most familiar with Java 17 (and older). I expect we'll look at moving to Java 21 (current LTS) next year.
- deleted 2y ago[deleted]
- worik 2y agoIt is not that hard: If there are side effects, use a for loop
- scotty79 2y agoI'm currently doing advent of code in Scala 3 to learn a bit of this language. For loop does weird things there. It can be used as a flat map as it can iterate over multiple iterators at once and yield a value for each combination. What's bitten me so far multiple times is that when you iterate over a Set or a Map the result of for expression is also a Set or a Map. But since you have access to keys during iteration then some iterations, if they return same map key or same set value, might get silently overwritten by others. I don't remember having this problem in Rust because there I had to be very intentional about iterators. The other thing that bit me was that arithmetic on Int overflows silently, but that's apparently a Java thing, which made me wonder how is Java an enterprise language. Otherwise Scala 3 is superb expeirience. Syntax is ultra-flexible and local extensibility of everything and access to things from the context of where your code is defined and even from the context where it's running is magical.
- winwang 2y agoglad to see another Scala fan :) keep in mind that "for loops" are really "for comprehensions" and desugae into flatMap/map
- CrendKing 2y agoMaybe I'm dumb, but I can't see how the code in the "Errors and map" section can compile. "transform_list" returns a Result<>, yet "result" is just a Vec. I thought you always need to wrap it with Ok()? Is that a new nightly feature?
- deleted 2y ago[deleted]
- orf 2y agoNo, it seems like an oversight in the code sample. It should be wrapped in Ok()
- xpe 2y agoDear author (and others who writing things like this): it looks you put in considerable time on code comparisons and benchmarks. Why not publish the source code for them? Benefits include: 1. The comparisons, as you've written them up, probably will get stale fast. It would be nice to be able to re-run them. 2. Some of the examples are surprising. Why? Any bugs? Some kind of weirdness? Readers would want to explore. 3. Readers can see how you did the benchmarks. We can put more/less stock in them. Hopefully you used `criterion` or similar, with warm ups, etc.