4 ms·

Sure, but when you use a parallel iterator like Rayon in Rust, memory and data-race safety is guaranteed at compile time. And using Rayon is as simple as changi

Sure, but when you use a parallel iterator like Rayon in Rust, memory and data-race safety is guaranteed at compile time. And using Rayon is as simple as changing “.iter()” to “.par_iter()” in your code thanks to the trait system. Not that you lose that safety doing the thread pools and such manually, it’s just unnecessary.

jandrese 8y ago
Is there any reason to use .iter() over .par_iter()? Does the latter have higher constant costs that only amortize on lists larger than some value? Or could you just replace .iter() with .par_iter() under the hood and reap the benefits for free? I guess if you are iterating over something declared unsafe? But the compiler should be able to recognize that and fall back to the linear version automatically.
- steveklabnik 8y ago
  iter is in the standard library, and par_iter isn't, for one. I'm not quite sure about the constant cost thing, but I would imagine that it's true, as it's inherently doing more work.
- pornel 8y ago
  Rayon has relatively small overhead, but that doesn't mean you can find'n'replace iter() with par_iter(): - not every collection/combination of iterators supports parallel work. For some it doesn't make sense, for some it's just not implemented. - while Rust prevents data races, it can't prevent deadlocks, so you need to be mindful of locks. - .iter() is allowed to mutate state outside of the iterator and use non-thread-safe objects, but .par_iter() isn't (without explicit synchronization primitives). So par_iter() will force you to use more expensive atomic and mutex-locked objects. For data-bound problems most of the time it just works great. I've ran into problems with some networking libraries having global state, which prevented use of par_iter() (but no crashes - caught at compile time!)
- TheCoelacanth 8y ago
  There are a lot of cases where using par_iter wouldn't be correct even without unsafe. If you are doing anything with side-effects par_iter would execute the side effects out of order.
anderspitman 8y ago
The first time I added rayon to a project and switched my main `iter` to `par_iter` and it compiled with no other changes was decidedly creepy. That particular code didn't run any faster, but the fact that it was still guaranteed to be safe just felt so counter to everything the last 10 years of programming had taught me. Rust is not a perfect language (whatever that means), but more so than any language I've ever used it gives me these strange sort of mind-blowing moments.

jandrese 8y ago

Is there any reason to use .iter() over .par_iter()? Does the latter have higher constant costs that only amortize on lists larger than some value? Or could you just replace .iter() with .par_iter() under the hood and reap the benefits for free? I guess if you are iterating over something declared unsafe? But the compiler should be able to recognize that and fall back to the linear version automatically.

steveklabnik 8y ago
iter is in the standard library, and par_iter isn't, for one. I'm not quite sure about the constant cost thing, but I would imagine that it's true, as it's inherently doing more work.
pornel 8y ago
Rayon has relatively small overhead, but that doesn't mean you can find'n'replace iter() with par_iter(): - not every collection/combination of iterators supports parallel work. For some it doesn't make sense, for some it's just not implemented. - while Rust prevents data races, it can't prevent deadlocks, so you need to be mindful of locks. - .iter() is allowed to mutate state outside of the iterator and use non-thread-safe objects, but .par_iter() isn't (without explicit synchronization primitives). So par_iter() will force you to use more expensive atomic and mutex-locked objects. For data-bound problems most of the time it just works great. I've ran into problems with some networking libraries having global state, which prevented use of par_iter() (but no crashes - caught at compile time!)
TheCoelacanth 8y ago
There are a lot of cases where using par_iter wouldn't be correct even without unsafe. If you are doing anything with side-effects par_iter would execute the side effects out of order.

anderspitman 8y ago

The first time I added rayon to a project and switched my main `iter` to `par_iter` and it compiled with no other changes was decidedly creepy. That particular code didn't run any faster, but the fact that it was still guaranteed to be safe just felt so counter to everything the last 10 years of programming had taught me. Rust is not a perfect language (whatever that means), but more so than any language I've ever used it gives me these strange sort of mind-blowing moments.