3 ms·
Isn’t it more often associativity that is required for parallel programming? Being able to parallelize “a+b+c+d” only requires associativity to order the opera
by MereInterest 3y ago
Isn’t it more often associativity that is required for parallel programming? Being able to parallelize “a+b+c+d” only requires associativity to order the operations as “(a+b) + (c+d)”. Sure, there’s additional benefits for memory locality if you can rearrange the term to “(a+c) + (b+d)”, assuming that the four terms are stored in contiguous memory and each sum is computed with vector operations, but that’s not strictly required for parallelization.
- phlakaton 3y agoIn parallel programming, subcomputations may complete out of order. The ability to combine those subcomputations in any order can be pretty useful!
- BeetleB 3y agoThat, though, is associativity, not commutativity. In FP arithmetic, operations are commutative, not associative.
- phlakaton 3y agoIf you receive b before a and combine them as b*a, that's commutative. Both properties are useful in parallel programming, but you may only need one in a specific case, depending on your application.
- GrumpySloth 3y agoBoth are important. I replied to a comment about commutativity, so I focused on that.