4 ms·
This seems to me like a parallel accumulation problem, why not have each thread accumulate a filter on a subset of the data (so no locking involved), and then r
by ot 3y ago
This seems to me like a parallel accumulation problem, why not have each thread accumulate a filter on a subset of the data (so no locking involved), and then reduce the results (which is just an OR of all the local accumulations)?
- sakras 3y agoParallel reductions are more heavy-weight synchronizations than locks. Say we have 64 partitions, then we need to perform 6 levels of tree reduction, or avoid parallelism completely and perform the reduction on a single thread. Either way it was slower. The locking strategy very rarely had any reduction in parallelism due to the randomized lock-taking. There were also other reasons, such as not wanting to replicate the filter per-thread.