4 ms·
The reason you need multiple sums is because there is a loop dependency which causes every iteration to stall and prevents pipelining and saturating the multipl
by mgaunard 3y ago
The reason you need multiple sums is because there is a loop dependency which causes every iteration to stall and prevents pipelining and saturating the multiple compute units of your superscalar processor.
Yes, floating-point isn't associative, so the results aren't numerically equivalent, and that is why you need to do that optimization yourself since the compiler isn't allowed to do it for you.
As per the exercise, there is no information about the distribution of the data, so there is no indication that any order would be any better than another for numerical stability.
If you care about stability you'd implement a kahan sum, which isn't really affected by changing the order of the sum so that you can parallelize it.
- dragontamer 3y agoKahan summing doesn't fix cancellation error. It just extends the precision by another 53-bits or so. The only way you fix cancellation error is summing the positives in one variable, and the negatives in another, and do a big cancellation at the end. Anyway, my overall point is not to criticize Kahan summing or other techniques. My point is that these techniques are incompatible with optimizations. The faster way is often times a less stable operation.