3 ms·
I think the shift at the end (>> qlp_shift) takes care of this. The multiplication and sum are done in 64-bit, and then the sum is presumably shifted enough so
by nkurz 10y ago
I think the shift at the end (>> qlp_shift) takes care of this. The multiplication and sum are done in 64-bit, and then the sum is presumably shifted enough so as to fit in 32-bits.
- junke 10y agoThanks. Isn't it possible to give a "coefficients" vector with large enough values so that computations with 64 bits would overflow too? I don't doubt the code works in practice, because the range of values are reasonable. However this is not explicit, just looking at the code. I'd prefer if the actual domains were being made obvious. Right now, 12, 32 and 64 feel like magic numbers in the code.