3 ms·
The accuracy and dynamic range comparison plots seem to say there are some "NaN" conditions, with the note that it aborts the calculation. It might be worth di
by sounds 10y ago
The accuracy and dynamic range comparison plots seem to say there are some "NaN" conditions, with the note that it aborts the calculation.
It might be worth digging into the bit representation to find out exactly what happens, for instance, when you multiply 0 * infinity. That was a case that supposedly outputs NaN.