3 ms·
There a maximum error of 0.5 ulp for all of add/sub/mul/div/sqrt on all inputs (ok sin/cos & co typically have a couple ulp of error but close enough). And ind
by dzaima 2y ago
There a maximum error of 0.5 ulp for all of add/sub/mul/div/sqrt on all inputs (ok sin/cos & co typically have a couple ulp of error but close enough).
And indeed it's well-understood that it can get problematic if you compose a bunch of such, esp. add/sub with a large magnitude difference. Indeed, especially matrix multiplication.