4 ms·
I would think overflow is a very rare problem for floats; I don't think any code I have ever written would need to handle the overflow case. The more significan
by thethirdone 6y ago
I would think overflow is a very rare problem for floats; I don't think any code I have ever written would need to handle the overflow case. The more significant problem should be precision loss.
Is the algorithm in [1] significantly more precise for inputs that don't result in overflow?
Edit:
I made a small program to check a few values. It is seems like the algorithm in [1] is better in terms on precision, but only by a small margin (on the small set of values I checked).
- deleted 6y ago[deleted]
- sjburt 6y agosqrt(x*x + y*y) loses a lot of precision if x and y differ greatly in magnitude.
- thethirdone 6y agoDo you have an example where this is true? I have not seen an example with > 2 ulp difference from the true value or > 1 ulp difference between the two algorithms.