3 ms·
> With floating-point numbers, I get this effect with the string “0.004221370676532388” (for example). When I round it directly to a float, I get 0.004221370909
by im3w1l 5y ago
> With floating-point numbers, I get this effect with the string “0.004221370676532388” (for example). When I round it directly to a float, I get 0.0042213709093630313873291015625. If I first round it to a double and then to a float, I get 0.0042213709093630313873291015625. You probably cannot tell unless you are a machine, but parsing directly to a float is 2e-7 % more accurate.
I scratched my head at this as those are literally the same number.
Looking into it a bit, the correctly rounded float should be 0.0042213709 and the incorrect, twice-rounded is 0.0042213704.
To see what is going on the number in question is in binary base
0.00000001000101001010011011011001000000000000000000000000000001...
Rounded to double is
0.000000010001010010100110110110010000000000000000000000000000
And when that is rounded to float (round to even mode) you get
0.0000000100010100101001101101100
While the correct number is
0.0000000100010100101001101101101