3 ms·

Thanks. What I didn’t realize is that although the sum is done precisely, the resulting 0.3 will be represented approximately once converted to float64. In the

by povik 7y ago

Thanks. What I didn’t realize is that although the sum is done precisely, the resulting 0.3 will be represented approximately once converted to float64. In the first case formatting hides that, in the last it doesn’t.

ehsankia 7y ago
I think in the last example, it's going straight from arbitrary precision to 54 significant digit, bypassing float64 entirely, hence why it looks different from the middle example.