3 ms·
No, it isn’t. The code doing the rounding cannot know that you computed 0.1025 × 14 and not, for example, 0.1025 × 14 - 0.000000000000000001 to get the
by Someone 3y ago
No, it isn’t. The code doing the rounding cannot know that you computed
0.1025 × 14
and not, for example,
0.1025 × 14 - 0.000000000000000001
to get the IEEE float that prints as
1.4349999999999998
All it knows is that it got that float, and its value is less than 1.435.
- ktowen 3y agoI will only be doing operations with numbers with max 4 decimal places
- ksherlock 3y agoYou might consider fixed-point arithmetic with a 1/10000 scaling.
- Someone 3y agoDoesn’t matter. If you use a fixed amount of bits to store a number and want to support even basic operations there always are calculations that lose precision. For example, 12.34 × 12.34 = 152.2756; 12.33 × 12.35 = 152.2755. In four decimal places, both become 152.3.