44 ms·
A simple example: (i will use decimal instead of binary as it is easier and binary suffers from the same stuff but for other numbers) (10/3)x(9/4) == 7.5 but
by callesgg 13y ago
A simple example: (i will use decimal instead of binary as it is easier and binary suffers from the same stuff but for other numbers)
(10/3)x(9/4) == 7.5
but due to the fact that computers store the value instead of the fraction it would come out as something like 7.4999999999
cause instead of doing
(10x9)/(3x4) it would do 3.3333333333333 x 2.25
- giovannibajo1 13y agoThis looks like a case for decimal floating point, not fractionals. In Python: >>> from decimal import Decimal as D >>> D("10")/D("3") * D("9")/D("4") Decimal('7.50000000000000000000000000')