I find it illuminating to look at the exact values of the floating point numbers. This takes away an air of mystery: floating point numbers do in fact have ver

I find it illuminating to look at the exact values of the floating point numbers. This takes away an air of mystery: floating point numbers do in fact have very specific values, which we can inspect exactly. Here's 0.1 + 0.2 + 0.3: 0.1 (0.1000000000000000055511151231257827021181583404541015625) + 0.2 (0.200000000000000011102230246251565404236316680908203125) = 0.3000000000000000166533453693773481063544750213623046875 rounded: 0.3000000000000000444089209850062616169452667236328125 + 0.3 (0.299999999999999988897769753748434595763683319091796875) = 0.600000000000000033306690738754696212708950042724609375 rounded: 0.600000000000000088817841970012523233890533447265625 The operands can't be represented exactly as floating point so they are rounded to the nearest floating-point number before the addition even happens. And once the addition does happen, it is rounded to the nearest representable floating point value. Now we can look at the same for 0.1 + (0.2 + 0.3): 0.2 (0.200000000000000011102230246251565404236316680908203125) + 0.3 (0.299999999999999988897769753748434595763683319091796875) = 0.500000000000000000000000000000000000000000000000000000 rounded: 0.500000000000000000000000000000000000000000000000000000 + 0.1 (0.1000000000000000055511151231257827021181583404541015625) = 0.6000000000000000055511151231257827021181583404541015625 rounded: 0.59999999999999997779553950749686919152736663818359375 It's interesting to note that the first addition ends up being exact! This is a bit lucky, because 0.5 is exactly representable in floating point and the initial operands got rounded in opposite directions. It's also interesting that 0.6... only turns into 0.599999... at the point that we round the answer. You can play around with this stuff conveniently using the Python "Decimal" module. $ python Python 2.7.5 (default, Aug 25 2013, 00:04:04)- [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import Decimal, getcontext >>> getcontext().prec = 1000 # To prevent truncation/rounding >>> Decimal(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') >>> a = Decimal(0.1) + Decimal(0.2) >>> a Decimal('0.3000000000000000166533453693773481063544750213623046875') >>> Decimal(float(a)) # get the value rounded to the nearest fp number Decimal('0.3000000000000000444089209850062616169452667236328125') You can also use my tool "dumpfp" (https://github.com/haberman/dumpfp https://github.com/haberman/dumpfp) to inspect the value of a floating-point number in detail: $ ./dumpfp 0.1 Single Precision (IEEE 32-bit): raw = 0x3dcccccd sign = 0x0 exponent = 0x7b (-4) significand = 0x4ccccd VALUE CALCULATION = significand (1 + 5033165/2^23 (1.60000002384185791015625)) * 2^exponent (2^-4) = VALUE (13421773/2^27 (0.100000001490116119384765625)) Double Precision (IEEE 64-bit): raw = 0x3fb999999999999a sign = 0x0 exponent = 0x3fb (-4) significand = 0x999999999999a VALUE CALCULATION = significand (1 + 1351079888211149/2^51 (1.600000000000000088817841970012523233890533447265625)) * 2^exponent (2^-4) = VALUE (3602879701896397/2^55 (0.1000000000000000055511151231257827021181583404541015625))