3 ms·

This isn't a bug ... you really need to understand how floating point numbers work if you're going to do ANYTHING with them. At one point, I wrote an entire si

by smoyer 14y ago

This isn't a bug ... you really need to understand how floating point numbers work if you're going to do ANYTHING with them. At one point, I wrote an entire single-precision IEEE compliant floating point library in assembly language and while that product failed (an embedded system), it's been invaluable in the 20 years since that project. When working with higher level languages, I completely understand how my numbers will be rounded, precision truncated, etc. Casting back and forth to integers is often problematic, but even more telling is that the programs that have to be very precise generally use fixed-point integers. Banking and accounting don't tend to like single "pixel" rounding errors!

toyg 14y ago
Banking and accounting don't tend to like single "pixel" rounding errors! Working in the field, I was actually shocked by how much people freely round up a million here and a million there, more often than not in official reports for the board and/or financial authorities. You wouldn't believe how many times I had to show floating-point "errors" in Excel, leaving young accountants close to tears. They only care about numbers "looking the same" and being able to justify most discrepancies.
acqq 14y ago
Correct. Not the bug of the floating point hardware. Floating point hardware which implements IEEE does what it is supposed to do. It's a "PEBKAC." Moreover, if OP understood the subject a little better, he'd know that 123 is stored exactly. Specifically, 123 is in binary 1111011. When stored in IEEE format you'll see in the mantissa bits the exact match: 111011 (the first bit is implicit). You can check it here: http://www.h-schmidt.net/FloatConverter/IEEE754.html http://www.h-schmidt.net/FloatConverter/IEEE754.html OP was bitten by assuming that decimal parts have exact representation, whereas from .1 to .9 only .5 has. Once he divides the exact 123 with the exact 10 he has to have 12.3 stored in the register, 12.3 has an infinite number of bits in the given representation and they have to be cut somewhere to fit any finite storage: 10001(0011)... If you check with the same converter, IEEE is careful enough to even know that after last 00 bits that fit in the 64-bits two 11 follow, so what's stored is 01 instead 00 for the last digits. The only solution for people who don't understand all the pitfalls of the standardized binary representation would be using base-10 floating point which is also standardized but sadly the last time I've checked existed only in some IBM hardware: http://en.wikipedia.org/wiki/IEEE_floating_point http://en.wikipedia.org/wiki/IEEE_floating_point (see IEEE 754-2008) The software support is equivalently rare.
- andrewcooke 14y ago
  from .1 to .9 only .5 has and .25 and 0.75 and... The only solution for people who don't understand and rational number libraries and multiple precision libraries and ...
  - acqq 14y ago
    I considered the results of the division of the integers by 10, then 0.25 is not there.
- MikeDucker 14y ago
  Hi acqq, I did type 123 into that float converter and the numbers I got back are what I posted. I'm not sure how we're getting different FP reps for the same number. I simply typed 123 into the decimal representation box. If I'm missing something please let me know.
  - andrewcooke 14y ago
    the problem is that for 0.123 (say) it's not 123 multiplied by 0.001 - the multiplier has to be a power of 2, so the thing being multiplied is no longer a simple integer. see my answer at http://stackoverflow.com/a/7289400/181772 http://stackoverflow.com/a/7289400/181772 for a similar case.
  - acqq 14y ago
    Read carefully. 123 is represented 100% exactly, bit for bit. Look at the bits in the mantissa, 111011, leading 1 is the implicit and it matches >>> bin( 123 ) '0b1111011' not at the "interpretation for humans" which shows 1.921875. All integers up to some very big one are represented exactly in IEEE base-2 FP (or you can say that the fractions that are the result from the storage of the integers have a finite binary representation, but then you make harder for you to have an easy insight). Disclaimer: I have similar experiences to smoyer, in the eighties of the previous century I've implemented a reasonable chunk of IEEE 754 in x86 assembly.
    - 14y ago
niggler 14y ago
Microsoft uses a 16-byte Decimal structure (with a 12 byte mantissa, 1 byte exponent, 1 bit sign) and recommends it for accounting. The arithmetic is treated as if it is base-10 fixed point

by smoyer 14y ago

toyg 14y ago

Banking and accounting don't tend to like single "pixel" rounding errors! Working in the field, I was actually shocked by how much people freely round up a million here and a million there, more often than not in official reports for the board and/or financial authorities. You wouldn't believe how many times I had to show floating-point "errors" in Excel, leaving young accountants close to tears. They only care about numbers "looking the same" and being able to justify most discrepancies.

acqq 14y ago

Correct. Not the bug of the floating point hardware. Floating point hardware which implements IEEE does what it is supposed to do. It's a "PEBKAC." Moreover, if OP understood the subject a little better, he'd know that 123 is stored exactly. Specifically, 123 is in binary 1111011. When stored in IEEE format you'll see in the mantissa bits the exact match: 111011 (the first bit is implicit). You can check it here: http://www.h-schmidt.net/FloatConverter/IEEE754.html http://www.h-schmidt.net/FloatConverter/IEEE754.html OP was bitten by assuming that decimal parts have exact representation, whereas from .1 to .9 only .5 has. Once he divides the exact 123 with the exact 10 he has to have 12.3 stored in the register, 12.3 has an infinite number of bits in the given representation and they have to be cut somewhere to fit any finite storage: 10001(0011)... If you check with the same converter, IEEE is careful enough to even know that after last 00 bits that fit in the 64-bits two 11 follow, so what's stored is 01 instead 00 for the last digits. The only solution for people who don't understand all the pitfalls of the standardized binary representation would be using base-10 floating point which is also standardized but sadly the last time I've checked existed only in some IBM hardware: http://en.wikipedia.org/wiki/IEEE_floating_point http://en.wikipedia.org/wiki/IEEE_floating_point (see IEEE 754-2008) The software support is equivalently rare.

andrewcooke 14y ago
from .1 to .9 only .5 has and .25 and 0.75 and... The only solution for people who don't understand and rational number libraries and multiple precision libraries and ...
- acqq 14y ago
  I considered the results of the division of the integers by 10, then 0.25 is not there.
MikeDucker 14y ago
Hi acqq, I did type 123 into that float converter and the numbers I got back are what I posted. I'm not sure how we're getting different FP reps for the same number. I simply typed 123 into the decimal representation box. If I'm missing something please let me know.
- andrewcooke 14y ago
  the problem is that for 0.123 (say) it's not 123 multiplied by 0.001 - the multiplier has to be a power of 2, so the thing being multiplied is no longer a simple integer. see my answer at http://stackoverflow.com/a/7289400/181772 http://stackoverflow.com/a/7289400/181772 for a similar case.
- acqq 14y ago
  Read carefully. 123 is represented 100% exactly, bit for bit. Look at the bits in the mantissa, 111011, leading 1 is the implicit and it matches >>> bin( 123 ) '0b1111011' not at the "interpretation for humans" which shows 1.921875. All integers up to some very big one are represented exactly in IEEE base-2 FP (or you can say that the fractions that are the result from the storage of the integers have a finite binary representation, but then you make harder for you to have an easy insight). Disclaimer: I have similar experiences to smoyer, in the eighties of the previous century I've implemented a reasonable chunk of IEEE 754 in x86 assembly.
  - 14y ago

niggler 14y ago

Microsoft uses a 16-byte Decimal structure (with a 12 byte mantissa, 1 byte exponent, 1 bit sign) and recommends it for accounting. The arithmetic is treated as if it is base-10 fixed point