5 ms·

Example 7 really got me, can anyone explain that? I’m not sure how “modulo” operation would be implemented in hardware, if it is a native instruction or not, bu

Example 7 really got me, can anyone explain that? I’m not sure how “modulo” operation would be implemented in hardware, if it is a native instruction or not, but one would hope it would give a result consistent with the matching divide operation. Edit: x87 has FPREM1 which can calculate a remainder (accurately one hopes), but I can’t find an equivalent in modern SSE or AVX. So I guess you are at the mercy of your language’s library and/or compiler? Is this a library/language bug rather than a Floating Point gotcha?

timerol 4y ago
Based on the nearest numbers that floats represent, the two numbers are Y = 13.715999603271484375 (https://float.exposed/0x415b74bc https://float.exposed/0x415b74bc) and X = 4.57200002670288085938 (https://float.exposed/0x40924dd3 https://float.exposed/0x40924dd3). The division of these numbers is 2.9999998957049091386350361962468173875300478102103478639802753918, but the nearest float to that is 3. (Exactly 3.) [2] The modulo operation can (presumably) determine that 3X > Y, so the modulo is Y - 2X, as normal. This gives inconsistent results, if you don't know that every float is actually a range, and "3" as a float includes some numbers that are smaller than 3. [1] https://www.wolframalpha.com/input?i=13.715999603271484375+%2F+4.57200002670288085938 https://www.wolframalpha.com/input?i=13.715999603271484375+%... [2] https://www.wolframalpha.com/input?i=2.9999998957049091386350361962468173875300478102103478639802753918+as+float+in+big+endian https://www.wolframalpha.com/input?i=2.999999895704909138635..., then https://float.exposed/0x40400000 https://float.exposed/0x40400000
- svat 4y ago
  This is useful but note that Python uses 64-bit floats (aka "double"), so the right values are: • "13.716" means 13.7159999999999993037 (https://float.exposed/0x402b6e978d4fdf3b https://float.exposed/0x402b6e978d4fdf3b) • "4.572" means 4.57200000000000006395 (https://float.exposed/0x401249ba5e353f7d https://float.exposed/0x401249ba5e353f7d) • "13.716 / 4.572" means the nearest representable value to 13.7159999999999993037 / 4.57200000000000006395 which (https://www.wolframalpha.com/input?i=13.7159999999999993037+%2F+4.57200000000000006395 https://www.wolframalpha.com/input?i=13.7159999999999993037+...) is 3.0 (https://float.exposed/0x4008000000000000 https://float.exposed/0x4008000000000000) • "13.716 % 4.572" means the nearest representable value to 13.7159999999999993037 % 4.57200000000000006395 namely to 4.5719999999999991758 (https://www.wolframalpha.com/input?i=13.7159999999999993037+mod+4.57200000000000006395 https://www.wolframalpha.com/input?i=13.7159999999999993037+...), which is 4.57199999999999917577 (https://float.exposed/0x401249ba5e353f7c https://float.exposed/0x401249ba5e353f7c) printed as 4.571999999999999. ———————— Edit: For a useful analogy (answering the GP), imagine you're working in decimal fixed-point arithmetic with two decimal digits (like dollars and cents), and someone asks you for 10.01/3.34 and 10.01%3.34. Well, • 10.01 / 3.34 is well over 2.99 (it's over 2.997 in fact) so you'd be justified in answering 3.00 (the nearest representable value). • 10.01 % 3.34 is 3.33 (which you can represent exactly), so you'd answer 3.33 to that one. (For an even bigger difference: try 19.99 and 6.67 to get 3.00 as quotient, but 6.65 as remainder.)
adrian_b 4y ago
This has nothing to do with the definition or implementation of the remainder or modulo function. It is a problem that appears whenever you compose an inexact function, like the conversion from decimal to binary, with a function that is not continuous, like the remainder a.k.a. modulo function. In decimal, 13.716 is exactly 3 times 4.572, so any kind of remainder must be null, but after conversion from decimal to binary that relationship is no longer true, and because the remainder is not a continuous function its value may be wildly different from the correct value. When you compute with approximate numbers, like the floating-point numbers, as long as you compose only continuous functions, the error in the final result remains bounded and smaller errors in inputs lead to a diminished error in the output. However, it is enough to insert one discontinuous function in the computation chain for losing any guarantee about the magnitude of the error in the final result. The conclusion is that whenever computing with approximate numbers (which may also use other representations, not only floating-point) you have to be exceedingly cautious when using any function that is not continuous.

timerol 4y ago

Based on the nearest numbers that floats represent, the two numbers are Y = 13.715999603271484375 (https://float.exposed/0x415b74bc https://float.exposed/0x415b74bc) and X = 4.57200002670288085938 (https://float.exposed/0x40924dd3 https://float.exposed/0x40924dd3). The division of these numbers is 2.9999998957049091386350361962468173875300478102103478639802753918, but the nearest float to that is 3. (Exactly 3.) [2] The modulo operation can (presumably) determine that 3X > Y, so the modulo is Y - 2X, as normal. This gives inconsistent results, if you don't know that every float is actually a range, and "3" as a float includes some numbers that are smaller than 3. [1] https://www.wolframalpha.com/input?i=13.715999603271484375+%2F+4.57200002670288085938 https://www.wolframalpha.com/input?i=13.715999603271484375+%... [2] https://www.wolframalpha.com/input?i=2.9999998957049091386350361962468173875300478102103478639802753918+as+float+in+big+endian https://www.wolframalpha.com/input?i=2.999999895704909138635..., then https://float.exposed/0x40400000 https://float.exposed/0x40400000

svat 4y ago
This is useful but note that Python uses 64-bit floats (aka "double"), so the right values are: • "13.716" means 13.7159999999999993037 (https://float.exposed/0x402b6e978d4fdf3b https://float.exposed/0x402b6e978d4fdf3b) • "4.572" means 4.57200000000000006395 (https://float.exposed/0x401249ba5e353f7d https://float.exposed/0x401249ba5e353f7d) • "13.716 / 4.572" means the nearest representable value to 13.7159999999999993037 / 4.57200000000000006395 which (https://www.wolframalpha.com/input?i=13.7159999999999993037+%2F+4.57200000000000006395 https://www.wolframalpha.com/input?i=13.7159999999999993037+...) is 3.0 (https://float.exposed/0x4008000000000000 https://float.exposed/0x4008000000000000) • "13.716 % 4.572" means the nearest representable value to 13.7159999999999993037 % 4.57200000000000006395 namely to 4.5719999999999991758 (https://www.wolframalpha.com/input?i=13.7159999999999993037+mod+4.57200000000000006395 https://www.wolframalpha.com/input?i=13.7159999999999993037+...), which is 4.57199999999999917577 (https://float.exposed/0x401249ba5e353f7c https://float.exposed/0x401249ba5e353f7c) printed as 4.571999999999999. ———————— Edit: For a useful analogy (answering the GP), imagine you're working in decimal fixed-point arithmetic with two decimal digits (like dollars and cents), and someone asks you for 10.01/3.34 and 10.01%3.34. Well, • 10.01 / 3.34 is well over 2.99 (it's over 2.997 in fact) so you'd be justified in answering 3.00 (the nearest representable value). • 10.01 % 3.34 is 3.33 (which you can represent exactly), so you'd answer 3.33 to that one. (For an even bigger difference: try 19.99 and 6.67 to get 3.00 as quotient, but 6.65 as remainder.)

adrian_b 4y ago

This has nothing to do with the definition or implementation of the remainder or modulo function. It is a problem that appears whenever you compose an inexact function, like the conversion from decimal to binary, with a function that is not continuous, like the remainder a.k.a. modulo function. In decimal, 13.716 is exactly 3 times 4.572, so any kind of remainder must be null, but after conversion from decimal to binary that relationship is no longer true, and because the remainder is not a continuous function its value may be wildly different from the correct value. When you compute with approximate numbers, like the floating-point numbers, as long as you compose only continuous functions, the error in the final result remains bounded and smaller errors in inputs lead to a diminished error in the output. However, it is enough to insert one discontinuous function in the computation chain for losing any guarantee about the magnitude of the error in the final result. The conclusion is that whenever computing with approximate numbers (which may also use other representations, not only floating-point) you have to be exceedingly cautious when using any function that is not continuous.