3 ms·

I never suggested this was a bug in how floating point works. I only meant I had a bug and it was due to my use of floats and how it bit me in the ass :) I am

I never suggested this was a bug in how floating point works. I only meant I had a bug and it was due to my use of floats and how it bit me in the ass :) I am still a little surprised (but not that surprised) that essentially 4.0 * 0.1 (and then later) / 0.1 didn't end up with a value that cast to int as 4. Sure I shouldn't make such presumptions, but if you wrote float floaty_var = 4.f; int inty_var = floaty_var; You would generally expect that inty_var would equal 4 and not 3. I have no idea if the IEEE standards mean that it will or it probably will or it very well might not. In my experience it always has, which is quite different from any certainty that 0.4 / 0.1 will cast to an int value of 4, but that's what happened when I tested with variables in a simple function, just not in the situation that led to my issue. Either way I'm well aware it was a PEBCAK, even if my semi-pun blog post title might have suggested otherwise ;)

CurtHagenlocher 14y ago
Computer math is not math. You can't generally assume that (a * b) / b will produce a. With floating-point, you have representational issues, and will all types you can get overflow.
- MikeDucker 14y ago
  I have certainly not had a bug that so plainly highlights this before. If it had been a bug about a more complex calculation than ( 4.0 * 0.1 ) / 0.1 I wouldn't have thought it worth mentioning. The fact that calc generally does result in a number that casts to 4 made it more interesting to me that in at least one case it didn't. I guess it was some state of the FPU (possibly) that gave a different result when there was sufficient time between the mult and the div. I will probably never know.
  - acqq 14y ago
    There's no "some state in the FPU" every partial result is repesented in 64 bits if you use doubles or in 32 bits if you use floats. Nothing is hidden "somewhere." It's especially nothing to "never know," every step can be followed exactly as long as you step through the assembly code or simply imagine that every partial result must also be represented as the FP number as specified not as you'd wish it to be. And don't think in decimal while you do the steps.
    - MikeDucker 14y ago
      I remember being told a long time ago by our graphics programmer that the result of a floating point operation might be different depending on the state of the FPU. A quick internet search brings up things like this: http://www.codercorner.com/FPUFun.htm http://www.codercorner.com/FPUFun.htm which talks about setting the default rounding mode of the FPU. Not sure if that's what he was talking about, but it certainly bears no reference to my issue as I'm certainly not explicitly setting anything like that. That being that case can you give a reasonable explanation why carrying out ( x * y ) then later that function / y gave a value for x that casting would keep as x, when leaving the y division until sometime much later during the run resulted in a value of x - 1? There must have been something different between the two runs apart from literal time, but if nothing is hidden, then what is the difference that made the difference?
      - acqq 14y ago
        As far as I understand, you are not sure what your code does when. If you'd follow the steps of the code you wrote, you'd see that when the same values travel the same code paths they end up being always exactly the same. In programming the different results can come only from the different steps, unless you change the environment (which would be your intentional rounding mode change or if you read some uninitialized memory).
        MikeDucker 14y ago
        I would follow the steps again (I am genuinely interested in learning why it does what it does), but I ripped out the scaling factor a few days ago and reintroducing it in the hope that the same thing happens would be a lot of work, hence asking if there was some obvious reason that might be causing what I'm seeing. There must be some step I'm not appreciating or some state that's different. If something like this happens again I'll take a closer look into it.
        acqq 14y ago
        When you cross the line from the integers to the ten times smaller values (step 1) you move from the finite universe to the "approximate" one, unless the staring number ended with 0 or 5 and you divided with the exact constant 10 (even multiplying with .1 is not for all inputs the same as dividing with 10 because "exactly .1" doesn't exist in base 2 FP -- as another commenter said it's not the math you know from the school -- look at the bits). Once you are in the approximate world (it's in fact exact for itself but from your starting position not what you had or you think you have) you can exactly follow when you get the expected result and when you get what you haven't expected, but the real cause is always that in the step 1 you've lost an infinite(!) number of bits. If you don't want to worry, just don't do divisions with 10. Divide with 8 or 16 or 32...

CurtHagenlocher 14y ago

Computer math is not math. You can't generally assume that (a * b) / b will produce a. With floating-point, you have representational issues, and will all types you can get overflow.

MikeDucker 14y ago
I have certainly not had a bug that so plainly highlights this before. If it had been a bug about a more complex calculation than ( 4.0 * 0.1 ) / 0.1 I wouldn't have thought it worth mentioning. The fact that calc generally does result in a number that casts to 4 made it more interesting to me that in at least one case it didn't. I guess it was some state of the FPU (possibly) that gave a different result when there was sufficient time between the mult and the div. I will probably never know.
- acqq 14y ago
  There's no "some state in the FPU" every partial result is repesented in 64 bits if you use doubles or in 32 bits if you use floats. Nothing is hidden "somewhere." It's especially nothing to "never know," every step can be followed exactly as long as you step through the assembly code or simply imagine that every partial result must also be represented as the FP number as specified not as you'd wish it to be. And don't think in decimal while you do the steps.
  - MikeDucker 14y ago
    I remember being told a long time ago by our graphics programmer that the result of a floating point operation might be different depending on the state of the FPU. A quick internet search brings up things like this: http://www.codercorner.com/FPUFun.htm http://www.codercorner.com/FPUFun.htm which talks about setting the default rounding mode of the FPU. Not sure if that's what he was talking about, but it certainly bears no reference to my issue as I'm certainly not explicitly setting anything like that. That being that case can you give a reasonable explanation why carrying out ( x * y ) then later that function / y gave a value for x that casting would keep as x, when leaving the y division until sometime much later during the run resulted in a value of x - 1? There must have been something different between the two runs apart from literal time, but if nothing is hidden, then what is the difference that made the difference?
    - acqq 14y ago
      As far as I understand, you are not sure what your code does when. If you'd follow the steps of the code you wrote, you'd see that when the same values travel the same code paths they end up being always exactly the same. In programming the different results can come only from the different steps, unless you change the environment (which would be your intentional rounding mode change or if you read some uninitialized memory).
      - MikeDucker 14y ago
        I would follow the steps again (I am genuinely interested in learning why it does what it does), but I ripped out the scaling factor a few days ago and reintroducing it in the hope that the same thing happens would be a lot of work, hence asking if there was some obvious reason that might be causing what I'm seeing. There must be some step I'm not appreciating or some state that's different. If something like this happens again I'll take a closer look into it.
        acqq 14y ago
        When you cross the line from the integers to the ten times smaller values (step 1) you move from the finite universe to the "approximate" one, unless the staring number ended with 0 or 5 and you divided with the exact constant 10 (even multiplying with .1 is not for all inputs the same as dividing with 10 because "exactly .1" doesn't exist in base 2 FP -- as another commenter said it's not the math you know from the school -- look at the bits). Once you are in the approximate world (it's in fact exact for itself but from your starting position not what you had or you think you have) you can exactly follow when you get the expected result and when you get what you haven't expected, but the real cause is always that in the step 1 you've lost an infinite(!) number of bits. If you don't want to worry, just don't do divisions with 10. Divide with 8 or 16 or 32...