3 ms·

I mention at the start that casting to double is not allowed. The motivation for that restriction is to allow the solution to generalize to double/int64 or even

I mention at the start that casting to double is not allowed. The motivation for that restriction is to allow the solution to generalize to double/int64 or even quad/int128. In a more practical sense, casting to a double is a good solution. Both int32s and floats can be represented faithfully as a double, and that's what I used to test my implementation against. However, the unreliable extra precision is still a problem. Casting to double may or may not remove it. You're still exposed to false positives and false negatives, depending on where and when precision is available. You should still run the float through a volatile float field, if you want consistent behavior. (Note: the volatile field doesn't have to be static. You can use a struct on the stack. That just took more code, and seemed like a superfluous detail. In C, instead of C#, you can just do a cast, instead of storing it in a field.)

nightowl03d 14y ago
Unless this is a numeric simulation where performance really matters, the correct answer is that you should not be using floats in the first place. Use a rational type, or scaled integers instead. If this is a numerical computation the comparison should be using an epsilon. Floats are strictly a performance optimization for doing computations whose solutions could be rational or irrational numbers. In all other cases you should either use integers or rational numbers. Bottom line is if you are not doing physics, graphics, signal processing, or system/financial modeling you should never use floats/doubles/quad If the need to do this comparison is because of a non-numerical third party library giving you a float, you should consider dropping the library. The library author, while not likely to be an idiot in the broader sense, is definitely a numerical idiot.
- Strilanc 14y ago
  I agree with everything you said. Well, one with clarification. Using an epsilon only partially solves the problem. It improves the situation, but there's still an unstable region where compiler quirks will flip the results of comparisons. The only way to avoid the issue is, as you said, using a more appropriate type like rational.
  - nightowl03d 14y ago
    I should have said a well chosen epsilon. If the computation is good to 0.001, (based on a detailed error propagation analysis of the algorithm against the expected input range), then I would use an epsilon of 1/512 when comparing against an int. The idea is that with a coarseness at that level the compiler rounding would likely have little effect. The comparison number would be (int + a power of two). But it sounds like for your scenario a scaled int or a rational would be more appropriate. If you have access to a university library, (most universities will let you sign in and browse), is Chapter One of Stoer and Bullrish text Introduction to Numerical Analysis. (ISBN 0-387-954452) It is on my desk because I am working out the epsilon for an algorithm a colleague designed. :-)

nightowl03d 14y ago

Unless this is a numeric simulation where performance really matters, the correct answer is that you should not be using floats in the first place. Use a rational type, or scaled integers instead. If this is a numerical computation the comparison should be using an epsilon. Floats are strictly a performance optimization for doing computations whose solutions could be rational or irrational numbers. In all other cases you should either use integers or rational numbers. Bottom line is if you are not doing physics, graphics, signal processing, or system/financial modeling you should never use floats/doubles/quad If the need to do this comparison is because of a non-numerical third party library giving you a float, you should consider dropping the library. The library author, while not likely to be an idiot in the broader sense, is definitely a numerical idiot.

Strilanc 14y ago
I agree with everything you said. Well, one with clarification. Using an epsilon only partially solves the problem. It improves the situation, but there's still an unstable region where compiler quirks will flip the results of comparisons. The only way to avoid the issue is, as you said, using a more appropriate type like rational.
- nightowl03d 14y ago
  I should have said a well chosen epsilon. If the computation is good to 0.001, (based on a detailed error propagation analysis of the algorithm against the expected input range), then I would use an epsilon of 1/512 when comparing against an int. The idea is that with a coarseness at that level the compiler rounding would likely have little effect. The comparison number would be (int + a power of two). But it sounds like for your scenario a scaled int or a rational would be more appropriate. If you have access to a university library, (most universities will let you sign in and browse), is Chapter One of Stoer and Bullrish text Introduction to Numerical Analysis. (ISBN 0-387-954452) It is on my desk because I am working out the epsilon for an algorithm a colleague designed. :-)