3 ms·
I agree with everything you said. Well, one with clarification. Using an epsilon only partially solves the problem. It improves the situation, but there's stil
by Strilanc 14y ago
I agree with everything you said.
Well, one with clarification. Using an epsilon only partially solves the problem. It improves the situation, but there's still an unstable region where compiler quirks will flip the results of comparisons. The only way to avoid the issue is, as you said, using a more appropriate type like rational.
- nightowl03d 14y agoI should have said a well chosen epsilon. If the computation is good to 0.001, (based on a detailed error propagation analysis of the algorithm against the expected input range), then I would use an epsilon of 1/512 when comparing against an int. The idea is that with a coarseness at that level the compiler rounding would likely have little effect. The comparison number would be (int + a power of two). But it sounds like for your scenario a scaled int or a rational would be more appropriate. If you have access to a university library, (most universities will let you sign in and browse), is Chapter One of Stoer and Bullrish text Introduction to Numerical Analysis. (ISBN 0-387-954452) It is on my desk because I am working out the epsilon for an algorithm a colleague designed. :-)