4 ms·

if( abs(x - y) < 1e-6 ). Testing for absolute error is often incorrect. Here is a better floating point comparator: http://floating-point-gui.de/errors/compar

if( abs(x - y) < 1e-6 ). Testing for absolute error is often incorrect. Here is a better floating point comparator: http://floating-point-gui.de/errors/comparison/ http://floating-point-gui.de/errors/comparison/

mortoray 11y ago
I (the author) agree. I'm mainly trying to point out all the issues with floating point rather than give a full solution. I believe the problem stems from people just not realizing a lot of these problems. Certainly I'm not offering the complete solution here, and for any one of those points could spawn off several articles just to give ideas. So yes, links are appreciated so readers can get more into the details.
danieldk 11y ago
I thought we had to use ulps and (worst-case) error bounds specified by a library in ulps? https://en.wikipedia.org/wiki/Unit_in_the_last_place https://en.wikipedia.org/wiki/Unit_in_the_last_place http://docs.oracle.com/javase/7/docs/api/java/lang/Math.html http://docs.oracle.com/javase/7/docs/api/java/lang/Math.html
- shoo 11y ago
  I am not sure I understand what you are saying exactly, but I believe in some circumstances you may be working with numerical values which you know to have errors > ulps. For example, suppose you working with data that you've obtained from some physical measurement, and your knowledge of the measurement process means you know it will have around 1% relative error. Or another example: suppose you are solving a linear system of equations Ax=b with highly precise right-hand-side data b (< \epsilon floating-point error), but the condition number of the matrix is large. Then perhaps it is only meaningful to expect the solution to be accurate to within k(A) \epsilon, the condition number of A. (I haven't thought about this stuff for a while, please correct me if this is wrong). edit: here's another reason. Suppose there's a heavy tradeoff between runtime and accuracy for some algorithm, so for some application it makes sense to use some cheap but somewhat inaccurate approximation. Then you're probably going to have and expect far larger errors. But, you might still be able to get some kind of (worst case?) bounds on the result, and want to assert something about that in test cases or sanity checks.
exDM69 11y ago
> Testing for absolute error is often incorrect True, but in this case it was to test for near-zero values so absolute error comparison is good enough. Here's a few macros I have been using to compare floating point values for approximate equality. It should be true if the values are equal to about 6-7 decimal places (in base-10). I use this in testing some code that deals with algorithms that have known numerical precision issues. The errors are inherent to the algorithms, not specifically floating point errors (e.g. solving Kepler's equation for near-parabolic trajectories). Note that it isn't specified whether "a" or "b" is the "gold" value to compare against, so it's measuring the relative error against a rough average of the two magnitudes. #define ZEROF(x) ((x)*(x) < DBL_EPSILON) #define EQF(a, b) ((ZEROF(a) && ZEROF(b)) || \ ZEROF(((a)-(b))*((a)-(b))/((a)*(a) + ((b)*(b))))) I am no scientist, so if anyone has improvement to the code above, I'll gladly take any feedback.
- stinos 11y ago
  but in this case it was to test for near-zero values so absolute error comparison is good enough Unless you are working with small numbers? Suppose you are dividing 1e-30 by 1e-30 than the result should still be 1 so if you return 0 when the second argument is < 1e-6 then you're screwed. I think boost's close_at_tolerance does what your macro tries to do; basically you say 'here's two numbers, check if their values differ no more than x% of their value' With the advantage that it is not a macro and guards against overflow etc: http://www.boost.org/doc/libs/1_37_0/boost/test/floating_point_comparison.hpp http://www.boost.org/doc/libs/1_37_0/boost/test/floating_poi....
  - exDM69 11y ago
    > Unless you are working with small numbers? Suppose you are dividing 1e-30 by 1e-30 than the result should still be 1 so if you return 0 when the second argument is < 1e-6 then you're screwed. A very good point which I didn't think of! Obviously the 1e-6 figure is specific to the application. How should one avoid division by zero then? You can't figure out the relative error without a division, can you? For normal comparisons, using a relative error is the way to go but the zero case is different. On the other hand, using quantities of magnitudes in 1e-30 is already using half the exponent, a change of units may be due here.
cnvogel 11y ago
I've seen those "abs(fp_number)<some_tiny_value" in front of divisions, because people fear that the division will "blow up" in some way if a zero or sufficiently small number is passed to it. if (abs(q) >= 1e-6) rel_val = abs_val / q; else overflow_flag = true; But it's quite pointless, as the semantics of floating point calculations give you the indication about overflow right in the division result, it will be marked as positive or negative infinity. rel_val = abs_val / q; if (isnan(rel_val) || !finite(rel_val) || ans(rel_val) > some_limit) overflow_flag = true; (of course your FPU should be configured to not produce exceptions in the case of division by zero)
- arielb1 11y ago
  That still won't prevent a zero-ish noisy `q` from causing a totally messed-up `rel_val`.

mortoray 11y ago

I (the author) agree. I'm mainly trying to point out all the issues with floating point rather than give a full solution. I believe the problem stems from people just not realizing a lot of these problems. Certainly I'm not offering the complete solution here, and for any one of those points could spawn off several articles just to give ideas. So yes, links are appreciated so readers can get more into the details.

danieldk 11y ago

I thought we had to use ulps and (worst-case) error bounds specified by a library in ulps? https://en.wikipedia.org/wiki/Unit_in_the_last_place https://en.wikipedia.org/wiki/Unit_in_the_last_place http://docs.oracle.com/javase/7/docs/api/java/lang/Math.html http://docs.oracle.com/javase/7/docs/api/java/lang/Math.html

shoo 11y ago
I am not sure I understand what you are saying exactly, but I believe in some circumstances you may be working with numerical values which you know to have errors > ulps. For example, suppose you working with data that you've obtained from some physical measurement, and your knowledge of the measurement process means you know it will have around 1% relative error. Or another example: suppose you are solving a linear system of equations Ax=b with highly precise right-hand-side data b (< \epsilon floating-point error), but the condition number of the matrix is large. Then perhaps it is only meaningful to expect the solution to be accurate to within k(A) \epsilon, the condition number of A. (I haven't thought about this stuff for a while, please correct me if this is wrong). edit: here's another reason. Suppose there's a heavy tradeoff between runtime and accuracy for some algorithm, so for some application it makes sense to use some cheap but somewhat inaccurate approximation. Then you're probably going to have and expect far larger errors. But, you might still be able to get some kind of (worst case?) bounds on the result, and want to assert something about that in test cases or sanity checks.

exDM69 11y ago

> Testing for absolute error is often incorrect True, but in this case it was to test for near-zero values so absolute error comparison is good enough. Here's a few macros I have been using to compare floating point values for approximate equality. It should be true if the values are equal to about 6-7 decimal places (in base-10). I use this in testing some code that deals with algorithms that have known numerical precision issues. The errors are inherent to the algorithms, not specifically floating point errors (e.g. solving Kepler's equation for near-parabolic trajectories). Note that it isn't specified whether "a" or "b" is the "gold" value to compare against, so it's measuring the relative error against a rough average of the two magnitudes. #define ZEROF(x) ((x)*(x) < DBL_EPSILON) #define EQF(a, b) ((ZEROF(a) && ZEROF(b)) || \ ZEROF(((a)-(b))*((a)-(b))/((a)*(a) + ((b)*(b))))) I am no scientist, so if anyone has improvement to the code above, I'll gladly take any feedback.

stinos 11y ago
but in this case it was to test for near-zero values so absolute error comparison is good enough Unless you are working with small numbers? Suppose you are dividing 1e-30 by 1e-30 than the result should still be 1 so if you return 0 when the second argument is < 1e-6 then you're screwed. I think boost's close_at_tolerance does what your macro tries to do; basically you say 'here's two numbers, check if their values differ no more than x% of their value' With the advantage that it is not a macro and guards against overflow etc: http://www.boost.org/doc/libs/1_37_0/boost/test/floating_point_comparison.hpp http://www.boost.org/doc/libs/1_37_0/boost/test/floating_poi....
- exDM69 11y ago
  > Unless you are working with small numbers? Suppose you are dividing 1e-30 by 1e-30 than the result should still be 1 so if you return 0 when the second argument is < 1e-6 then you're screwed. A very good point which I didn't think of! Obviously the 1e-6 figure is specific to the application. How should one avoid division by zero then? You can't figure out the relative error without a division, can you? For normal comparisons, using a relative error is the way to go but the zero case is different. On the other hand, using quantities of magnitudes in 1e-30 is already using half the exponent, a change of units may be due here.

cnvogel 11y ago

I've seen those "abs(fp_number)<some_tiny_value" in front of divisions, because people fear that the division will "blow up" in some way if a zero or sufficiently small number is passed to it. if (abs(q) >= 1e-6) rel_val = abs_val / q; else overflow_flag = true; But it's quite pointless, as the semantics of floating point calculations give you the indication about overflow right in the division result, it will be marked as positive or negative infinity. rel_val = abs_val / q; if (isnan(rel_val) || !finite(rel_val) || ans(rel_val) > some_limit) overflow_flag = true; (of course your FPU should be configured to not produce exceptions in the case of division by zero)

arielb1 11y ago
That still won't prevent a zero-ish noisy `q` from causing a totally messed-up `rel_val`.