3 ms·

sin(pi+eps) is a quantity on the order of eps. Not only is it non-zero, it is also so far from zero that it can be represented quite accurately with a floating-

sin(pi+eps) is a quantity on the order of eps. Not only is it non-zero, it is also so far from zero that it can be represented quite accurately with a floating-point number (which is a separate concern from the accuracy with which it is computed). The relative error introduced in rounding 1e-16 down to 0 is 100%, or approximately 1e16 ulp, which is why that would be unacceptable. Floating point arithmetic cares about relative errors much much more than absolute. Also, while dividing by zero is a problem, dividing by 1e-16 should not be a problem, at all. 1e-16 is a perfectly ordinary representable number.

colanderman 12y ago
[eps] is also so far from zero that it can be represented quite accurately with a floating-point number What is missing from your claim is where you say, "Specifically, the delta between pi and its float-point approximation is greater than half the minimum representable float." This is almost certainly true of pi, but it is NOT true of all real numbers! 1+(1e-1000), for example, fails this test. You cannot – logically! – make the claim that the error of any close approximations of real numbers should not round to zero because the error is not small enough; clearly in many cases it is! Yes, dividing by 1e-16 is not a problem! My question is whether this is a desirable thing to do in an algorithm that should be stable. Floating point arithmetic cares about relative errors much much more than absolute. This hints more at a reason, but why is this true? Yes, I know floating-point numbers are spaced logarithmically, and yes, relative error is obviously the correct measure when dealing with multiplication and division. But if you are not dividing due to stability concerns, so your value eventually finds its way to an addition, isn't absolute error more important?
- SamReidHughes 12y ago
  Nobody here besides you is talking about numbers that are less than half the minimum representable float. Edit: Oh my god now I get what you are saying. There are plenty of implicit reasons why the value would end up being non-zero -- in particular, any unit-scale continuous function would have to be artificially toxic for epsilon to be smaller than the smallest float. There is no reason for the article to point out that nit.
  - colanderman 12y ago
    Well, thanks at least for taking the time to understand my point. Though I don't consider it a "nit" when the article implies that "never round to zero" is some sort of rule (such as "round toward even LSBs" is), as opposed to a consequence of the properties of floats and the constants and functions involved.

colanderman 12y ago

[eps] is also so far from zero that it can be represented quite accurately with a floating-point number What is missing from your claim is where you say, "Specifically, the delta between pi and its float-point approximation is greater than half the minimum representable float." This is almost certainly true of pi, but it is NOT true of all real numbers! 1+(1e-1000), for example, fails this test. You cannot – logically! – make the claim that the error of any close approximations of real numbers should not round to zero because the error is not small enough; clearly in many cases it is! Yes, dividing by 1e-16 is not a problem! My question is whether this is a desirable thing to do in an algorithm that should be stable. Floating point arithmetic cares about relative errors much much more than absolute. This hints more at a reason, but why is this true? Yes, I know floating-point numbers are spaced logarithmically, and yes, relative error is obviously the correct measure when dealing with multiplication and division. But if you are not dividing due to stability concerns, so your value eventually finds its way to an addition, isn't absolute error more important?

SamReidHughes 12y ago
Nobody here besides you is talking about numbers that are less than half the minimum representable float. Edit: Oh my god now I get what you are saying. There are plenty of implicit reasons why the value would end up being non-zero -- in particular, any unit-scale continuous function would have to be artificially toxic for epsilon to be smaller than the smallest float. There is no reason for the article to point out that nit.
- colanderman 12y ago
  Well, thanks at least for taking the time to understand my point. Though I don't consider it a "nit" when the article implies that "never round to zero" is some sort of rule (such as "round toward even LSBs" is), as opposed to a consequence of the properties of floats and the constants and functions involved.