4 ms·

> The IEEE guarantee is not that the result is accurate to a relative epsilon but instead is the closest representable number to the true result. The latter im

by kd5bjo 6y ago

> The IEEE guarantee is not that the result is accurate to a relative epsilon but instead is the closest representable number to the true result. The latter implies the former within the magnitude range that allows normalized representation, which is the vast majority of calculations. As an introductory article for a technical audience, it does a good job of describing the abilities and limitations of floating point numbers. Going into more detail about the representation runs the risk of sacrificing clarity for precision.

tgb 6y ago
I disagree and think that the article manages to fall into an "uncanny valley" of not-actually-understanding. Why else would it need the "A Cautionary Tale" section at the end? That's the first time he mentions that the previous sections only hold for positive numbers and shows that the statements made before are bad. And personally I prefer an "exact" guarantee that the result is as close-as-possible rather than merely "within some relative epsilon". Compare his (incorrect) "in mathematical terms" description with the actual statement that would be that, if re(x) gives the real number represented by a floating point number x, then: fl(re(x) + re(y)) = x + y where addition on the left is of reals and on the right is the floating point addition operator. This is simpler and more accurate than his inequality! And it works for sub-normals etc.
- johnbcoughlin 6y ago
  The axiom holds for all numbers and all single operations. The final section presents a gotcha that arises when reasoning about the amount of _propagated_ error in certain calculations (called catastrophic cancellation). The "exact" guarantee that you mention is precisely the same as saying that it's "within some relative epsilon". Using machine epsilon tells you _how_ close "as close as possible" is. Your proposed improvement to the inequality is true, but is not really useful for computing the amount of error you can expect from numerical algorithms. For that, you want some quantification of how close is "as close as possible". Having the relative error bounded by machine epsilon is an extremely strong guarantee, as the article shows.
  - tgb 6y ago
    The key I think your article is missing though is that it should explain which numbers are representable exactly as floating points. This is very important information for other reasons too and that tells you how close "as close as possible" is. I can tell you that when I started learning about floating point numbers I knew there was some relative error and believed that operations like addition and square root were just "close enough"- in fact they're as good as possible and the only difficulty is which numbers we're allowed to write down! Enlightenment! And everything else falls out from there. I do think your edit is a good clarification to make. My point about the "in mathematical notation" part though is that it assumes a and b are exactly represented as floating point numbers (else catastrophic cancellation can happen) though the sentence after the equation implies they are not representable.
- kd5bjo 6y ago
  > I prefer an "exact" guarantee that the result is as close-as-possible rather than merely "within some relative epsilon". This is the core requirement of any system that quantizes the real numbers in order to perform arithmetic. The thing that distinguishes fixed-point from floating-point systems is the quantization error that gets introduced in this process: For floating point, the error is proportional to the magnitude of the number being reprsented. The article is primarily exploring the implications of this property. For fixed point, the quantization error has an absolute magnitude that doesn’t depend on the size of the input numbers. This has a completely different set of benefits and drawbacks, but still maintains your “preferred” property.

by kd5bjo 6y ago

tgb 6y ago

I disagree and think that the article manages to fall into an "uncanny valley" of not-actually-understanding. Why else would it need the "A Cautionary Tale" section at the end? That's the first time he mentions that the previous sections only hold for positive numbers and shows that the statements made before are bad. And personally I prefer an "exact" guarantee that the result is as close-as-possible rather than merely "within some relative epsilon". Compare his (incorrect) "in mathematical terms" description with the actual statement that would be that, if re(x) gives the real number represented by a floating point number x, then: fl(re(x) + re(y)) = x + y where addition on the left is of reals and on the right is the floating point addition operator. This is simpler and more accurate than his inequality! And it works for sub-normals etc.

johnbcoughlin 6y ago
The axiom holds for all numbers and all single operations. The final section presents a gotcha that arises when reasoning about the amount of _propagated_ error in certain calculations (called catastrophic cancellation). The "exact" guarantee that you mention is precisely the same as saying that it's "within some relative epsilon". Using machine epsilon tells you _how_ close "as close as possible" is. Your proposed improvement to the inequality is true, but is not really useful for computing the amount of error you can expect from numerical algorithms. For that, you want some quantification of how close is "as close as possible". Having the relative error bounded by machine epsilon is an extremely strong guarantee, as the article shows.
- tgb 6y ago
  The key I think your article is missing though is that it should explain which numbers are representable exactly as floating points. This is very important information for other reasons too and that tells you how close "as close as possible" is. I can tell you that when I started learning about floating point numbers I knew there was some relative error and believed that operations like addition and square root were just "close enough"- in fact they're as good as possible and the only difficulty is which numbers we're allowed to write down! Enlightenment! And everything else falls out from there. I do think your edit is a good clarification to make. My point about the "in mathematical notation" part though is that it assumes a and b are exactly represented as floating point numbers (else catastrophic cancellation can happen) though the sentence after the equation implies they are not representable.
kd5bjo 6y ago
> I prefer an "exact" guarantee that the result is as close-as-possible rather than merely "within some relative epsilon". This is the core requirement of any system that quantizes the real numbers in order to perform arithmetic. The thing that distinguishes fixed-point from floating-point systems is the quantization error that gets introduced in this process: For floating point, the error is proportional to the magnitude of the number being reprsented. The article is primarily exploring the implications of this property. For fixed point, the quantization error has an absolute magnitude that doesn’t depend on the size of the input numbers. This has a completely different set of benefits and drawbacks, but still maintains your “preferred” property.