4 ms·

As to [2] I am very skeptical of the value of all the NaNs, -0 and Infs floating point has. NaN breaks x==x which is a pretty fundamental relationship for numbe

As to [2] I am very skeptical of the value of all the NaNs, -0 and Infs floating point has. NaN breaks x==x which is a pretty fundamental relationship for numbers to have. +-Inf sound useful in theory, but in practice they rarely give you a more useful result than NaN or the maximum/minimum value of your type (returning Inf on overflow has infinitely more error than returning the largest positive value, and if that isn't meaningful than an Inf probably wasn't either). Once you've gotten rid of -Inf, it becomes clear that -0.0 is a mistake. It breaks the identity 0+x==x and 0-x==-x. Furthermore, IEEE specifies sqrt(-0.0)==-0.0 and log(-0.0)==-Inf which are both nonsensical if you consider -0.0 as a limit from the negatives. Floats also have the unfortunate property that inv(x) can be infinite for finite x.

SideQuark 4y ago
>NaN breaks x==x which is a pretty fundamental relationship for numbers to have NaN is not a number, so it should NOT satisfy "fundamental relationships for numbers to have". >+-Inf sound useful in theory, but in practice they rarely give you a more useful resu There are algorithms that are more performant using infs, and without having a way to denote overflow, you'd have to pre-check evedry operation to do serious numerical work, which basically cuts your performance in half. >Once you've gotten rid of -Inf, it becomes clear that -0.0 is a mistake >It breaks the identity 0+x==x and 0-x==-x. No, you have some fundamental misunderstanding. IEEE explicitly guarantees these hold, even for -0. > Furthermore, IEEE specifies sqrt(-0.0)==-0.0 and log(-0.0)==-Inf which are both nonsensical if you consider -0.0 as a limit from the negatives. You're making up strawmen. -0 is not a "limit from the negatives" any more than +0 is a limit form the positives, which would break other made up requirements. That is why making up stuff that has zero bearing on what IEEE 754 specifies is arguing strawmen. >Floats also have the unfortunate property that inv(x) can be infinite for finite x. Integers have the same property: -(X) can not be the negative of X. So this is not a problem except in made up goofiness. Every objection you post is a lack of understanding numerical analysis and the needs of actual scientific software. So you're skeptical- do you write numerical software professionally? I do, and have, and will do it in the future. There are very, very good reasons for all of those pieces you don't see the need for. There's a reason unums have not caught on with the field of numerical software or numerical analysis - they simply don't allow writing robust, performant software, they solve no real issues, and add significant problems.
- adgjlsfhk1 4y ago
  >so it should NOT satisfy "fundamental relationships for numbers to have". If you have a list with a NaN in it, how should you make sort terminate (and where should the NaN end up)? I understand that in theory it is kind of arguable that NaN should be different, but breaking the total order is a really dumb decision. >you'd have to pre-check evedry operation to do serious numerical work, which basically cuts your performance in half. Can you give an example? Saturating overflow tends to do the same thing. >IEEE explicitly guarantees these hold This is kind of true. -0.0+0.0==0.0 and 0.0-0.0==0.0. IEEE does define -0.0==0.0 so IEEE does technically make this hold, but only by redefining == so that two different numbers are == > -0 is not a "limit from the negatives" Then what is it? it's not a real number, and Kahan's justification of them comes from branch cuts of analytic functions which is only makes sense in the context of limits https://homes.cs.washington.edu/~ztatlock/599z-17sp/papers/branch-cuts-kahan-87.pdf https://homes.cs.washington.edu/~ztatlock/599z-17sp/papers/b... > Integers have the same property Yeah and it sucks there too. In the fp case it makes it really annoying to do things like calculate divide an array by a float quickly and accurately. You would want to take the inverse of the divisor and multiply by that, but doing so isn't safe if the divisor is subnormal. Yes. My day job is in solving Differential Algebraic equations, but I also have written a bunch of Julia's Libm.
  - SideQuark 4y ago
    >If you have a list with a NaN in it, how should you make sort terminate Do whatever you want. If you're sorting floats, sort them to the front. Every language I've ever used for developing numerical software has a trivial IsNaN equivalent. So that's not a complaint worthy of claiming NaNs are not useful. I've written lots of numerical software and not once has this been an issue for me. What value do you assign sqrt of a negative without some NaN type item? Or any of tons of other "not a number" results? >so IEEE does technically make this hold, but only by redefining == There's no "redefining ==" here. You are upset that bit patterns are different, but == is not for bit patterns. You are confusing == for floats with == for bit patterns, which are not and need not be the same thing. I've never seen a language that gets these confused. If you want float ==, simply use language ==. If you want bitwise ==, then you usually have to do (often not portable) fiddling to convert to a bit pattern. It's like claiming reference == and structure field == should be the same, but both have uses. So languages have all sorts of ways to use the concept of equality, and they are all useful. Confusing them does not make the ones you don't like invalid or not extremely useful for people that do understand and use them. >Yes. My day job is in solving Differential Algebraic equations, but I also have written a bunch of Julia's Libm. Good. Then you should understand why, as an example, C++ std lib has a massive amount of functions like fma, expm1, log1p, hypot, and many more. Sure you can simply write log(1+x) instead of using log1p, but log1p is vastly better in this case because properties of IEEE 754 allow more precision. instead of hypot(x,y) you could write sqrt(xx+yy), but hypot is much better. These functions exist since IEEE provides tools to analyze these and make much better versions than the naive way to write them. Unums, with varying precision, make this vastly harder (and losing precision over the domain, making it hard to analyze anything). So unums, with varying precision, violate fundamental properties for scientific computing, namely, they lose precision in really messy ways. You cannot start with P digits of precision and do even simple math and get an answer with P digits of precision. IEEE does allow this. For example, sqrt(x^2)=|x| in IEEE (for no under/overflow). This does not work in unums, since they lose precision. Square something and lose digits. Fundamental to lots of scientific computation is the requirement to maintain precision throughout a calculation. Unums fail this spectacularly, making it incredibly messy to do correct scientific work.
jcranmer 4y ago
The value of -0 as distinguished from +0 has a few uses. The most obvious one is preserving sign in the case of overflow. A less obvious use case is handling branch cuts. There are uses in a few more cases: I've heard it's occasionally useful in things like coordinate systems, since something like "0°5'3" W" can be stored as (-0.0, 5, 3) after explosion and still display correctly. It's definitely niche, but it does have its uses. Returning a distinct value that retains the fact that it overflowed is quite useful--if you get that result out of the computation, you know you overflowed the computation rather than silently getting a meaningless result. Note in particular that infinities end up being sticky values: once a value goes infinite, it tends to stay infinity, which isn't true for largest finite values. Distinguishing between various kinds of "invalid" values turns out to be moderately useful in practice--I've used infinities a couple of times in my own code. NaNs are useful in representing a different kind of error than overflowed computation. Now there is a lot of room to criticize IEEE 754 here: "x != x" was quite frankly a mistake (basically the primary reason for it was the creators wanted to make testing for NaN easier than calling isnan(x)...). sNaNs are of course an abomination that just makes things worse. Multiple NaN payloads were originally intended (in part) to let developers debug the sources of NaNs, but this requires support that never really materialized. However, NaN payloads did find new use in making NaN-boxing a useful technique, and dedicating an entire exponent to special values simplifies several numerical analysis lemmas.
- adgjlsfhk1 4y ago
  > handling branch cuts I agree this sounds great in theory, but I don't think it works very well in practice. i.e. what about 1/(x+1)? Also branch cuts matter most for complex arithmetic, and there +-0 doesn't help since you don't know the phase of the zero. Also, realistically, floating point has finite precision so there are very few non-toy examples where you can do an actual computation and reliably end up on the correct branch. I'd rather have all the real numbers represented before we start adding hyper-reals to the number system. > Returning a distinct value that retains the fact that it overflowed is quite useful Agreed, and I think that NaR in Posits does a good job of that while not taking a ridiculous number of values.
  - 4y ago

SideQuark 4y ago

>NaN breaks x==x which is a pretty fundamental relationship for numbers to have NaN is not a number, so it should NOT satisfy "fundamental relationships for numbers to have". >+-Inf sound useful in theory, but in practice they rarely give you a more useful resu There are algorithms that are more performant using infs, and without having a way to denote overflow, you'd have to pre-check evedry operation to do serious numerical work, which basically cuts your performance in half. >Once you've gotten rid of -Inf, it becomes clear that -0.0 is a mistake >It breaks the identity 0+x==x and 0-x==-x. No, you have some fundamental misunderstanding. IEEE explicitly guarantees these hold, even for -0. > Furthermore, IEEE specifies sqrt(-0.0)==-0.0 and log(-0.0)==-Inf which are both nonsensical if you consider -0.0 as a limit from the negatives. You're making up strawmen. -0 is not a "limit from the negatives" any more than +0 is a limit form the positives, which would break other made up requirements. That is why making up stuff that has zero bearing on what IEEE 754 specifies is arguing strawmen. >Floats also have the unfortunate property that inv(x) can be infinite for finite x. Integers have the same property: -(X) can not be the negative of X. So this is not a problem except in made up goofiness. Every objection you post is a lack of understanding numerical analysis and the needs of actual scientific software. So you're skeptical- do you write numerical software professionally? I do, and have, and will do it in the future. There are very, very good reasons for all of those pieces you don't see the need for. There's a reason unums have not caught on with the field of numerical software or numerical analysis - they simply don't allow writing robust, performant software, they solve no real issues, and add significant problems.

adgjlsfhk1 4y ago
>so it should NOT satisfy "fundamental relationships for numbers to have". If you have a list with a NaN in it, how should you make sort terminate (and where should the NaN end up)? I understand that in theory it is kind of arguable that NaN should be different, but breaking the total order is a really dumb decision. >you'd have to pre-check evedry operation to do serious numerical work, which basically cuts your performance in half. Can you give an example? Saturating overflow tends to do the same thing. >IEEE explicitly guarantees these hold This is kind of true. -0.0+0.0==0.0 and 0.0-0.0==0.0. IEEE does define -0.0==0.0 so IEEE does technically make this hold, but only by redefining == so that two different numbers are == > -0 is not a "limit from the negatives" Then what is it? it's not a real number, and Kahan's justification of them comes from branch cuts of analytic functions which is only makes sense in the context of limits https://homes.cs.washington.edu/~ztatlock/599z-17sp/papers/branch-cuts-kahan-87.pdf https://homes.cs.washington.edu/~ztatlock/599z-17sp/papers/b... > Integers have the same property Yeah and it sucks there too. In the fp case it makes it really annoying to do things like calculate divide an array by a float quickly and accurately. You would want to take the inverse of the divisor and multiply by that, but doing so isn't safe if the divisor is subnormal. Yes. My day job is in solving Differential Algebraic equations, but I also have written a bunch of Julia's Libm.
- SideQuark 4y ago
  >If you have a list with a NaN in it, how should you make sort terminate Do whatever you want. If you're sorting floats, sort them to the front. Every language I've ever used for developing numerical software has a trivial IsNaN equivalent. So that's not a complaint worthy of claiming NaNs are not useful. I've written lots of numerical software and not once has this been an issue for me. What value do you assign sqrt of a negative without some NaN type item? Or any of tons of other "not a number" results? >so IEEE does technically make this hold, but only by redefining == There's no "redefining ==" here. You are upset that bit patterns are different, but == is not for bit patterns. You are confusing == for floats with == for bit patterns, which are not and need not be the same thing. I've never seen a language that gets these confused. If you want float ==, simply use language ==. If you want bitwise ==, then you usually have to do (often not portable) fiddling to convert to a bit pattern. It's like claiming reference == and structure field == should be the same, but both have uses. So languages have all sorts of ways to use the concept of equality, and they are all useful. Confusing them does not make the ones you don't like invalid or not extremely useful for people that do understand and use them. >Yes. My day job is in solving Differential Algebraic equations, but I also have written a bunch of Julia's Libm. Good. Then you should understand why, as an example, C++ std lib has a massive amount of functions like fma, expm1, log1p, hypot, and many more. Sure you can simply write log(1+x) instead of using log1p, but log1p is vastly better in this case because properties of IEEE 754 allow more precision. instead of hypot(x,y) you could write sqrt(xx+yy), but hypot is much better. These functions exist since IEEE provides tools to analyze these and make much better versions than the naive way to write them. Unums, with varying precision, make this vastly harder (and losing precision over the domain, making it hard to analyze anything). So unums, with varying precision, violate fundamental properties for scientific computing, namely, they lose precision in really messy ways. You cannot start with P digits of precision and do even simple math and get an answer with P digits of precision. IEEE does allow this. For example, sqrt(x^2)=|x| in IEEE (for no under/overflow). This does not work in unums, since they lose precision. Square something and lose digits. Fundamental to lots of scientific computation is the requirement to maintain precision throughout a calculation. Unums fail this spectacularly, making it incredibly messy to do correct scientific work.

jcranmer 4y ago

The value of -0 as distinguished from +0 has a few uses. The most obvious one is preserving sign in the case of overflow. A less obvious use case is handling branch cuts. There are uses in a few more cases: I've heard it's occasionally useful in things like coordinate systems, since something like "0°5'3" W" can be stored as (-0.0, 5, 3) after explosion and still display correctly. It's definitely niche, but it does have its uses. Returning a distinct value that retains the fact that it overflowed is quite useful--if you get that result out of the computation, you know you overflowed the computation rather than silently getting a meaningless result. Note in particular that infinities end up being sticky values: once a value goes infinite, it tends to stay infinity, which isn't true for largest finite values. Distinguishing between various kinds of "invalid" values turns out to be moderately useful in practice--I've used infinities a couple of times in my own code. NaNs are useful in representing a different kind of error than overflowed computation. Now there is a lot of room to criticize IEEE 754 here: "x != x" was quite frankly a mistake (basically the primary reason for it was the creators wanted to make testing for NaN easier than calling isnan(x)...). sNaNs are of course an abomination that just makes things worse. Multiple NaN payloads were originally intended (in part) to let developers debug the sources of NaNs, but this requires support that never really materialized. However, NaN payloads did find new use in making NaN-boxing a useful technique, and dedicating an entire exponent to special values simplifies several numerical analysis lemmas.

adgjlsfhk1 4y ago
> handling branch cuts I agree this sounds great in theory, but I don't think it works very well in practice. i.e. what about 1/(x+1)? Also branch cuts matter most for complex arithmetic, and there +-0 doesn't help since you don't know the phase of the zero. Also, realistically, floating point has finite precision so there are very few non-toy examples where you can do an actual computation and reliably end up on the correct branch. I'd rather have all the real numbers represented before we start adding hyper-reals to the number system. > Returning a distinct value that retains the fact that it overflowed is quite useful Agreed, and I think that NaR in Posits does a good job of that while not taking a ridiculous number of values.
- 4y ago