4 ms·

>If you have a list with a NaN in it, how should you make sort terminate Do whatever you want. If you're sorting floats, sort them to the front. Every language

>If you have a list with a NaN in it, how should you make sort terminate Do whatever you want. If you're sorting floats, sort them to the front. Every language I've ever used for developing numerical software has a trivial IsNaN equivalent. So that's not a complaint worthy of claiming NaNs are not useful. I've written lots of numerical software and not once has this been an issue for me. What value do you assign sqrt of a negative without some NaN type item? Or any of tons of other "not a number" results? >so IEEE does technically make this hold, but only by redefining == There's no "redefining ==" here. You are upset that bit patterns are different, but == is not for bit patterns. You are confusing == for floats with == for bit patterns, which are not and need not be the same thing. I've never seen a language that gets these confused. If you want float ==, simply use language ==. If you want bitwise ==, then you usually have to do (often not portable) fiddling to convert to a bit pattern. It's like claiming reference == and structure field == should be the same, but both have uses. So languages have all sorts of ways to use the concept of equality, and they are all useful. Confusing them does not make the ones you don't like invalid or not extremely useful for people that do understand and use them. >Yes. My day job is in solving Differential Algebraic equations, but I also have written a bunch of Julia's Libm. Good. Then you should understand why, as an example, C++ std lib has a massive amount of functions like fma, expm1, log1p, hypot, and many more. Sure you can simply write log(1+x) instead of using log1p, but log1p is vastly better in this case because properties of IEEE 754 allow more precision. instead of hypot(x,y) you could write sqrt(xx+yy), but hypot is much better. These functions exist since IEEE provides tools to analyze these and make much better versions than the naive way to write them. Unums, with varying precision, make this vastly harder (and losing precision over the domain, making it hard to analyze anything). So unums, with varying precision, violate fundamental properties for scientific computing, namely, they lose precision in really messy ways. You cannot start with P digits of precision and do even simple math and get an answer with P digits of precision. IEEE does allow this. For example, sqrt(x^2)=|x| in IEEE (for no under/overflow). This does not work in unums, since they lose precision. Square something and lose digits. Fundamental to lots of scientific computation is the requirement to maintain precision throughout a calculation. Unums fail this spectacularly, making it incredibly messy to do correct scientific work.

adgjlsfhk1 4y ago
the posit standard has a NaR value that does everything I wish NaN and Inf were in ieee it is the result of 1/0, and sqrt(-1) etc. there is only one of them and it compares equal with itself and is defined as less than all other posits. Real numbers have a total order so it's silly that floating point doesn't. Furthermore the Posit ordering operations (bitwise) are the same as the signed integer ones which makes your processor simpler and makes it easier to do things like write radix sorts for floats. > You are confusing == for floats with == for bit patterns The problem is that == for floats doesn't behave like an equality operation. x==x doesn't hold (reflexivity) and x==y => f(x)==f(y) doesn't hold. These are The two most important parts of what equality means. To take your example of sqrt(xx), for Float16, of the 65k values, 34k give exact answers (counting NaNs as exact otherwise subtract 2k), 16k overflow and 5k underflow. There are also 9k inexact answers of which 6k are within 2 ULPs, and the others are further off (since xx loses precision due to subnormals). so in other words you get exact answers 1/2 of the time and close answers 60% of the time. With Posit16, you get 47k exact answers, and 18k inexact answers. How inexact are these inexact answers? 15k are within 2 ULP and only 2.9k aren't. (Of the 2.9k that aren't, Float16 would have overflowed or underflowed in all but 278 of the cases and these 278 cases are all accurate to less than 4 ULPs). Posits do lose the ability to do error free transforms, but IMO for 32 bit and smaller math, this isn't a major loss as if you want more accuracy you can use more bits and it will usually be faster than the error free transform.
- adgjlsfhk1 4y ago
  I've done a similar experiment with log1p(expm1(x)) and for that FLoat16 has 35k exact, 26k overflow, 1.3k within 4 ULP and 3k with more than 4 ULP error. Posits for comparison are 38k exact, 19k within 4 ULP, and 8k more than 4 ULP.
- SideQuark 4y ago
  >To take your example of sqrt(xx), Yes, for small floats posits do ok, but they fail for other sizes. For example, here's float32 vs posit32 for 100,000 random values in ranges 1e2,1e4,..,1e18. Posit32 fails on (respectively) 21%, 71%, 91%, 97%, 99%, 99.9%, 99.97%, 99.99% of the cases. Float32 fails on 0 of them. Julia code at the bottom. Posit even fails on simple integer multiplication so often that you'd be terribly pressed to know ahead of time when it happens. For example, take integers 1 to 40 for i and j, multiple as posit16 and as float16, an see how they do. Posit fails 1.75% of the values, float fails none. This is simple multiplication of numbers well within range. The same problem happens in posit32,64,anysize, but not for the same sized floats. >These are The two most important parts of what equality means. As a PhD in math, this is not what equality means. You'll find nothing like that here for example https://en.wikipedia.org/wiki/Equality_(mathematics) https://en.wikipedia.org/wiki/Equality_(mathematics) And if you're worried about equality, you might notice that in posit16, 2739 gives 1052 instead of 1053, which is real (in)equality. You worry so much about made up concerns that you miss the crazy bad results scattered throughout posits. Posits of all sizes make errors when multiplying by powers of two that floats do not make (die to their inability to keep digits). For example, in posit8, 2.01.03125 returns 2.0, 102 returns 16, and examples this bad can be found for any size posit. To see this, take 1e6 random values in 0-100, mult by 2, then divide by 2, and see how many made it round trip. All float16 values do. 4% of posit16 values do not round trip. These are small numbers - the entire computation stays in the range 0-200, and this is even the base of the underlying number. Posit32 has the same failure rate for the same reason: posits lose precision even under small multiplications. As a result, posits fail at x-y=0 means x=y, which is also pretty fundamental, is it not? Want to compute a discriminant sqrt(bb-4ac)? Good luck, nearby values for a,b,c don't give smooth results, and routinely give imaginary numbers when they should be real (due to the above screwiness around powers of 2). There's so many failure cases, not even at the edge of the ranges, where posits fail and equivalent sized floats don't, that doing any simple computations is error prone. Here's the Julia code for the sqrt failures. You can do similar error checks for a ton of computations and you'll find posits failing a significant amount of them. # count failures of float32 and Posit32 in Julia # for sqrt(x*x) ==?= x using Random Random.seed!(1234) # make reproducible scale = 1.0f0 # try exponent 4,6,8,10,etc for s in 1:9 # powers 2,4,5,8,10,...18 scale *= 100.0f0 badF,goodF = 0,0 badP,goodP = 0,0 for i in 1:10000 f = rand()*scale f1::Float32 = f f2 = f1*f1 f3 = sqrt(f2) @assert typeof(f1) == Float32 @assert typeof(f2) == Float32 @assert typeof(f3) == Float32 p1 = Posit32(f) p2 = p1*p1 p3 = sqrt(p2) @assert typeof(p1) == Posit32 @assert typeof(p2) == Posit32 @assert typeof(p3) == Posit32 if f1 != f3 badF+=1 else goodF+=1 end if p1 != p3 badP+=1 else goodP+=1 end end println("Scaling: $(scale)") println("float: $(goodF) good, $(badF) bad, $(100*badF/(goodF+badF)) % failed") println("posit: $(goodP) good, $(badP) bad, $(100*badP/(goodP+badP)) % failed")

adgjlsfhk1 4y ago

the posit standard has a NaR value that does everything I wish NaN and Inf were in ieee it is the result of 1/0, and sqrt(-1) etc. there is only one of them and it compares equal with itself and is defined as less than all other posits. Real numbers have a total order so it's silly that floating point doesn't. Furthermore the Posit ordering operations (bitwise) are the same as the signed integer ones which makes your processor simpler and makes it easier to do things like write radix sorts for floats. > You are confusing == for floats with == for bit patterns The problem is that == for floats doesn't behave like an equality operation. x==x doesn't hold (reflexivity) and x==y => f(x)==f(y) doesn't hold. These are The two most important parts of what equality means. To take your example of sqrt(xx), for Float16, of the 65k values, 34k give exact answers (counting NaNs as exact otherwise subtract 2k), 16k overflow and 5k underflow. There are also 9k inexact answers of which 6k are within 2 ULPs, and the others are further off (since xx loses precision due to subnormals). so in other words you get exact answers 1/2 of the time and close answers 60% of the time. With Posit16, you get 47k exact answers, and 18k inexact answers. How inexact are these inexact answers? 15k are within 2 ULP and only 2.9k aren't. (Of the 2.9k that aren't, Float16 would have overflowed or underflowed in all but 278 of the cases and these 278 cases are all accurate to less than 4 ULPs). Posits do lose the ability to do error free transforms, but IMO for 32 bit and smaller math, this isn't a major loss as if you want more accuracy you can use more bits and it will usually be faster than the error free transform.

adgjlsfhk1 4y ago
I've done a similar experiment with log1p(expm1(x)) and for that FLoat16 has 35k exact, 26k overflow, 1.3k within 4 ULP and 3k with more than 4 ULP error. Posits for comparison are 38k exact, 19k within 4 ULP, and 8k more than 4 ULP.
SideQuark 4y ago
>To take your example of sqrt(xx), Yes, for small floats posits do ok, but they fail for other sizes. For example, here's float32 vs posit32 for 100,000 random values in ranges 1e2,1e4,..,1e18. Posit32 fails on (respectively) 21%, 71%, 91%, 97%, 99%, 99.9%, 99.97%, 99.99% of the cases. Float32 fails on 0 of them. Julia code at the bottom. Posit even fails on simple integer multiplication so often that you'd be terribly pressed to know ahead of time when it happens. For example, take integers 1 to 40 for i and j, multiple as posit16 and as float16, an see how they do. Posit fails 1.75% of the values, float fails none. This is simple multiplication of numbers well within range. The same problem happens in posit32,64,anysize, but not for the same sized floats. >These are The two most important parts of what equality means. As a PhD in math, this is not what equality means. You'll find nothing like that here for example https://en.wikipedia.org/wiki/Equality_(mathematics) https://en.wikipedia.org/wiki/Equality_(mathematics) And if you're worried about equality, you might notice that in posit16, 2739 gives 1052 instead of 1053, which is real (in)equality. You worry so much about made up concerns that you miss the crazy bad results scattered throughout posits. Posits of all sizes make errors when multiplying by powers of two that floats do not make (die to their inability to keep digits). For example, in posit8, 2.01.03125 returns 2.0, 102 returns 16, and examples this bad can be found for any size posit. To see this, take 1e6 random values in 0-100, mult by 2, then divide by 2, and see how many made it round trip. All float16 values do. 4% of posit16 values do not round trip. These are small numbers - the entire computation stays in the range 0-200, and this is even the base of the underlying number. Posit32 has the same failure rate for the same reason: posits lose precision even under small multiplications. As a result, posits fail at x-y=0 means x=y, which is also pretty fundamental, is it not? Want to compute a discriminant sqrt(bb-4ac)? Good luck, nearby values for a,b,c don't give smooth results, and routinely give imaginary numbers when they should be real (due to the above screwiness around powers of 2). There's so many failure cases, not even at the edge of the ranges, where posits fail and equivalent sized floats don't, that doing any simple computations is error prone. Here's the Julia code for the sqrt failures. You can do similar error checks for a ton of computations and you'll find posits failing a significant amount of them. # count failures of float32 and Posit32 in Julia # for sqrt(x*x) ==?= x using Random Random.seed!(1234) # make reproducible scale = 1.0f0 # try exponent 4,6,8,10,etc for s in 1:9 # powers 2,4,5,8,10,...18 scale *= 100.0f0 badF,goodF = 0,0 badP,goodP = 0,0 for i in 1:10000 f = rand()*scale f1::Float32 = f f2 = f1*f1 f3 = sqrt(f2) @assert typeof(f1) == Float32 @assert typeof(f2) == Float32 @assert typeof(f3) == Float32 p1 = Posit32(f) p2 = p1*p1 p3 = sqrt(p2) @assert typeof(p1) == Posit32 @assert typeof(p2) == Posit32 @assert typeof(p3) == Posit32 if f1 != f3 badF+=1 else goodF+=1 end if p1 != p3 badP+=1 else goodP+=1 end end println("Scaling: $(scale)") println("float: $(goodF) good, $(badF) bad, $(100*badF/(goodF+badF)) % failed") println("posit: $(goodP) good, $(badP) bad, $(100*badP/(goodP+badP)) % failed")