7 ms·

When does this need arise? Well, otherwise inverting a value can change it's sign and in particular inverting -∞ twice will give you +∞, and being off by "2∞"

by patrec 6y ago

When does this need arise? Well, otherwise inverting a value can change it's sign and in particular inverting -∞ twice will give you +∞, and being off by "2∞" is a pretty large error for a lot of computations ;) You can end up with zeros and infinities pretty easily because you overflow or underflow the range of floating point precision, and generally you want something sensible to happen in typical cases, even if some common arithmetic identities necessarily break down. I would actually like a true signed zero (or rather "epsilon" value ), so -0 and +0 as distinct from "normal" 0 which is truly unsigned, neither positive nor negative. The former two would only arise from underflow, and the reason this is useful that if you underflow from below zero and invert that you want to get -∞ and if you underflow from above zero and invert that you want to get +∞. Inverting a signless zero should give NaN (instead it gives +∞, which is nonsense in basically any case where the domain is not inherently the non-negative reals already and the 0 did not come about by and underflow; in particular 1/0 should be NaN). If anyone knows why this design was not chosen and what fundamental downsides it has, I'd love to hear it. Obviously representing three zeros is a tad more annoying, but IEEE754 has a lot of stuff that's annoying implementation wise but was added for nicer numerical behavior (e.g. denormals, and of course various "global" rounding modes etc. which probably qualify as a mistake in retrospect).

amelius 6y ago
∞ is not a number. Using it as a number is a hack invented by mathematicians.
- steerablesafe 6y ago
  ∞ is not used as a number by mathematicians. Maybe by engineers.
  - chithanh 6y ago
    That is not entirely correct. Schmieden and Laugwitz for example developed in the 1950s a nonstandard Analysis which adjoins an infinitely large element (called Ω) to the natural numbers. The basic idea was a formula A(Ω) was true if A(n) was true for almost all finite natural n. While it wasn't immensely useful going forward, it helped to clarify the use of infinity and infinitesimals in earlier work.
    - steerablesafe 6y ago
      I'm well aware of nonstandard analysis, but ∞ is still not a number there, even though there are infinitely many infinitely large elements .
      - Spivak 6y ago
        The extended complex plane is a space where inf is actually number and where division by zero is allowed. Same with the extended real line. Infinity is as much a number as it is useful to define it as such.
        steerablesafe 6y ago
        Ah right, I forget that extending with a single infinity element is useful with complex numbers and with geometry. It's still not very common with the reals alone as +∞ and -∞ are reasonable to want as separate elements there, but it doesn't play nicely with 1/0 that way.
  - prionassembly 6y ago
    Ehh. https://en.wikipedia.org/wiki/Extended_real_number_line https://en.wikipedia.org/wiki/Extended_real_number_line NB this is hardly nonstandard analysis.
- analog31 6y ago
  That's right. It's a symbol. When you see it in an expression, you're probably expected to interpret it in light of a limit of some kind. It's just convenient to write it into an expression rather than use cumbersome limit notation all over the place. Like many notational shortcuts, it's a hack supported by proof. ;-)
  - colejohnson66 6y ago
    I was taught that you’re supposed to read $y = \inf$ as: y = lim_{x->\inf} x Basically, $y$ isn’t necessarily infinity, but just a number larger than you could ever write. The number at the end of the number line if it existed.
  - taeric 6y ago
    This explanation feels off, to me. Aren't all numbers symbols? Pi, e, 10, 0xA, 9.99999...?
    - analog31 6y ago
      In my view, numbers are numbers, and symbols are symbols. There's an agreement that a symbol represents a number, but there's not a one-to-one relationship between available symbols and available numbers. Normally this isn't a problem, and we treat them interchangeably. And indeed the distinction may only be a philosophical oddity or a matter for mathematicians. But I believe nonetheless that there is a distinction. Now I was merely an undergrad math major, which means I topped out before learning this stuff in a formal way. But at my primitive level of understanding, I think of a number as something that behaves like a number within a given system. What I learned in my courses was how different kinds of numbers behaved: Whole numbers, reals, complex, vectors and tensors, etc. I remember reading a definition of "tensor" that was to the effect of: A tensor is something that behaves like a tensor, meaning that the important thing is the behavior. Another post in this page expressed that we should be particularly cautious when dealing with numbers, symbols, and IEEE floats, notably to beware that IEEE floats and real numbers don't always behave the same. That was treated in one of my math courses, "Numerical Analysis." You could also get CS credit for that course, suggesting its practical importance.
phkahler 6y ago
Posits handle this by using the smallest non-zero number where floating point would go to zero. They also use the largest represent able number instead of infinity. These exist for both positive and negative numbers. At least that's the way I read it.
- dnautics 6y ago
  Co-inventor of posits here, this is basically correct, there still is "infinity" in posits, it's strictly reachable by inverting 0 directly.
  - patrec 6y ago
    Having a single (unsigned) infinity (and a single zero!) seems cleaner in some ways (and I dimly seem to recall that some pre-ieee754 floating point hardware worked that way). On the other hand, having e.g. a neutral element for both max and min also seems pretty nice to have, although without infinities, the maximal and minimal floating point value will equally do the trick in most cases. How do inequalities work for posit infinity? Is posit infinity both larger and smaller than any other posit?
  - phkahler 6y ago
    I thought that was essentially NaN and could also result from square root of negative numbers?
- afiori 6y ago
  a cons of that is that there is no longer a distinction between positive and negative infinity
Const-me 6y ago
> If anyone knows why this design was not chosen and what fundamental downsides it has, I'd love to hear it. No fundamental ones, but a few practical. 32-bit numbers have 2^32 unique values, the number is even. Your approach makes the range asymmetrical like it happens with integers. The range for 8-bit signed integers is [ -128 .. +127 ]. On ARM NEON there’re two versions of integer negate and absolute instructions, some (like vqnegq_s8 or vqabsq_s8) do saturation i.e. transform -128 into +127, others (vnegq_s8, vabsq_s8) don’t change -128. Neither of them is particularly good: the saturated version violates -(-x) == x, non-saturated version violates abs(x)>=0. Same applies to the rest of the signed integers (16, 32, 64 bits), an all modern platforms. With IEEE floats the range is symmetrical and none of that is needed. Moreover, PCs don’t have absolute or negate instructions, but they instead have bitwise instructions processing floats, like andps, orps, xorps, andnps, they allow to flip, clear or set just the sign bit, very fast. Another useful property of IEEE representation is that for two intervals [ 0 .. FLT_MAX ] and [-FLT_MAX .. -0.0f ] sort order of floats corresponds to [inverted] sort order of 32-bit integers.
- bluecalm 6y ago
  I wonder if it would be useful to have a signed integer that has symmetric range and the one that is left is used as NaN. Overflows would set it to NaN for example. Then again, once that's on the table it's very tempting to steal two more values for +/- inf. I think it's very useful to have full range unsigned ints but signed ones could have range reduced to make them less error prone.
- patrec 6y ago
  > Your approach makes the range asymmetrical like it happens with integers. Not necessarily since you've got (a lot of different) NaNs anyway. For the sake of argument, you could give up one one of them and make it positive zero (since this representation would be unnatural, it would slow stuff down, just as non-finite values do on many CPUs. Wouldn't matter that much since signed zeros would only arise from underflow). > Another useful property of IEEE representation is that for two intervals [ 0 .. FLT_MAX ] and [-FLT_MAX .. -0.0f ] sort order of floats corresponds to [inverted] sort order of 32-bit integers. I'm aware, but as you correctly note this only works in the right direction for unsigned values. And it's just not that important a benefit, I'd much rather have my calculations come out right than being able to sort positive floating point numbers with an integer sort routine.
jayd16 6y ago
So no then? As you say, epsilon should be used in this case.
- patrec 6y ago
  I meant as in infinitesimal, not as in machine epsilon (which lacks the required properties) -- poor wording on my part. If you have numbers of impossibly large magnitude you probably also want corresponding numbers of impossibly small magnitude. You can do this in a nice, algebraically satisfying way with infinitely many such numbers (hyperreals), but I think if it weren't for the signless zero and positive zero conflation, IEEE754's way would be quite a reasonable finite precision approximation to this.

by patrec 6y ago

amelius 6y ago

∞ is not a number. Using it as a number is a hack invented by mathematicians.

steerablesafe 6y ago
∞ is not used as a number by mathematicians. Maybe by engineers.
- chithanh 6y ago
  That is not entirely correct. Schmieden and Laugwitz for example developed in the 1950s a nonstandard Analysis which adjoins an infinitely large element (called Ω) to the natural numbers. The basic idea was a formula A(Ω) was true if A(n) was true for almost all finite natural n. While it wasn't immensely useful going forward, it helped to clarify the use of infinity and infinitesimals in earlier work.
  - steerablesafe 6y ago
    I'm well aware of nonstandard analysis, but ∞ is still not a number there, even though there are infinitely many infinitely large elements .
    - Spivak 6y ago
      The extended complex plane is a space where inf is actually number and where division by zero is allowed. Same with the extended real line. Infinity is as much a number as it is useful to define it as such.
      - steerablesafe 6y ago
        Ah right, I forget that extending with a single infinity element is useful with complex numbers and with geometry. It's still not very common with the reals alone as +∞ and -∞ are reasonable to want as separate elements there, but it doesn't play nicely with 1/0 that way.
- prionassembly 6y ago
  Ehh. https://en.wikipedia.org/wiki/Extended_real_number_line https://en.wikipedia.org/wiki/Extended_real_number_line NB this is hardly nonstandard analysis.
analog31 6y ago
That's right. It's a symbol. When you see it in an expression, you're probably expected to interpret it in light of a limit of some kind. It's just convenient to write it into an expression rather than use cumbersome limit notation all over the place. Like many notational shortcuts, it's a hack supported by proof. ;-)
- colejohnson66 6y ago
  I was taught that you’re supposed to read $y = \inf$ as: y = lim_{x->\inf} x Basically, $y$ isn’t necessarily infinity, but just a number larger than you could ever write. The number at the end of the number line if it existed.
- taeric 6y ago
  This explanation feels off, to me. Aren't all numbers symbols? Pi, e, 10, 0xA, 9.99999...?
  - analog31 6y ago
    In my view, numbers are numbers, and symbols are symbols. There's an agreement that a symbol represents a number, but there's not a one-to-one relationship between available symbols and available numbers. Normally this isn't a problem, and we treat them interchangeably. And indeed the distinction may only be a philosophical oddity or a matter for mathematicians. But I believe nonetheless that there is a distinction. Now I was merely an undergrad math major, which means I topped out before learning this stuff in a formal way. But at my primitive level of understanding, I think of a number as something that behaves like a number within a given system. What I learned in my courses was how different kinds of numbers behaved: Whole numbers, reals, complex, vectors and tensors, etc. I remember reading a definition of "tensor" that was to the effect of: A tensor is something that behaves like a tensor, meaning that the important thing is the behavior. Another post in this page expressed that we should be particularly cautious when dealing with numbers, symbols, and IEEE floats, notably to beware that IEEE floats and real numbers don't always behave the same. That was treated in one of my math courses, "Numerical Analysis." You could also get CS credit for that course, suggesting its practical importance.

phkahler 6y ago

Posits handle this by using the smallest non-zero number where floating point would go to zero. They also use the largest represent able number instead of infinity. These exist for both positive and negative numbers. At least that's the way I read it.

dnautics 6y ago
Co-inventor of posits here, this is basically correct, there still is "infinity" in posits, it's strictly reachable by inverting 0 directly.
- patrec 6y ago
  Having a single (unsigned) infinity (and a single zero!) seems cleaner in some ways (and I dimly seem to recall that some pre-ieee754 floating point hardware worked that way). On the other hand, having e.g. a neutral element for both max and min also seems pretty nice to have, although without infinities, the maximal and minimal floating point value will equally do the trick in most cases. How do inequalities work for posit infinity? Is posit infinity both larger and smaller than any other posit?
- phkahler 6y ago
  I thought that was essentially NaN and could also result from square root of negative numbers?
afiori 6y ago
a cons of that is that there is no longer a distinction between positive and negative infinity

Const-me 6y ago

> If anyone knows why this design was not chosen and what fundamental downsides it has, I'd love to hear it. No fundamental ones, but a few practical. 32-bit numbers have 2^32 unique values, the number is even. Your approach makes the range asymmetrical like it happens with integers. The range for 8-bit signed integers is [ -128 .. +127 ]. On ARM NEON there’re two versions of integer negate and absolute instructions, some (like vqnegq_s8 or vqabsq_s8) do saturation i.e. transform -128 into +127, others (vnegq_s8, vabsq_s8) don’t change -128. Neither of them is particularly good: the saturated version violates -(-x) == x, non-saturated version violates abs(x)>=0. Same applies to the rest of the signed integers (16, 32, 64 bits), an all modern platforms. With IEEE floats the range is symmetrical and none of that is needed. Moreover, PCs don’t have absolute or negate instructions, but they instead have bitwise instructions processing floats, like andps, orps, xorps, andnps, they allow to flip, clear or set just the sign bit, very fast. Another useful property of IEEE representation is that for two intervals [ 0 .. FLT_MAX ] and [-FLT_MAX .. -0.0f ] sort order of floats corresponds to [inverted] sort order of 32-bit integers.

bluecalm 6y ago
I wonder if it would be useful to have a signed integer that has symmetric range and the one that is left is used as NaN. Overflows would set it to NaN for example. Then again, once that's on the table it's very tempting to steal two more values for +/- inf. I think it's very useful to have full range unsigned ints but signed ones could have range reduced to make them less error prone.
patrec 6y ago
> Your approach makes the range asymmetrical like it happens with integers. Not necessarily since you've got (a lot of different) NaNs anyway. For the sake of argument, you could give up one one of them and make it positive zero (since this representation would be unnatural, it would slow stuff down, just as non-finite values do on many CPUs. Wouldn't matter that much since signed zeros would only arise from underflow). > Another useful property of IEEE representation is that for two intervals [ 0 .. FLT_MAX ] and [-FLT_MAX .. -0.0f ] sort order of floats corresponds to [inverted] sort order of 32-bit integers. I'm aware, but as you correctly note this only works in the right direction for unsigned values. And it's just not that important a benefit, I'd much rather have my calculations come out right than being able to sort positive floating point numbers with an integer sort routine.

jayd16 6y ago

So no then? As you say, epsilon should be used in this case.

patrec 6y ago
I meant as in infinitesimal, not as in machine epsilon (which lacks the required properties) -- poor wording on my part. If you have numbers of impossibly large magnitude you probably also want corresponding numbers of impossibly small magnitude. You can do this in a nice, algebraically satisfying way with infinitely many such numbers (hyperreals), but I think if it weren't for the signless zero and positive zero conflation, IEEE754's way would be quite a reasonable finite precision approximation to this.