4 ms·

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound lik

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson). It does not help either that Gustafson work only seems to attire the attention of semi-illiterate journalists who do not really understand what they are talking about.

coldtea 7y ago
>IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson What about this then, which somebody comments below: "On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time, even though the machine did things 80-bits at a time)." And below: "I spent a long time once debugging an issue arising from this. Code was basically: double x = y; [...] if (x>y) fail; Nothing in between the assignment and the check changed the value of x or y, yet the check triggered the fail condition. Turned out to be that one of the value stayed in the 80 bit x87 register and one was written to RAM as a 64 bit value, then loaded back into an x87 register for the check, resulting in the inequality. Running in a debugger wrote both values out of the registers before the check, making the problem unreproducible." This behavior might not be IEEE defined, but even as IEEE undefined (and implementation dependent), it still qualifies those IEEE compliant implementations as non-deterministic. Or how about: "parallel systems, most likely GPUs, where it is common to have slightly different results even on the same operation, on the same system. This is due to the fact that several pipelines will do calculations in parallel and depending on which pipeline ends first (which can depend on many factors like current temperature) the sums can happen in a different order, leading to different rounding results." due to fp associativity?
- deleted 7y ago
  [deleted]
- enriquto 7y ago
  > What about this then (...) This is exactly the kind of fringe shitfuckery that must be avoided at all costs if you want to promote posits seriously.
- deleted 7y ago
  [deleted]
- yorwba 7y ago
  Those are not problems a new number format can fix though. The issues with x87 precision are due to compilers freely converting between different representations without this being indicated in the program. Posits also can't achieve associativity in all conditions, you need to use "quire" accumulators with higher precision. However, if the decision of using quires were left to the compiler, you'd end up with the exact same problem of unclear precision as for x87. To avoid that, those accumulators will need to be annotated in the code, and straightforward programs that do not use them will continue to exhibit non-associativity.
  - Lerc 7y ago
    The observation that all of the +-*/ operations in a basic block can be reduced to a single round with the quire would get you a reasonable way with the compiler alone. Overall I'm not sold on the quire thing. It seems like a great big accumulator is a solution that is fairly independent from the floating point format.
    - vanderZwan 7y ago
      > It seems like a great big accumulator is a solution that is fairly independent from the floating point format. It could still be a good idea to mandate a standard version in the spec for consistent behavior.
- namibj 7y ago
fluffything 7y ago
> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson How can that be? The IEEE standard does not place many requirements on the precision of the transcendental functions, and AFAICT a transcendental doesn't even need to return the exact same answer twice. That is, it is correct for a compiler to, if it can, evaluate a transcendental at compile-time with infinite precision, and if it cannot, call a run-time function that computes it. Math libraries can be dynamically linked, and all can produce different results here, so it is not that a compiler can go out of its way to make things deterministic either.
deleted 7y ago
[deleted]
piadodjanho 7y ago
I am implementing a high performance posit arithmetic unit for uni thesis. The idea of using Golomb-Rice prefix tree to encode the exponent is genius. Having said that. I totally agree his papers make a lot of bogus claim. For instance, the claim that "posits can be a direct replacement of floating point" is not true. There are a few tricks, such the Fast Invertion Square Root that abuses the floating point format to do fast arithmetic operation. This would break replacing with posit. Also he claims that early FPGA posit implementation uses less area and is has high performance than a Floating Point implementation. This cannot be true^. The posit unit IS A floating point unit with a encoder/decoder unit. The posit floating point unit will actually have a larger word size when decoded than ieee floating point because it has more precision than ieee format. Also the features such different rounding supported by ieee but not by posit uses a negligible extra area. ^ Unless, ofc, he is comparing apples to oranges.
- enriquto 7y ago
  I'm eager to read your thesis, then! Please, make sure to include a (short) historical account from a neutral point of view. Most notably, Kahan's rejection of unums/posits mixes mostly reasonable criticism with a few points that are not relevant; and then Gustafson holds on to these irrelevant points and replies with his usual word salad. It is quite a sad state of affairs for what could be an enlightening scientific discussion. The Golomb-Rice prefix tree for the exponents is certainly cool. It has the air of Gosper's "continued logarithms", which are like the same idea but turned to the max: use always the same mantissa (which is implicit) and represent only the exponent to a high precision.
  - xenadu02 7y ago
    Papers have already been written. This one implemented posits on an FPGA: https://hal.inria.fr/hal-02131982 https://hal.inria.fr/hal-02131982 The summary: These architectures are evaluated on recent FPGA hardware and compared to their IEEE-754 counterpart. The standard 32 bits posit adder is found to be twice as large as the corresponding floating-point adder. Posit multiplication requires about 7 times more LUTs and a few more DSPs for a latency which is 2x worst than the IEEE-754 32 bit multiplier.
kazinator 7y ago
> posits are floating point numbers, thus they cannot replace them Everyone who reads that who is cognitively typical will understand "floating-point" (which, by the way, needs a dash, if we are going to nitpick) as a nickname for "IEEE 754 floating-point" or "traditional floating-point", rather than an insinuation that posits are, in contrast, fixed-point or something other than floating-point.

coldtea 7y ago

>IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson What about this then, which somebody comments below: "On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time, even though the machine did things 80-bits at a time)." And below: "I spent a long time once debugging an issue arising from this. Code was basically: double x = y; [...] if (x>y) fail; Nothing in between the assignment and the check changed the value of x or y, yet the check triggered the fail condition. Turned out to be that one of the value stayed in the 80 bit x87 register and one was written to RAM as a 64 bit value, then loaded back into an x87 register for the check, resulting in the inequality. Running in a debugger wrote both values out of the registers before the check, making the problem unreproducible." This behavior might not be IEEE defined, but even as IEEE undefined (and implementation dependent), it still qualifies those IEEE compliant implementations as non-deterministic. Or how about: "parallel systems, most likely GPUs, where it is common to have slightly different results even on the same operation, on the same system. This is due to the fact that several pipelines will do calculations in parallel and depending on which pipeline ends first (which can depend on many factors like current temperature) the sums can happen in a different order, leading to different rounding results." due to fp associativity?

deleted 7y ago
[deleted]
enriquto 7y ago
> What about this then (...) This is exactly the kind of fringe shitfuckery that must be avoided at all costs if you want to promote posits seriously.
deleted 7y ago
[deleted]
yorwba 7y ago
Those are not problems a new number format can fix though. The issues with x87 precision are due to compilers freely converting between different representations without this being indicated in the program. Posits also can't achieve associativity in all conditions, you need to use "quire" accumulators with higher precision. However, if the decision of using quires were left to the compiler, you'd end up with the exact same problem of unclear precision as for x87. To avoid that, those accumulators will need to be annotated in the code, and straightforward programs that do not use them will continue to exhibit non-associativity.
- Lerc 7y ago
  The observation that all of the +-*/ operations in a basic block can be reduced to a single round with the quire would get you a reasonable way with the compiler alone. Overall I'm not sold on the quire thing. It seems like a great big accumulator is a solution that is fairly independent from the floating point format.
  - vanderZwan 7y ago
    > It seems like a great big accumulator is a solution that is fairly independent from the floating point format. It could still be a good idea to mandate a standard version in the spec for consistent behavior.
namibj 7y ago

fluffything 7y ago

> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson How can that be? The IEEE standard does not place many requirements on the precision of the transcendental functions, and AFAICT a transcendental doesn't even need to return the exact same answer twice. That is, it is correct for a compiler to, if it can, evaluate a transcendental at compile-time with infinite precision, and if it cannot, call a run-time function that computes it. Math libraries can be dynamically linked, and all can produce different results here, so it is not that a compiler can go out of its way to make things deterministic either.

deleted 7y ago

[deleted]

piadodjanho 7y ago

I am implementing a high performance posit arithmetic unit for uni thesis. The idea of using Golomb-Rice prefix tree to encode the exponent is genius. Having said that. I totally agree his papers make a lot of bogus claim. For instance, the claim that "posits can be a direct replacement of floating point" is not true. There are a few tricks, such the Fast Invertion Square Root that abuses the floating point format to do fast arithmetic operation. This would break replacing with posit. Also he claims that early FPGA posit implementation uses less area and is has high performance than a Floating Point implementation. This cannot be true^. The posit unit IS A floating point unit with a encoder/decoder unit. The posit floating point unit will actually have a larger word size when decoded than ieee floating point because it has more precision than ieee format. Also the features such different rounding supported by ieee but not by posit uses a negligible extra area. ^ Unless, ofc, he is comparing apples to oranges.

enriquto 7y ago
I'm eager to read your thesis, then! Please, make sure to include a (short) historical account from a neutral point of view. Most notably, Kahan's rejection of unums/posits mixes mostly reasonable criticism with a few points that are not relevant; and then Gustafson holds on to these irrelevant points and replies with his usual word salad. It is quite a sad state of affairs for what could be an enlightening scientific discussion. The Golomb-Rice prefix tree for the exponents is certainly cool. It has the air of Gosper's "continued logarithms", which are like the same idea but turned to the max: use always the same mantissa (which is implicit) and represent only the exponent to a high precision.
- xenadu02 7y ago
  Papers have already been written. This one implemented posits on an FPGA: https://hal.inria.fr/hal-02131982 https://hal.inria.fr/hal-02131982 The summary: These architectures are evaluated on recent FPGA hardware and compared to their IEEE-754 counterpart. The standard 32 bits posit adder is found to be twice as large as the corresponding floating-point adder. Posit multiplication requires about 7 times more LUTs and a few more DSPs for a latency which is 2x worst than the IEEE-754 32 bit multiplier.

kazinator 7y ago

> posits are floating point numbers, thus they cannot replace them Everyone who reads that who is cognitively typical will understand "floating-point" (which, by the way, needs a dash, if we are going to nitpick) as a nickname for "IEEE 754 floating-point" or "traditional floating-point", rather than an insinuation that posits are, in contrast, fixed-point or something other than floating-point.