6 ms·

>> Having investigated posits by running RTL implementations through synthesis on 7 and 28 nm nodes... Is it possible to implement 32 and 64 bit posits in simi

>> Having investigated posits by running RTL implementations through synthesis on 7 and 28 nm nodes... Is it possible to implement 32 and 64 bit posits in similar area to floating point? Can the calculations be had in the same number of cycles or fewer? I feel like these are the two most important questions. If the answers are yes, then I think it may be worth doing implementations for the increased precision and simplicity (lack on NaN and other IEEE quirks).

Someone 5y ago
I don’t understand why not having NaNs would increase simplicity. Isn’t that just moving the problem of detecting various forms of over/underflow from the CPU to the programmer using it?
- adgjlsfhk1 5y ago
  It's not quite right to say that Posits don't have NaN. They have a NaR (not a real) value that's fairly similar. The simplification is that they don't have 2^52 of them, and don't have -0, Inf, -Inf, or the mess with subnormals (which are necessary but not always well supported). Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point.
  - DaiPlusPlus 5y ago
    > Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point. I thought that was by-design? Just like how NULL != NULL in SQL?
    - adgjlsfhk1 5y ago
      As far as I can tell (see https://stackoverflow.com/questions/1565164/what-is-the-rationale-for-all-comparisons-returning-false-for-ieee754-nan-values https://stackoverflow.com/questions/1565164/what-is-the-rati...), the only reason for it was that early chips had no other single instruction that could be used to implement isnan, so they decided to use == for that purpose.
      - adrian_b 5y ago
        No, a NaN not being equal to itself is the correct behavior for an order relation that is a partial order, not a total order. The bug is neither in the FP standard nor in the implementations. The bug is in almost all programming languages, which annoyingly provide only the 6 relational operators needed for total order relations. To deal with partial order relations, which appear not only in operations with floating-point numbers, but also in many other applications, you need 14 relational operators. Only when they are provided by the programming language you no longer need to test whether a number is a NaN, because you can always use the appropriate relational operator, depending on what the test must really do. The 2 relational operators "is equal to" and "is either equal to or unordered", are distinct operators, exactly like "is equal to" and "is equal to or greater" are distinct. If you do not have distinct symbols for"is equal to" and "is either equal to or unordered", the compiler cannot know whether the test must be true or false when a NaN operand is encountered. If you do not know which of the 2 operators must be used, you must think more about it, because your program will be wrong in exceptional cases. Specifying a stupid rule like a NaN being equal to itself and claiming that the correct behavior is a bug shows a dangerous lack of understanding of mathematics, which is not acceptable for someone designing a new number representation method (even if a single NaN value is used, it can be generated as a result of different, completely unrelated computations, and there is no meaningful way in which those results can be considered equal). In general almost all programming languages are designed by people without experience in numerical computation, who completely neglect to include in the language the features required for the operations with floating-point numbers, resulting in an usually dreadful support for the IEEE floating-point arithmetic standard, even after almost 40 years since it became widespread. It is not the standard which must be corrected, but the programming languages, which do not provide good access to what the hardware can do. Also the education of the programmers is lacking because most of them are familiar only with the 6 relational operators for total orders, instead of being equally familiar with the 14 operators for partial orders. Partial orders are encountered extremely frequently. If only the 6 relational operators are available in such cases, then they must always be preceded by a test for unordered operands. Forgetting the additional test guarantees errors. To avoid the double tests that are needed in most programming languages, it may be possible to define macros corresponding to all the 14 relational operators, but they cannot be as convenient as proper support in the language.
  - zokier 5y ago
    That's a recent change, right? They switched inf to more generic nar?
    - adgjlsfhk1 5y ago
      I think it's been there from the beginning, but I'm not 100% sure. Realistically, Inf and -Inf are pretty much never useful in the real world, and having them makes -0.0 almost seem reasonable (since branch cuts almost always result in Inf/-Inf).
      - adrian_b 5y ago
        In the initial proposal for the IEEE floating-point standard, whether to have 1 or 2 values for infinities was configurable, like the rounding mode. However before the standard was adopted it was decided to reduce the number of configurable modes and the single-value infinity was dropped.
      - FullyFunctional 5y ago
        No, originally there were -/+ inf. NaR replaced this at some point, but semantically not much changed.
      - zokier 5y ago
        > I think it's been there from the beginning, but I'm not 100% sure The 2017 Gustafson paper linked in the article has only inf, no nar https://posithub.org/docs/Posits4.pdf https://posithub.org/docs/Posits4.pdf
        FullyFunctional 5y ago
        It's way out of date
        raymondh 5y ago
        What is the recommended latest equivalent of that document?
  - nwallin 5y ago
    > Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point. Wait, so with posits, sqrt(-7) == sqrt(-4)? And 1e42^42 == 0.0/0.0? And this is supposed to be a good thing? The semantics around NaN, +/- 0/Inf are really important. I feel like most of the proponents of posits don't understand why it's so vitally important that these constructs operate the way that they do. Maybe there are some useful things to be learned from posits, but it's difficult to take posits seriously when two values can compare equal when they are quite obviously not.
    - adgjlsfhk1 5y ago
      In floating point 2^54 == 1+2^54 and 1/0 == 2/0 == -ln(0). Every floating point value is an ambiguous result of infinitely many calculations with real numbers. When you do an invalid calculation, the most important thing to know is that you did something invalid.
- FullyFunctional 5y ago
  Have you read the NaN rules? Know when NaN is converted to the various forms? It's maddening, truly. A posit implementation is definitely simpler from the point of view of dealing with that, but outside the NaN and Inf numbers, it's no easier (and technically a wee bit more work to deal with the regiments).
  - throwaway_posit 5y ago
    I will say one thing: I pity the person (grad student?) that has to do error propagation analysis on a research project using posits (I'm the original implementor)
    - adgjlsfhk1 5y ago
      Yeah, I pretty much think that posit only makes sense for 32 bit and smaller, and that you want your 64 bit numbers to be closer to float64 (although with the posit semantics for Inf/NaN/-0).
      - throwaway_posit 5y ago
        Oh I actually only think it's useful for machine learning. I have some unpublished, crudely done research showing that the extended accumulator is only necessary for the Kronecker delta stage of the back propagation (posits trivially convert to higher precision by zero-padding).. you can see what I'm talking about I'm the Stanford video. Fun fact: John sometimes claims he invented the name, but this is untrue; my old college website talks about building "positronic brains" and it's long been a goal of mine to somehow "retcon" Asimov/tng data, into being a real thing, and when this opportunity for some clever wordsmithing arrived I coined the term with the hopes that someone would make a posit-based perceptron, or "positron".

Someone 5y ago

I don’t understand why not having NaNs would increase simplicity. Isn’t that just moving the problem of detecting various forms of over/underflow from the CPU to the programmer using it?

adgjlsfhk1 5y ago
It's not quite right to say that Posits don't have NaN. They have a NaR (not a real) value that's fairly similar. The simplification is that they don't have 2^52 of them, and don't have -0, Inf, -Inf, or the mess with subnormals (which are necessary but not always well supported). Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point.
- DaiPlusPlus 5y ago
  > Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point. I thought that was by-design? Just like how NULL != NULL in SQL?
  - adgjlsfhk1 5y ago
    As far as I can tell (see https://stackoverflow.com/questions/1565164/what-is-the-rationale-for-all-comparisons-returning-false-for-ieee754-nan-values https://stackoverflow.com/questions/1565164/what-is-the-rati...), the only reason for it was that early chips had no other single instruction that could be used to implement isnan, so they decided to use == for that purpose.
    - adrian_b 5y ago
      No, a NaN not being equal to itself is the correct behavior for an order relation that is a partial order, not a total order. The bug is neither in the FP standard nor in the implementations. The bug is in almost all programming languages, which annoyingly provide only the 6 relational operators needed for total order relations. To deal with partial order relations, which appear not only in operations with floating-point numbers, but also in many other applications, you need 14 relational operators. Only when they are provided by the programming language you no longer need to test whether a number is a NaN, because you can always use the appropriate relational operator, depending on what the test must really do. The 2 relational operators "is equal to" and "is either equal to or unordered", are distinct operators, exactly like "is equal to" and "is equal to or greater" are distinct. If you do not have distinct symbols for"is equal to" and "is either equal to or unordered", the compiler cannot know whether the test must be true or false when a NaN operand is encountered. If you do not know which of the 2 operators must be used, you must think more about it, because your program will be wrong in exceptional cases. Specifying a stupid rule like a NaN being equal to itself and claiming that the correct behavior is a bug shows a dangerous lack of understanding of mathematics, which is not acceptable for someone designing a new number representation method (even if a single NaN value is used, it can be generated as a result of different, completely unrelated computations, and there is no meaningful way in which those results can be considered equal). In general almost all programming languages are designed by people without experience in numerical computation, who completely neglect to include in the language the features required for the operations with floating-point numbers, resulting in an usually dreadful support for the IEEE floating-point arithmetic standard, even after almost 40 years since it became widespread. It is not the standard which must be corrected, but the programming languages, which do not provide good access to what the hardware can do. Also the education of the programmers is lacking because most of them are familiar only with the 6 relational operators for total orders, instead of being equally familiar with the 14 operators for partial orders. Partial orders are encountered extremely frequently. If only the 6 relational operators are available in such cases, then they must always be preceded by a test for unordered operands. Forgetting the additional test guarantees errors. To avoid the double tests that are needed in most programming languages, it may be possible to define macros corresponding to all the 14 relational operators, but they cannot be as convenient as proper support in the language.
- zokier 5y ago
  That's a recent change, right? They switched inf to more generic nar?
  - adgjlsfhk1 5y ago
    I think it's been there from the beginning, but I'm not 100% sure. Realistically, Inf and -Inf are pretty much never useful in the real world, and having them makes -0.0 almost seem reasonable (since branch cuts almost always result in Inf/-Inf).
    - adrian_b 5y ago
      In the initial proposal for the IEEE floating-point standard, whether to have 1 or 2 values for infinities was configurable, like the rounding mode. However before the standard was adopted it was decided to reduce the number of configurable modes and the single-value infinity was dropped.
    - FullyFunctional 5y ago
      No, originally there were -/+ inf. NaR replaced this at some point, but semantically not much changed.
    - zokier 5y ago
      > I think it's been there from the beginning, but I'm not 100% sure The 2017 Gustafson paper linked in the article has only inf, no nar https://posithub.org/docs/Posits4.pdf https://posithub.org/docs/Posits4.pdf
      - FullyFunctional 5y ago
        It's way out of date
        raymondh 5y ago
        What is the recommended latest equivalent of that document?
- nwallin 5y ago
  > Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point. Wait, so with posits, sqrt(-7) == sqrt(-4)? And 1e42^42 == 0.0/0.0? And this is supposed to be a good thing? The semantics around NaN, +/- 0/Inf are really important. I feel like most of the proponents of posits don't understand why it's so vitally important that these constructs operate the way that they do. Maybe there are some useful things to be learned from posits, but it's difficult to take posits seriously when two values can compare equal when they are quite obviously not.
  - adgjlsfhk1 5y ago
    In floating point 2^54 == 1+2^54 and 1/0 == 2/0 == -ln(0). Every floating point value is an ambiguous result of infinitely many calculations with real numbers. When you do an invalid calculation, the most important thing to know is that you did something invalid.
FullyFunctional 5y ago
Have you read the NaN rules? Know when NaN is converted to the various forms? It's maddening, truly. A posit implementation is definitely simpler from the point of view of dealing with that, but outside the NaN and Inf numbers, it's no easier (and technically a wee bit more work to deal with the regiments).
- throwaway_posit 5y ago
  I will say one thing: I pity the person (grad student?) that has to do error propagation analysis on a research project using posits (I'm the original implementor)
  - adgjlsfhk1 5y ago
    Yeah, I pretty much think that posit only makes sense for 32 bit and smaller, and that you want your 64 bit numbers to be closer to float64 (although with the posit semantics for Inf/NaN/-0).
    - throwaway_posit 5y ago
      Oh I actually only think it's useful for machine learning. I have some unpublished, crudely done research showing that the extended accumulator is only necessary for the Kronecker delta stage of the back propagation (posits trivially convert to higher precision by zero-padding).. you can see what I'm talking about I'm the Stanford video. Fun fact: John sometimes claims he invented the name, but this is untrue; my old college website talks about building "positronic brains" and it's long been a goal of mine to somehow "retcon" Asimov/tng data, into being a real thing, and when this opportunity for some clever wordsmithing arrived I coined the term with the hopes that someone would make a posit-based perceptron, or "positron".