19 ms·
The Cult of Posits
- whatshisface 5y agoHow do you compute with posits? I can imagine vaguely how an FPU would work, by applying the rules of algebra to m*2^e and cleaning up afterwards to keep things normalized, but I do not know what operations on the projective circle are homeomorphic to addition or multiplication on the real line.
- adgjlsfhk1 5y agoposits can be also projected to m*2^e, just with larger m and e.
- whatshisface 5y agoConverting to floats, computing and converting back would hardly result in a smaller FPU, unless I am missing something about it. Edit: a dead comment replies with, > I'm the implementor for posits (deactivated my primary hn account) -- I built circuit diagrams for computation with posits. So, we have them -- and they are smaller. There is also a key insight into the representation of posits (negative numbers have a "hidden -2" instead of a "hidden 1") that I cracked early on in my fiddling with circuits that was completely missed by all other implementers until earlier this year, even after I communicated it to them through email correspondence several times.
- throwaway_posit 5y agoI'm the implementor for posits (deactivated my primary hn account) -- I built circuit diagrams for computation with posits. So, we have them -- and they are smaller. There is also a key insight into the representation of posits (negative numbers have a "hidden -2" instead of a "hidden 1") that I cracked early on in my fiddling with circuits that was completely missed by all other implementers until earlier this year, even after I communicated it to them through email correspondence several times.
- FullyFunctional 5y agoI don't know how you can claim to be "the" implementor as there are many implementations. However, your explanation doesn't have enough context. Do you have a paper describing this in better detail?
- throwaway_posit 5y agoSorry, should have specified: I'm the original implementor. I'm on the paper with John Gustafson, and presenter/live-demoer of the second half the Stanford video. There is a paper coming out with details on the yonemoto -2 hidden bit method... I don't know if it's still preprint or embargoed or what but it is recent. I'm really not involved in the project anymore so my knowledge of the existence of this paper is only due to the courtesy of the authors.
- FullyFunctional 5y agoIs the source code available? It would be amazing to the best possible implementation available as Verilog. There are quite a few pretty good IEEE 754 implementations.
- adrian_b 5y agoI can believe that for not too large number sizes a posit implementation might be smaller than for the traditional FP format. However "smaller" must be qualified, because the size of a FPU varies enormously depending on the speed target. A completely different size results when the target is to do a fused multiply-add in 4 cycles at 5 GHz than when the target is to do it in 100 cycles at 200 MHz. So unless you give more information about what you have compared, we cannot know whether it is true that a posit implementation can be smaller.
- a1369209993 5y ago> negative numbers have a "hidden -2" instead of a "hidden 1" Isn't this just (what I assumed was) the standard implementation technique for FPUs that aren't[0] stuck with IBM/Intel's braindead sign-magnitude junk? 0: so (eg, for a 1.7.8 float) -1.996..-1.000 would be 3F00-3FFF, and +1.000..+1.996 would be C000-C0FF, making 0x[-2].00-0x[-2].FF and 0x[+1].00-0x[+1].FF (with implicit -2/+1 corresponding to 3F/C0)
- adgjlsfhk1 5y agoI don't mean converting to floats, I mean treating the exponent and mantissa separately. Processors already do something similar for floats. For addition, (for example), you de-normalize the smaller input by an amount to make the exponents line up, add and re-normalize. The steps would be pretty much exactly the same for posits, the only difference is how you calculate the difference in exponents (which just boils down to a few shifts based on the useed value).
- scythe 5y agoMore info here: http://johngustafson.net/pdfs/BeatingFloatingPoint.pdf http://johngustafson.net/pdfs/BeatingFloatingPoint.pdf
- jhj 5y agoHaving investigated posits by running RTL implementations through synthesis on 7 and 28 nm nodes, I don't buy the claim that a posit FPU is smaller than a IEEE float FPU. The implication is probably more around that one could use a 32 bit word size posit than a 64 bit word size posit, or similar, for many applications which would make this true. This is still TBD for many classical HPC problems I think though. On an equivalent word size basis, the maximum precision of a posit (assuming reasonable exponent scale) is much larger than an IEEE float at a given word size. Adders and multipliers must be sized to handle this (e.g., a multiplication between two posits with maximal precision (1 <= x < 2) involves requires the full multiplier to handle this case). Multipliers have a quadratic dependency on the significand size. Subnormal handling in IEEE adds a lot of complexity, but not as much as a significantly larger multiplier.
- phkahler 5y ago>> Having investigated posits by running RTL implementations through synthesis on 7 and 28 nm nodes... Is it possible to implement 32 and 64 bit posits in similar area to floating point? Can the calculations be had in the same number of cycles or fewer? I feel like these are the two most important questions. If the answers are yes, then I think it may be worth doing implementations for the increased precision and simplicity (lack on NaN and other IEEE quirks).
- Someone 5y agoI don’t understand why not having NaNs would increase simplicity. Isn’t that just moving the problem of detecting various forms of over/underflow from the CPU to the programmer using it?
- adgjlsfhk1 5y agoIt's not quite right to say that Posits don't have NaN. They have a NaR (not a real) value that's fairly similar. The simplification is that they don't have 2^52 of them, and don't have -0, Inf, -Inf, or the mess with subnormals (which are necessary but not always well supported). Also, NaR compares equal to itself, fixing one of the biggest bugs in floating point.
- deleted 5y ago[deleted]
- scrubs 5y agoNice! Where can I get the posit tee-shirt? Asking for others :)
- deleted 5y ago[deleted]
- pmarreck 5y agoIs it true that these are just a curiosity (albeit a compelling one) and not practical until hardware support for them comes along, because until then they will always be slower than floating-point on supporting hardware?
- FullyFunctional 5y agoYou could say that about anything at some point, not least RISC-V. However, there _are_ hardware implementations of posit available, but given the inertial of existing code and data that assumes IEEE-754, posits are more like to see adoption in specialized areas where the higher information density is enough of a win. Or in green field application without concerns for legacy.
- pmarreck 5y agoI think one all-around-good use case/feature is the fact that multiplying a very large number by a very small number has significantly less error in posits vs. IEEE754.
- adgjlsfhk1 5y agoI don't think this is true. In floats, multiplying a big number by a small number has the same accuracy of any other non-overflowing/underflowing multiplication. In posits, the multiplication will be exact, but only because the big and small numbers were less accurate in the first place. If you have something like f(x)*g(y) where f(x) is really big and g(y) is really small, floats will give more accuracy.
- pmarreck 5y agoI'm going off the chart shown in Figure 15 of the cited Cornell post about it, which seems to disagree with your assertion
- adgjlsfhk1 5y ago
- Lerc 5y agoIs there any good documentation about the algorithms for adding and multiplying posits? Most of the stuff I have found focuses on how well they represent numbers, and less on how to manipulate the bits.
- adgjlsfhk1 5y agoThe basics of the algorithms are essentially the same as for floating point. You separate the mantissa, exponent, and sign and then do the calculations in the same way.
- ncmncm 5y agoI was beginning to think posits had been forgotten. I wonder about the prospects for (maybe 16-bit) posits in AI engines. It seems like a natural fit not so constrained by history, unlike traditional languages.