3 ms·

> and designed the first circuits, including a cool optimization for addition Curious, what trick? :) Wishing for mainstream CPU support for anything but IEEE

> and designed the first circuits, including a cool optimization for addition Curious, what trick? :) Wishing for mainstream CPU support for anything but IEEE numbers was always a pipe dream on anything but the decades-long term, but I gotta be honest, I was hoping the current AI hype wave would bring some custom silicon for alternative float formats, Posits included. > the addition operation takes up quite a bit more floorspace, and just quantizing to int4 is with grouped scaling factors is easier with existing gpu pipelines Can you elaborate on this part?

dnautics 1y ago
> trick Normally fp numbers have an "invisible leading one bit", so an ieee fp is (1+0.nnnn) x 2^E where nnn is the mantissa and E is the exponent. as posits are always two's complement, you can model negative posits as having an "invisible leading two bit", aka (2-0.nnnn) x 2^E-1. this greatly simplifies circuitry for addition and multiplication. > the addition operation takes up quite a bit more floorspace, I miswrote, it's multiplication. This is simple: since the math requires floorspace of size n^2 where n is the maximum resolution of the mantissa, and posits have more resolution around unity. we did floorspace estimates for fp32 versus p32 and it looked like the extra space required for the mantissa did not overtake the savings from the crazy logic required for sign flipping, nans, and denormals that IEEE requires that posits don't. > I was hoping the current AI hype wave would bring some custom silicon for alternative float formats, Posits included i was hoping for this too!
- adgjlsfhk1 1y ago
  I really want a standards body to standardize a positized version of floating point numbers. Everything about the encoding of posits (i.e. NaR, no -0.0, exponent encoding) is so much nicer. The tapered precision part of posits is IMO the least interesting part.
  - dnautics 1y ago
    the twos complement thing is really elegant (-p == (posit -(int p))) in c notation
    - adgjlsfhk1 1y ago
      relatedly, having the same ordering as signed ints just makes everything so much easier. You can just use your integer radix sort to sort your floats without having to deal with sign and nans first!
      - immibis 1y ago
        So do floats if you don't care about NaN
        tomsmeding 1y ago
        They do not; if you sort as unsigned ints then negative floats sort above positive ones, and if you sort as signed ints then the negative range is reversed. IEEE floats are sign-magnitude, not twos-complement.
  - andrepd 1y ago
    Add "simple and consistent rounding" to that list! There's a tentative standard btw: https://posithub.org/docs/posit_standard-2.pdf https://posithub.org/docs/posit_standard-2.pdf
- andrepd 1y ago
  > as posits are always two's complement, you can model negative posits as having an "invisible leading two bit", aka (2-0.nnnn) x 2^E-1. this greatly simplifies circuitry for addition and multiplication. Ahh indeed! I'm working on a software posit implementation (for fun, I'll eventually post it as a rust crate) so I already make use of this trick :) decoding a p64 into an exponent and twos complement signed mantissa can be done in 9 instructions on x86!

dnautics 1y ago

> trick Normally fp numbers have an "invisible leading one bit", so an ieee fp is (1+0.nnnn) x 2^E where nnn is the mantissa and E is the exponent. as posits are always two's complement, you can model negative posits as having an "invisible leading two bit", aka (2-0.nnnn) x 2^E-1. this greatly simplifies circuitry for addition and multiplication. > the addition operation takes up quite a bit more floorspace, I miswrote, it's multiplication. This is simple: since the math requires floorspace of size n^2 where n is the maximum resolution of the mantissa, and posits have more resolution around unity. we did floorspace estimates for fp32 versus p32 and it looked like the extra space required for the mantissa did not overtake the savings from the crazy logic required for sign flipping, nans, and denormals that IEEE requires that posits don't. > I was hoping the current AI hype wave would bring some custom silicon for alternative float formats, Posits included i was hoping for this too!

adgjlsfhk1 1y ago
I really want a standards body to standardize a positized version of floating point numbers. Everything about the encoding of posits (i.e. NaR, no -0.0, exponent encoding) is so much nicer. The tapered precision part of posits is IMO the least interesting part.
- dnautics 1y ago
  the twos complement thing is really elegant (-p == (posit -(int p))) in c notation
  - adgjlsfhk1 1y ago
    relatedly, having the same ordering as signed ints just makes everything so much easier. You can just use your integer radix sort to sort your floats without having to deal with sign and nans first!
    - immibis 1y ago
      So do floats if you don't care about NaN
      - tomsmeding 1y ago
        They do not; if you sort as unsigned ints then negative floats sort above positive ones, and if you sort as signed ints then the negative range is reversed. IEEE floats are sign-magnitude, not twos-complement.
- andrepd 1y ago
  Add "simple and consistent rounding" to that list! There's a tentative standard btw: https://posithub.org/docs/posit_standard-2.pdf https://posithub.org/docs/posit_standard-2.pdf
andrepd 1y ago
> as posits are always two's complement, you can model negative posits as having an "invisible leading two bit", aka (2-0.nnnn) x 2^E-1. this greatly simplifies circuitry for addition and multiplication. Ahh indeed! I'm working on a software posit implementation (for fun, I'll eventually post it as a rust crate) so I already make use of this trick :) decoding a p64 into an exponent and twos complement signed mantissa can be done in 9 instructions on x86!