3 ms·

Any idea how these formats compare to POSITs for AI computations? Also: It's not cool IMHO that they have two distinct formats for FP8.

pclmulqdq 3y ago
The promises of POSITs don't seem to hold water in terms of their application-level benefits, n-bit posits need equivalent-sized hardware to 2n-bit floats, and the numerical analysis on them is hell. All in all, they were an interesting thought experiment. Compressed/quantized storage of floating point numbers seems to just be better.
- Dylan16807 3y ago
  > n-bit posits need equivalent-sized hardware to 2n-bit floats Not in general. Every 8 bit posit with es=1 can be decoded to a 10 bit E5M4 float. Every 8 bit posit with es=2 can be decoded to E6M3, and almost all fit into E5M3. Though overall I'm not sure if posits are particularly useful here. Posits give you more precision near 1.0, and they give you better range for outliers. The latter almost certainly helps, but I don't know about the former. Saving 1 bit wouldn't hurt though.
  - pclmulqdq 3y ago
    If you think of posits as a storage format, it's true that you only need an extra posit->float decoder, but you need bfloat16 to exactly decode all 8-bit posits into a single, common FP format. In terms of hardware to operate directly on posits, things get ugly. Floats are actually relatively easy to work with due to the normalized storage format and fixed precision across the range of numbers, while posits don't have that.
    - Dylan16807 3y ago
      > If you think of posits as a storage format, it's true that you only need an extra posit->float decoder, but you need bfloat16 to exactly decode all 8-bit posits into a single, common FP format. But you're not limited to powers of two. If you're decoding to 16 bit floats then you can use posits up to 13 bits wide. > In terms of hardware to operate directly on posits, things get ugly. Floats are actually relatively easy to work with due to the normalized storage format and fixed precision across the range of numbers, while posits don't have that. But is that extra shifting bigger than the multiplier unit? It's hard for me to see it growing the circuit that much.

pclmulqdq 3y ago

The promises of POSITs don't seem to hold water in terms of their application-level benefits, n-bit posits need equivalent-sized hardware to 2n-bit floats, and the numerical analysis on them is hell. All in all, they were an interesting thought experiment. Compressed/quantized storage of floating point numbers seems to just be better.

Dylan16807 3y ago
> n-bit posits need equivalent-sized hardware to 2n-bit floats Not in general. Every 8 bit posit with es=1 can be decoded to a 10 bit E5M4 float. Every 8 bit posit with es=2 can be decoded to E6M3, and almost all fit into E5M3. Though overall I'm not sure if posits are particularly useful here. Posits give you more precision near 1.0, and they give you better range for outliers. The latter almost certainly helps, but I don't know about the former. Saving 1 bit wouldn't hurt though.
- pclmulqdq 3y ago
  If you think of posits as a storage format, it's true that you only need an extra posit->float decoder, but you need bfloat16 to exactly decode all 8-bit posits into a single, common FP format. In terms of hardware to operate directly on posits, things get ugly. Floats are actually relatively easy to work with due to the normalized storage format and fixed precision across the range of numbers, while posits don't have that.
  - Dylan16807 3y ago
    > If you think of posits as a storage format, it's true that you only need an extra posit->float decoder, but you need bfloat16 to exactly decode all 8-bit posits into a single, common FP format. But you're not limited to powers of two. If you're decoding to 16 bit floats then you can use posits up to 13 bits wide. > In terms of hardware to operate directly on posits, things get ugly. Floats are actually relatively easy to work with due to the normalized storage format and fixed precision across the range of numbers, while posits don't have that. But is that extra shifting bigger than the multiplier unit? It's hard for me to see it growing the circuit that much.