4 ms·

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/ https://code.fb.com/ai-research/floating-point-math

• lonelappde 7y ago
  bfloat16 is very similar to traditional IEEE 754 floating point, and Google+Intel already have history building hardware for it.
• somethingnot 7y ago
  bfloat16 sounds like it could be supported with minimal changes to existing floating point units, maybe with just some improved microcode. FB's approach on the other side requires entirely redesigned and separate execution units. That's harder to justify, that silicon will remain dark for non-DL usage.
  • mjevans 7y ago
    It looks like a standard 32 bit float but with the mantissa truncated to fit in to 16 bits of storage. I imagine it still keeps most of the performance benefits since it's eliminating around 2/3rds of the longest binary component.
• kristianp 7y ago
  What does the "int8/32" mean in that paper?
  • hedgehog 7y ago
    In that context it's likely 8 bit operands and a 32 bit accumulator.
  • msclrhd 7y ago
    4x8-bit values in a 32-bit value? SIMD does this to perform an operation on 4 8-bit values in a single 32-bit register. There are other configurations, IIRC. https://en.wikipedia.org/wiki/SIMD#Software https://en.wikipedia.org/wiki/SIMD#Software
• Veedrac 7y ago
  ‘(8, 1, alpha, beta, gamma) log’ isn't a posit; it just shares the encoding of the exponent.