3 ms·

Conversions from IEEE-32 to BF16 don't round?

• londons_explore 3y ago
  I don't believe the standard defines it. I believe implementations truncate (ie. round towards zero). Remember BF16 was invented specifically to be able to be backwards compatible with existing silicon - and pulling 2 bytes out of 4 is a far cheaper operation than any rounding.
  • kelnos 3y ago
    Just to elaborate, as I was confused about this and had to look it up: BF16 is indeed designed to just be a truncated F32: you can grab the top 16 bits of a F32 value and it'll still "make sense": the sign bits are in the same place in both (unsurprisingly), and the exponent part of BF16 and F32 are both 8 bits. In the case of the mantissa, you end up grabbing the top 7 bits of the F32's 23-bit mantissa, so it all works out, as this will "round" the value toward zero.
  • pclmulqdq 3y ago
    There's no standardized definition of BF16.