4 ms·

Is it possible to "pretend" that one's float16 datatype is in fact bfloat16? Since they occupy the same about of memory?

• lloydatkinson 3y ago
  It's possible to act, in software, as any type of arbitrary length numeric value. It's much faster to do it in hardware thought.
  • tomtom1337 3y ago
    Oh, I should have clarified - could one start with a bfloat16 on software-side, convert to float16 (so that e.g a 3.4E38 float16 becomes a 65504 float16), then do any "heavy math" in fast hardware float16 instructions, and then convert back at the end?
    • warpspin 3y ago
      No, you can't. The exponent in a float16 is too small. You'd rather convert back to a 32-bit float, do your operations, and then throw away the surplus precision and convert back to bfloat16.
• tryfinally 3y ago
  You can store it and move it around, but arithmetic operations are prohibitively expensive without hardware acceleration. (Note that bfloat16 has a different range than float16, so you can't interpret one as the other)
  • tomtom1337 3y ago
    Oh, I should have clarified - could one start with a bfloat16 on software-side, convert to float16 (so that e.g a 3.4E38 float16 becomes a 65504 float16), then do any "heavy math" in fast hardware float16 instructions, and then convert back at the end?
    • sapiogram 3y ago
      Nothing necessarily wrong with that code, but it also kinda smells. Why even store it as bfloat16 at all? You risk getting the numerical disadvantages of both float16-representation, and none of the advantages.