3 ms·
It's not really first class if you have to CVT BF16 to FP32 if you want to use FMA.
by CalChris 7y ago
It's not really first class if you have to CVT BF16 to FP32 if you want to use FMA.
- bonzini 7y agoFMA on BF16 uses FP32 for the accumulator for accuracy reasons, but you can have instructions for "Rm = Rm + low(Rn) * low(Rp)" and "Rm = Rm + high(Rn) * high(Rp)", and they can be faster than FP32 "Rm=Rm+Rn*Rp" because there are fewer bits in the mantissas. Also, converting BF16 to FP32 and back is just a vector shuffle that sticks/drops 16 extra mantissa bits at the end of each FP16 value, so it's cheaper than other floating point conversions. This means that even if you occasionally have to escape to FP32, the overhead is low and you keep all of the memory bandwidth benefits.
- CalChris 7y agoThat's a good analysis. I particularly like the idea that if the overhead is low it gets hidden in the memory BW savings.