4 ms·
Wouldn't conversion between the steps lead to round off?
by noobermin 8y ago
Wouldn't conversion between the steps lead to round off?
- Tuna-Fish 8y agoConversion from Bfloat16 to f32 is just extending the mantissa with zeroes. Basically, what they are computing is: f32 acc = C_a0 * C_b0 + Ca1 * Cb1 + Ca2 * Cb2 ... with very many coefficients, all of which are Bfloat16. The precision of the coefficients is not that important, but they can be of substantially different magnitude, so the coefficients can use few bits in the mantissa but the accumulator needs to be wider.