4 ms·
They're not really "grafting" it, they're implementing it as a first-class data type.
by fortran77 7y ago
They're not really "grafting" it, they're implementing it as a first-class data type.
- _Nat_ 7y agoDefinitely feels click-bait-y. Ditto for them calling it "Brain Floating Point" [1]. I mean it appears to be a typical floating-point numeric data type with some of the precision truncated to reduce its cost. I guess they might be trying to make it sound super-fancy to preempt people seeing it as cheap or/and low-tech? [1]: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://en.wikipedia.org/wiki/Bfloat16_floating-point_format
- exsf0859 7y agoIt's called brain floating point because it was developed by Google's "Google Brain" machine learning project.
- _Nat_ 7y agoYeah, I get the marketing perspective; just, we typically describe primitive data types in terms of what they are from a technical perspective rather than a marketing perspective.
- vlovich123 7y agofp16 & fp8 refer to IEEE 754 floating point which has notable differences from bfloat16 & make them have worse performance for machine learning.
- Traster 7y agoYeah the point is 'reduced precision FP16' would make a lot more sense - since it actually describes what it does. 'Brain' floating point makes you wonder how Neuralink are planning to use this datatype.
- CalChris 7y agoIt's not really first class if you have to CVT BF16 to FP32 if you want to use FMA.
- bonzini 7y agoFMA on BF16 uses FP32 for the accumulator for accuracy reasons, but you can have instructions for "Rm = Rm + low(Rn) * low(Rp)" and "Rm = Rm + high(Rn) * high(Rp)", and they can be faster than FP32 "Rm=Rm+Rn*Rp" because there are fewer bits in the mantissas. Also, converting BF16 to FP32 and back is just a vector shuffle that sticks/drops 16 extra mantissa bits at the end of each FP16 value, so it's cheaper than other floating point conversions. This means that even if you occasionally have to escape to FP32, the overhead is low and you keep all of the memory bandwidth benefits.
- CalChris 7y agoThat's a good analysis. I particularly like the idea that if the overhead is low it gets hidden in the memory BW savings.