3 ms·
fp16 & fp8 refer to IEEE 754 floating point which has notable differences from bfloat16 & make them have worse performance for machine learning.
by vlovich123 7y ago
fp16 & fp8 refer to IEEE 754 floating point which has notable differences from bfloat16 & make them have worse performance for machine learning.
- Traster 7y agoYeah the point is 'reduced precision FP16' would make a lot more sense - since it actually describes what it does. 'Brain' floating point makes you wonder how Neuralink are planning to use this datatype.