3 ms·
I am not that much into ML, just fiddled with it a bit, is tf32=fp16?
by XCSme 6y ago
I am not that much into ML, just fiddled with it a bit, is tf32=fp16?
- tbenst 6y agoNot quite, but close. “tf32” is 18 bits, but with the same 10 bits of exponent that fp32 has. It’s the range of fp32 with the precision of fp16. It’s a shame to see such unoriginality in new number representations. I’d much rather see Posit hardware acceleration: https://web.stanford.edu/class/ee380/Abstracts/170201-slides.pdf https://web.stanford.edu/class/ee380/Abstracts/170201-slides...
- rajnathani 6y agoTF32 is 19 bits, not 18 bits. There's an additional bit for sign. https://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precision-format/ https://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-prec...
- slaymaker1907 6y agoNo, tf32 has the same size exponent field as f32 but the mantissa size of f16, 10 bits.