4 ms·
'brain floating point' is also bad, but no one cares because it's just bfloat16. If this becomes popular, it will just be tf32 or tfloat32 or something.
by rrss 6y ago
'brain floating point' is also bad, but no one cares because it's just bfloat16. If this becomes popular, it will just be tf32 or tfloat32 or something.
- QuixoticQuibit 6y agoI never knew what the “b” in “bfloat” was in all these new DL chips… until today. Man that’s bad.
- boulos 6y agoDisclosure: I work on Google Cloud. I wouldn't worry about it. Looks like the Anandtech article [1] doesn't either :) > bfloat16, a format popularized by Intel [1] https://www.anandtech.com/show/15801/nvidia-announces-ampere-architecture-and-a100-products https://www.anandtech.com/show/15801/nvidia-announces-ampere...
- joaogui1 6y agoIt's (Google) Brain float
- solidasparagus 6y agoI mostly agree. But bfloat16 at least uses 16 bits. tfloat32 is a 19 bit format and that just seems like a terrible naming decision to me.