3 ms·
Asked elsewhere, but why FP16 as opposed to BF16?
by lostmsu 5y ago
Asked elsewhere, but why FP16 as opposed to BF16?
- cjbgkagh 5y agoI'm using older Turing GPUs BF16 would require Ampere. The weights in my models tend to be normalized so the fraction would be more important than the exponent so I would probably still use FP16. I would need to test it though.