3 ms·
I think it's BF16 trained then quantized to FP8, but unsure fully - I was also trying to find out if they used FP8 for training natively!
by danielhanchen 1y ago
I think it's BF16 trained then quantized to FP8, but unsure fully - I was also trying to find out if they used FP8 for training natively!
- jychang 1y agoQwen uses 16bit, Kimi and Deepseek uses FP8.
- danielhanchen 1y agoOh ok cool thanks!