2 ms·
Is this the first major model to be natively FP8? I was wondering why people hadn't done it yet. Seems like a big win when hardware supports it.
by modeless 3y ago
Is this the first major model to be natively FP8? I was wondering why people hadn't done it yet. Seems like a big win when hardware supports it.
- a_wild_dandan 3y agoNo, e.g. Yi-34B.
- modeless 3y agoAs far as I can tell Yi-34B is natively 16 bit float, the 8 bit version is quantized. https://huggingface.co/01-ai/Yi-34B#quantization https://huggingface.co/01-ai/Yi-34B#quantization