4 ms·
No, e.g. Yi-34B.
by a_wild_dandan 3y ago
No, e.g. Yi-34B.
- modeless 3y agoAs far as I can tell Yi-34B is natively 16 bit float, the 8 bit version is quantized. https://huggingface.co/01-ai/Yi-34B#quantization https://huggingface.co/01-ai/Yi-34B#quantization