7 ms·
4-bit quantization is native for Kimi 2.x series.
by zozbot234 6mo ago
4-bit quantization is native for Kimi 2.x series.
- CamperBob2 6mo agoYou're right, I was thinking of Qwen. K2.6 will run at UD-Q2_K_XL precision on 4x RTX6000 boards, but I have no idea if it's worthwhile.