4 ms·
You should mention that it is 4bit quant. Still very impressive!
by Alifatisk 11mo ago
You should mention that it is 4bit quant. Still very impressive!
- geerlingguy 11mo agoKiki K2 was made to be optimized at 4-bit, though.
- natrys 11mo agoThat's the Kimi K2 Thinking, this post seems to be talking about original Kimi K2 Instruct though, I don't think INT4 QAT (quantization aware training) version was released for this.
- elif 11mo agoI think when you say trillion parameters, it's implied that it's quantized