3 ms·
KTransformers now supports Kimi K2 for MoE offloading They claim 14 tps for the 4-bit quant on a single socket system with 600 GB RAM and 14 GB GPU memory.
by chithanh 1y ago
KTransformers now supports Kimi K2 for MoE offloading
They claim 14 tps for the 4-bit quant on a single socket system with 600 GB RAM and 14 GB GPU memory.