I do not see a quantized version of kimi-k3 on huggingface that can fit into 128GB. The smallest Unsloth q version is 594GB https://huggingface.co/unsloth/Kimi

I do not see a quantized version of kimi-k3 on huggingface that can fit into 128GB. The smallest Unsloth q version is 594GB https://huggingface.co/unsloth/Kimi-K3-GGUF https://huggingface.co/unsloth/Kimi-K3-GGUF