4 ms·

Can you explain the type of quantization you support? would https://docs.unsloth.ai/basics/kimi-k2-how-to-run-locally https://docs.unsloth.ai/basics/kimi-k2-ho

by homarp 1y ago

Can you explain the type of quantization you support? would https://docs.unsloth.ai/basics/kimi-k2-how-to-run-locally https://docs.unsloth.ai/basics/kimi-k2-how-to-run-locally be faster with mirai?

AlekseiSavin 1y ago
right now, we support AWQ but are currently working on various quantization methods in https://github.com/trymirai/lalamo https://github.com/trymirai/lalamo