3 ms·
do you think it would be possible to quantize this model and still get good results?
by tmaly 1y ago
do you think it would be possible to quantize this model and still get good results?
- sparacha 1y agoyes - we have already published a quantized version here: https://huggingface.co/katanemo/Arch-Router-1.5B.gguf https://huggingface.co/katanemo/Arch-Router-1.5B.gguf. The performance difference with a quant version is negligible. I'll run another analysis and update the thread shortly
- sparacha 1y agoOverall performance degrades from 93.17 -> 92.99 with a quantized version
- deleted 1y ago[deleted]