3 ms·
Q5_1 is already old news too, K quants are faster and more space efficient for the same perplexity loss. https://old.reddit.com/r/LocalLLaMA/comments/142q5k5/u
by moffkalast 3y ago
Q5_1 is already old news too, K quants are faster and more space efficient for the same perplexity loss.
https://old.reddit.com/r/LocalLLaMA/comments/142q5k5/updated_relative_comparison_of_ggml_quantization/ https://old.reddit.com/r/LocalLLaMA/comments/142q5k5/updated...
- lolinder 3y agoFor sure, but I couldn't find numbers for the K quants that included inference speeds, so I settled on the older one. If MK-1 were trying to be honest they'd definitely want to benchmark against the newest methods!