3 ms·
I can say that this really works great, I'm a heavy user of the unsloth dyanmic quants. I run DeepSeek v3/r1 in Q3, and ernie-300b and KimiK2 in Q3 too. Amaz
by segmondy 1y ago
I can say that this really works great, I'm a heavy user of the unsloth dyanmic quants. I run DeepSeek v3/r1 in Q3, and ernie-300b and KimiK2 in Q3 too. Amazing performance. I run Qwen3-235b in both Q4 and Q8 and can barely tell the difference so much so that I just keep Q4 since it's twice as fast.
- someone13 1y agoWhat hardware do you use, out of curiosity?
- jychang 1y agoIn the current era of MoE models, the system RAM memory bandwidth determines your speed more than the GPU does.
- danielhanchen 1y agoThanks for using them! :)