3 ms·

Thanks! Oh Qwen3's own GGUFs also works, but ours are dynamically quantized and calibrated with a reasonably large diverse dataset, whilst Qwen's ones are not -

Thanks! Oh Qwen3's own GGUFs also works, but ours are dynamically quantized and calibrated with a reasonably large diverse dataset, whilst Qwen's ones are not - see https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs

bityard 8mo ago
I've read that page before and although it all certainly sounds very impressive, I'm not an AI researcher. What's the actual goal of dynamic quantization? Does it make the model more accurate? Faster? Smaller?
- itake 8mo ago
  More accurate and smaller. quantization = process to make the model smaller (lossy) dynamic = being smarter about the information loss, so less information is lost
  - bityard 8mo ago
    Thanks, that makes sense.