3 ms·
The default Qwen "quantization" is not "bad", it's "large". Unsloth releases lower-quality versions of the model (Qwen in this case). Think about taking a 95%
by dist-epoch 7mo ago
The default Qwen "quantization" is not "bad", it's "large".
Unsloth releases lower-quality versions of the model (Qwen in this case). Think about taking a 95% quality JPEG and converting it to a 40% quality JPEG.
Models are quantized to lower quality/size so they can run on cheaper/consumer GPUs.
- danielhanchen 7mo agoLove the JPEG analogy :)