3 ms·
Looking at their benchmarks there doesn't appear to be meaningful difference between their quants and bartowsky quants.
by lostmsu 8mo ago
Looking at their benchmarks there doesn't appear to be meaningful difference between their quants and bartowsky quants.
- danielhanchen 8mo agoNo our Qwen3.5 new ones show the opposite see https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks
- lostmsu 8mo agoAm I misreading the table? Unsloth Q4_K_M PPL: 6.6053 KLD 99.9%: 0.5478 KLD mean: 0.0192 bartowski Qwen_Q4_K_M PPL: 6.6097 KLD 99.9%: 0.5771 KLD mean: 0.0182 Barely noticeable drop in PPL; noticeable KLD drop (good, 5%); but worse KLD mean (bad, 5%).
- danielhanchen 8mo agoYou forgot to check the disk sapce - _M and _XL are not the same across quants: Unsloth Q4_K_M 18.49GB 0.5478 KLD 99.9% 0.0192 mean Unsloth Q4_K_XL 19.17GB 0.4097 KLD 99.9% 0.0137 mean bartowski Q4_K_M 19.77GB 0.5771 KLD 99.9% 0.0182 mean
- lostmsu 8mo agoThe table doesn't have bartowski Q4_K_XL to compare, but given the metrics of _Ms aren't universally better it's unclear if smaller size doesn't come with a cost.