3 ms·
Am I misreading the table? Unsloth Q4_K_M PPL: 6.6053 KLD 99.9%: 0.5478 KLD mean: 0.0192 bartowski Qwen_Q4_K_M PPL: 6.6097 K
by lostmsu 8mo ago
Am I misreading the table?
Unsloth Q4_K_M
PPL: 6.6053 KLD 99.9%: 0.5478 KLD mean: 0.0192
bartowski Qwen_Q4_K_M
PPL: 6.6097 KLD 99.9%: 0.5771 KLD mean: 0.0182
Barely noticeable drop in PPL; noticeable KLD drop (good, 5%); but worse KLD mean (bad, 5%).
- danielhanchen 8mo agoYou forgot to check the disk sapce - _M and _XL are not the same across quants: Unsloth Q4_K_M 18.49GB 0.5478 KLD 99.9% 0.0192 mean Unsloth Q4_K_XL 19.17GB 0.4097 KLD 99.9% 0.0137 mean bartowski Q4_K_M 19.77GB 0.5771 KLD 99.9% 0.0182 mean
- lostmsu 8mo agoThe table doesn't have bartowski Q4_K_XL to compare, but given the metrics of _Ms aren't universally better it's unclear if smaller size doesn't come with a cost.