3 ms·
Unsloth has been dethroned by ISTA: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF Th
by WithinReason 17d ago
Unsloth has been dethroned by ISTA:
https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
The 3-bit quant is lossless based on benchmarks.
- raylad 16d agoI just checked that model (ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S) and it does much much worse on the "Please recite Jabberwocky" test than the original bf16 does. The bf16 only misses "snicker-snack" and this quantization becomes confused after the first stanza.
- sorenjan 16d agoI think it should be expected that a smaller model is worse at reciting memorized data than a big one. I also don't think it's a good use case of small local models. Can it find and recite Jabberwocky if given access to a web search tool?
- 0xbadcafebee 16d agoForgetting things isn't lossless though is it? Makes the benchmark and the finding quite suspect
- sorenjan 15d agoIt depends on what you mean by lossless. If both models can perform the same tasks it can be considered lossless for those tasks. That task might be more related to language understanding rather than memorizing, they have several benchmarks in the article.
- WithinReason 16d ago[dead]