2 ms·
> I've noticed the reasoning behaves... weirdly Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model tha
by kouteiheika 17d ago
> I've noticed the reasoning behaves... weirdly
Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).
- spijdar 17d agoIt's Unsloth's UD-IQ4_XS, and it appears to actually work pretty well, regardless of the occasional CoT amnesia. Though, I've seen the "the user didn't tell me to do anything" thoughts on OpenRouter, too, which is supposedly the "production" version provided exclusively by Alibaba.
- seemaze 17d agoCan confirm here as well. Running ilintar/qwen3.8-flash-next-gguf-strix-halo (IQ4) on pwilkin/strix-llama.
- ryan-c 17d agoI see the same thing with Q8.