3 ms·
It seems worse than 3.6, but a bit smaller. UPD. was wrong on smaller, it's actually much larger
by lostmsu 2mo ago
It seems worse than 3.6, but a bit smaller.
UPD. was wrong on smaller, it's actually much larger
- IsTom 2mo agoHow is 30B smaller than 27B?
- LeBit 2mo agoIt uses fractal compression
- lostmsu 2mo agoThey say it is trained with quantization awareness, so it should only be 15GB or so. Qwen was only trained in FP8 with QAT. UPD, NVM, got misled by comments here. It is actually almost 60 GB so much larger
- xienze 2mo agoYou're mixing up sizes of different quants. The 60GB is unquantized, and Qwen's unquantized size is around 54GB. Their sizes as like quantization levels are similar.
- lostmsu 2mo agoFrom my perspective it doesn't make sense to talk about the number of parameters. What matters is model size in bytes and its performance at that certain size. Meta actually relesed official 4 bit quants in 17GB, but I haven't seen any indication that training was quant-aware, so the quants are not going to have same performance. 3.6 27B has official FP8 quant that AFAIR was trained with quantization awareness. The best example is last year's gpt-oss which was released prequantized in mxfp4 so 20B parameter model was under 14GB and 120B was under 70GB right away.
- petu 2mo ago> but I haven't seen any indication that training was quant-aware readme on huggingface says they've benchmarked the quants -- for 17GB quant reported 1% avg loss across 15 benchmarks (sadly no breakdown). I assume that's strong enough signal for QAT. Not just first party quants, but they cared to monitor degradation.
- lostmsu 2mo ago> sadly no breakdown That's exactly the point. We know short context knowledge stuff does not regress with quantization. But I expect agentic intelligence to suffer greatly. If I were to pick one bench, I would like to compare quants on TerminalBench Hard. But then Glimmer already loses to 3.6 27B on it by a large margin.
- ricardobeat 2mo agoQuantization awareness doesn’t change the size of the weights, just means it won’t degrade when quantized. QAT = quantization aware training. They will both be very similar in size at the same quant.
- jakswa 2mo agoI'll back up your smaller claim, but be specific that it's UD-Q4_K_XL size: - muse glimmer: 15.9GB - qwen 3.6 27B: 17.6GB My video card is so close to its limit that these GB thresholds are mattering too much for me :D