3 ms·
Qwen3.5 Small: 0.8B, 2B, 4B, 9B Released
- powera 7mo agoThis looks like somebody re-releasing QWEN models to promote their own company. https://news.ycombinator.com/item?id=47217305 https://news.ycombinator.com/item?id=47217305 is the link to QWEN's repo.
- cpburns2009 7mo agoIf you want to have a chance at running a large model, it needs to be quantized. The unsloth user on Huggingface manages popular quantizations for many models, Qwen included, and I think he developed dynamic GGUF quantization. Take Qwen/Qwen3.5-35B-A3B for example. It's 72 GB. While unsloth/Qwen3.5-35B-A3B-GGUF has quantizations from 9-38 GB.
- karmakaze 7mo agoUnsloth is one of, if not the most well-known provider of model quantizations. The release post of course should reference the source, but most probably use unsloth or bartowski quantized models being my go-tos so relevant/convenient.
- throwaway2027 7mo agoSo 27B at Q3 or 9B at Q8?
- karmakaze 7mo agoChart of how these compare[0] to the Qwen3 235B-A22B, Next-80B-A3B-Thinking, 30B-A3B-Thinking, 4B, 1.7B models. These new ones are very much punching above their weights. [0] https://www.reddit.com/r/LocalLLaMA/comments/1rivckt/visualizing_all_qwen_35_vs_qwen_3_benchmarks/ https://www.reddit.com/r/LocalLLaMA/comments/1rivckt/visuali...