3 ms·

Qwen 32B is a dense model. The competitors for GPT-OSS 20B are max 8B dense models. You’re comparing it against models it’s not competing with and calling it cr

Qwen 32B is a dense model. The competitors for GPT-OSS 20B are max 8B dense models. You’re comparing it against models it’s not competing with and calling it crap. That’s like saying Ferrari is better than Toyota. Sure, but only if we’re comparing 0-60. If we add a budget of 20K, suddenly the Toyota starts looking more competitive. I never claimed it was a frontier model. Just best in class for the performance it can achieve and the memory footprint it can fit in. And btw OSS did super well on domain specific tests without fine tuning. A model I don’t need to fine tune beats one that does.

Der_Einzige 1y ago
You don't mention that MoE models horrible damage logprobs (by definition) and basically need a whole new theory of LLM sampling written for them. Dense models are better for a reason, and the idea that "everyone is doing MoE now and dense models are dead" is total bunk nonsense. You can quantize dense models, and 4 bit quantized Qwen 32B is still better than full precision GPT-OSS. Luckily Unsloth even gives you tools to go down to 1.58bits!
stingraycharles 1y ago
Excuse my ignorance, but what is a “dense” model? Why is comparing GPT-OSS 20B against 8B dense models “fair” but not when comparing it against 32B models?
- jychang 1y ago
  So gpt-oss-20b is a sparse MoE model Which means it has ~3b parameters active per token. Qwen3-32b has 32b params active per token
- spwa4 1y ago
  An MoE model will not activate most of the model for any given query, so there is zero compute happening across an increasing part of the model. Dense model means 32B parameters => 32B get used in calculation for every token. Every calculation takes time, and assuming similar latent space size (which they all have). For example Qwen-32B MoE model has for example 80B parameters, but only 3B get used in calculation for any given token. For example Qwen3-Next 80B A3B Performance comparison: Qwen-32B => 56 tok/sec, 32 GB of VRAM Qwen3-Next 80B A3B => 167 tok/sec, 85 GB of VRAM So despite being close to 3x "bigger", Qwen3-Next is more than 3 times faster with the same compute capacity. There's a but though. But because what gets activated from one token to the next is a different subset of the model, it is still critical to have all 80B parameters loaded into memory. So MoE performs much better with less compute, at the cost of more memory. It also performs better on benchmarks, it is a better model. Similar techniques have long been used in ML to great success, rather than trying to create one brilliant model, create many that each have pros and cons, and then train a second model to figure out the best model for the task in front of you. There's even a name for the practice "ensemble models". MoE only kind-of fits (because you can't easily swap out models) There's other factors, the big one being attention. That's why non-attention models, like MAMBA, will wipe the floor in terms of performance per flop (a compute unit), with anything else. When it comes to intelligence however ...
yowlingcat 1y ago
Qwen3 Next and Qwen3-30b-a3b are pretty decent proxies for GPT-OSS 120B and 20B respectively (and in fact are both MoEs with 3B active rather than 8B active parameters), and they lap GPT-OSS pretty hard in this specific benchmark, getting to #17 and #33 respectively. That being said, it's hard to take benchmarks beyond a grain of salt because real world tasks that I try to use these models for always have a lot more variation than the benchmarks illustrate. I do view GPT-OSS as a pretty good alternative to the Qwen models in some cases but there are tradeoffs -- while I see better reasoning from GPT-OSS sometimes, the prompt adherence and overall flexibility of the Qwen models makes them a lot better IMO as general purpose local open weight models.
- vlovich123 1y ago
  Those models got released later than GPT-OSS. It’s like saying the Android phone released 6 months after the iPhone is faster. Maybe, but it also had 6 extra months of development.

Der_Einzige 1y ago

You don't mention that MoE models horrible damage logprobs (by definition) and basically need a whole new theory of LLM sampling written for them. Dense models are better for a reason, and the idea that "everyone is doing MoE now and dense models are dead" is total bunk nonsense. You can quantize dense models, and 4 bit quantized Qwen 32B is still better than full precision GPT-OSS. Luckily Unsloth even gives you tools to go down to 1.58bits!

stingraycharles 1y ago

Excuse my ignorance, but what is a “dense” model? Why is comparing GPT-OSS 20B against 8B dense models “fair” but not when comparing it against 32B models?

jychang 1y ago
So gpt-oss-20b is a sparse MoE model Which means it has ~3b parameters active per token. Qwen3-32b has 32b params active per token
spwa4 1y ago
An MoE model will not activate most of the model for any given query, so there is zero compute happening across an increasing part of the model. Dense model means 32B parameters => 32B get used in calculation for every token. Every calculation takes time, and assuming similar latent space size (which they all have). For example Qwen-32B MoE model has for example 80B parameters, but only 3B get used in calculation for any given token. For example Qwen3-Next 80B A3B Performance comparison: Qwen-32B => 56 tok/sec, 32 GB of VRAM Qwen3-Next 80B A3B => 167 tok/sec, 85 GB of VRAM So despite being close to 3x "bigger", Qwen3-Next is more than 3 times faster with the same compute capacity. There's a but though. But because what gets activated from one token to the next is a different subset of the model, it is still critical to have all 80B parameters loaded into memory. So MoE performs much better with less compute, at the cost of more memory. It also performs better on benchmarks, it is a better model. Similar techniques have long been used in ML to great success, rather than trying to create one brilliant model, create many that each have pros and cons, and then train a second model to figure out the best model for the task in front of you. There's even a name for the practice "ensemble models". MoE only kind-of fits (because you can't easily swap out models) There's other factors, the big one being attention. That's why non-attention models, like MAMBA, will wipe the floor in terms of performance per flop (a compute unit), with anything else. When it comes to intelligence however ...

yowlingcat 1y ago

Qwen3 Next and Qwen3-30b-a3b are pretty decent proxies for GPT-OSS 120B and 20B respectively (and in fact are both MoEs with 3B active rather than 8B active parameters), and they lap GPT-OSS pretty hard in this specific benchmark, getting to #17 and #33 respectively. That being said, it's hard to take benchmarks beyond a grain of salt because real world tasks that I try to use these models for always have a lot more variation than the benchmarks illustrate. I do view GPT-OSS as a pretty good alternative to the Qwen models in some cases but there are tradeoffs -- while I see better reasoning from GPT-OSS sometimes, the prompt adherence and overall flexibility of the Qwen models makes them a lot better IMO as general purpose local open weight models.

vlovich123 1y ago
Those models got released later than GPT-OSS. It’s like saying the Android phone released 6 months after the iPhone is faster. Maybe, but it also had 6 extra months of development.