4 ms·
> In fact it's worse than many local models that can do it, including e.g. QwQ-32b. I'm not going to be surprised that a 20B 4/32 MoE model (3.6B parameters ac
by dragonwriter 1y ago
> In fact it's worse than many local models that can do it, including e.g. QwQ-32b.
I'm not going to be surprised that a 20B 4/32 MoE model (3.6B parameters activated) is less capable at a particular problem category than a 32B dense model, and its quite possible for both to be SOTA, as state of the art at different scale (both parameter count and speed which scales with active resource needs) is going to have different capabilities. TANSTAAFL.
- __alexs 1y ago[flagged]
- whynotminot 1y agoHe’s saying there’s different goalposts at different model sizes. Is that unreasonable?
- tomhow 1y agoPlease don't post snark like this on HN. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
- lannisterstark 1y agoThis isn't reddit.