4 ms·
I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases wh
by KronisLV 2mo ago
I hope really badly that we'll get a new 35B A3B or similar MoE model!
I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model <100B would be the sweet spot (when you have the VRAM but not the TDP or compute power). Heck, I'd gladly take A5B or A8B or even A10B as a sort of middle ground.
Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304 https://xcancel.com/Alibaba_Qwen/status/2088280182356611304
- Casteil 2mo agoI'm hoping too that they'll put out some MoE variants. Qwen3.5:122b:a10b can run about twice as fast as this 27b dense model. Edit: Like its predecessors, 3.8 seems really inclined to overthinking, and on a 27b dense model that's kind of painful. I think I'm going to stick with gemma4:26b-a3b as my go-to because it runs about 4x as fast and tends to only need a fraction of the tokens in its 'thinking' stage to get the same or similar answer.
- Phemist 2mo agoDid you try the claude reasoning traces finetune for qwen3.6? I find that it works muuch better. I assume the same 3.8 finetune will be released at some pointas well. Edit: link - https://huggingface.co/rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled https://huggingface.co/rico03/Qwen3.6-27B-Claude-Opus-Reason...
- satvikpendem 2mo agoReduce or turn down thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
- Casteil 2mo agoYeah, that's probably the answer given that it apparently defaults to 'xhigh'.
- dannyw 2mo agoProbably helps it score a little bit better in benchmarks :) `medium` seems like a nice balance so far; along with some light steering to vary think effort as needed for task and being pragmatic.
- wgd 2mo agoInterestingly 'medium' is the closest thing the _model itself_ has to a default thinking level. The chat template injects directions [1] at the very start of the system message when the reasoning effort is 'xhigh' or 'low' but 'medium' implicitly just means no added reasoning-level instructions. [1] The specific directions are "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer." and "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."
- mirekrusin 2mo ago[flagged]
- isomorphic 2mo ago"inclined to overthinking," holy cats you're not kidding! On a Mac mini M4 Pro 64GB I prompted it with "svg owl" and it thought for 17m12s, outputting 36.3KiB of thinking chatter. It did end up producing a 20.2KiB HTML+JS+SVG file with a very nice owl, including cursor-tracking animation, but it ran for more than a half hour! The MoE models are stupendously faster.
- kzrdude 2mo agoI think that's called test-time scaling i.e using more tokens at infer time to squeeze out higher model performance. That's must be part of the explanation for good benchmark results.
- Casteil 2mo agoYep.. it's pretty obnoxious for real-world use with the default 'xhigh' thinking. Ridiculous amount of "Wait, actually.." which might help for complex coding tasks but makes it unbearable for general purpose use.
- try-working 2mo agothat seems to be how most Chinese models achieve increased benchmark scores. GLM and Kimi models are "thinkslop" models that reason over their own thinking, which increases cost and decreases speed significantly. That's why GPT is in a different tier altogether - faster, smarter, and sometimes cheaper.
- nubg 2mo agowhat harness did you use?
- isomorphic 2mo agoNo agentic harness; I just prompted literally "svg owl" into LM Studio 0.4.21+2. macOS Tahoe 26.6.1, Mac mini M4 Pro (14 core, 20 graphics cores) 64GB RAM. (I put the machine config so the 17m12s of thinking has some context.) It got about 12.75 tok/s and used 21,769 tokens. For comparison's sake, Qwen 3.6 35B A3B MLX, same prompt, same machine, thought for 1.59 seconds, used 2,398 tokens, and hit 80.83 tok/s. It actually output just a straight SVG file of an owl (not a bad owl either!), whereas Qwen 3.8 27B spent a zillion times longer gold-plating everything and output HTML+JS+SVG with interactive animation, sound, etc., etc.
- peri-cl 2mo agoSame here! Qwen3.6-35B-A3B is the only local model I've found that runs reasonably on my iGPU. Looks like me and and my noisily-wheezing laptop will be sitting out this upgrade.
- expedited123 2mo agoMind sharing your laptops specs? Just interested to see what is needed to locally run Qwen3.6-35B-A3B
- peri-cl 2mo agoDon't mind! It's 64 GiB dual-channel DDR5-6400, i.e. roughly 100 GiB/s of bandwidth. (AMD 7840U (Zen 4)) I'm using a Q4 quantization from unsloth (Qwen3.6-35B-A3B-UD-Q4_K_XL). It gets up to ~20 tokens/second in generation. I don't know precisely how much KV cache I can safely use, but it's in between 140k–256k. (I.e., 140k reliably works, 256k kernel-crashes from OOM. Don't feel like bisecting). Inference is llama.cpp with the Vulkan GPU backend on Linux. (I.e., -DGGML_VULKAN=1 on the llama.cpp build, and --gpu-layers all on llama-cli or llama-server. (And for my specific setup, two kernel parameters specific to amdgpu: ttm.pages_limit and ttm.page_pool_size. A driver VRAM limiter. Look it up if you're on amdgpu!)).
- expedited123 2mo agoThanks! I don't have any knowledge of running models locally. I assume it would not be able to handle an unquantized Qwen3.6-35B or is it irrelevant as you almost always would want to run a quantized version of the model on consumer hardware?
- seanmcdirmid 2mo agonot parent, but 4-bit quantization is generally consider a good trade off for speed/performance, so you might use it even when you aren't on consumer hardware, but definitely when you are on consumer hardware.
- 2mo ago
- Alifatisk 2mo ago> I'd gladly take A5B or A8B or even A10B as a sort of middle ground. Whats up with focusing on the active param count? Do yall fiddle with the weights or something?
- martinald 2mo agoYou can run these on CPUs at a somewhat reasonable speed.
- KronisLV 2mo agoOr (somewhat) low TDP GPUs for that matter, like workstation ones, that might have enough total VRAM but not the best bandwidth/compute.
- kennywinker 2mo agoTotal param count decides how much vram you need to run it. Active param count decides how fast it runs. My 10 year old GPU can load quantized 35B or 27B, but it can’t process 27B parameters per token faster than 2-4tok/s, while it can do A3B at >40tok/s
- Alifatisk 2mo agoThank you Kenny
- jwr 2mo agoMe too. 35B A3B runs really fast on my MacBook Pro (M4 Max) and is suitable for real-time tasks like dictation post-processing. The dense model is not.
- colordrops 2mo agoI'm still confused about Qwen 3.6 35B A3B. Everything I read said that the 27B model performs better at coding tasks, so what's the purpose of the 35B model?
- formerly_proven 2mo agoRuns faster on bad hardware.
- apothegm 2mo agoOr even on fairly high end (by general public standards, not gaming community standards) consumer hardware. Fantastic fit for M-series Macs with 32-96GB RAM.
- Fervicus 2mo agoIt's better for VRAM poor people. I get 4-5 t/s with 27B and 20-30 t/s with 35B A3B.
- dofm 2mo agoAlso radically better on an M1 Max. I get well up into the 60s t/s with the A3B, stuck at 9.5 or so with this new 27B, though perhaps an MLX build will help.
- pickledish 2mo agoThe "a3b" refers to its active parameters -- unlike 27b it is a mixture of experts model, so it runs much faster, about as fast as a 3b model, but needs as much memory as a 35b model! So good for unified memory systems like macs :)
- jakswa 2mo agoanecdotes: 35B-A3B does want more memory, bigger model. But if you get it running it will be faster and more enjoyable to use -- text will fly by -- due to only 3B params being active, in my experience at least.
- 2mo ago
- SwellJoe 2mo agoAgentWorld is a pretty recent MoE release from the Qwen folks with quite a bit better performance than 3.6, released around the same time as 3.7 Plus/Max, I guess. Not sure why it didn't get more attention, as it is definitely better than 3.6 35B A3B on all dimensions, but especially for agentic use. Still nowhere near the dense models (even 3.6 27B), but clearly an upgrade in the small Qwen MoE line. https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B KAT Coder is another Qwen 3.6 MoE fine-tune that also improves on 3.6 by a measurable amount. But, I do hope for a bigger MoE, in the 70B to 120B range, something in the Coder Next lineage. I've got a Strix Halo that isn't getting used to its best ability because the best models all run fine on my faster desktop dual 32GB GPU setup, and they run too slow for comfort on the Strix Halo.
- nullc 2mo agoHUH? AgentWorld is a simulation of the world (e.g. tools and programs) for use by an agent!
- SwellJoe 2mo agoAnd, yet it outperforms Qwen 3.6 35B A3B on Terminal Bench and SWE, etc. I dunno. Edit: I guess you're right; apparently it's for simulation. I didn't look into it beyond the benchmarks. But, it does work in an agentic context, regardless. It'll write code, and drive an agent.