2 ms·
Could you share some of your hardware details for Qwen 3.6? And are you using the dense or MoE variant?
by bobim 6mo ago
Could you share some of your hardware details for Qwen 3.6? And are you using the dense or MoE variant?
- deleted 6mo ago[deleted]
- regexorcist 6mo agoSure, I have a 64G MBP with an M1 Ultra. The best model for me by far has been the 35B A3B, in particular the 8Q_KL unslouth variant. The dense model works but it's much slower, and I don't really see a difference in quality with a good harness.
- koyote 6mo agoWhat do you use as a harness?
- bobim 6mo agoThis is also of interest!
- regexorcist 6mo agoI use oh-my-openagent. It does an incredible job at planning and executing by orchestrating subagents with different roles.
- hypfer 6mo agoQwen3.6-27B-UD-Q4_K_XL can run at 45t/s with 131k q8 context on an RTX 4090. That is pretty usable. You could get 65t/s or more with MTP, but only if you drop the context size, which I would advise against. Results are better with 256k context and a larger quant, however, that's not going to fit on the 4090 you already had lying around for playing cyberpunk 2077. The MoE models make me rather unhappy. Idk. They feel braindead to me, but YMMV.