3 ms·
> I tried running a smaller model locally, and it's not usable for me. If you have the hardware, a MacBook Pro for Qwen 3.6 35B A3B and Gemma 4 26B A4B for exa
by ch_sm 2mo ago
> I tried running a smaller model locally, and it's not usable for me.
If you have the hardware, a MacBook Pro for Qwen 3.6 35B A3B and Gemma 4 26B A4B for example, they are absolutely usable, both in terms of speed and quality. Anecdotally, I can use Qwen for day-to-day coding tasks in TS and Go, without hickups.
- everyone 2mo agoYou let a hiccup slip through in your comment though.
- embedding-shape 2mo agoI'm unable to find a local model that comes close to the effectiveness of GPT models in Codex, and I have 96GB of VRAM available and tried every local model under the sun so far. Neither of those you mention I'd say are good enough for day to day software engineering for me, but I'm also really strict about code quality and iterate on what outputs agents give me a lot before I'm happy. With local models, this iteration cycle takes maybe 30 minutes for a single fix or feature, rather than 10 minutes with GPT+Codex, as there is so many corrections and iterations needed, although I will say that the speed I'm able to get locally makes it more fun that any of the remote models.
- rapind 1mo ago> although I will say that the speed I'm able to get locally makes it more fun that any of the remote models. This is becoming increasingly important to me. Super smart max reasoning frontier is fine if I leave it running overnight on some prepared set of clearly defined tasks, but when I want to work with the LLM, throughput really matters, and I'll go with a dumber model to get there. At some point though, it's fast enough and any speed gains beyond that just makes me the bottleneck. I also am seeing the smaller models gaining big strides lately, closing the gap on frontier models (still a decent sized gap though). I don't even run the small models like Qwen 3.8 27B locally. I just try them out in the cloud to see how they are progressing, and I'm definitely able to be productive.
- ch_sm 1mo ago> I'm unable to find a local model that comes close to the effectiveness of GPT models in Codex I agree with you there, the local models are not as capable as frontier remote LLMs. If you‘re used to letting fable run for an hour to do a novel or complex task, you‘re (probably?) not going to be happy with local LLMs. But, to me, for my daily work, they‘re still usable, and often surprisingly capable. YMMV
- gessha 2mo agoI’ve been experimenting with Qwen 3.8 27B and I believe I can totally use it as my main coding model provided I have the hardware for the full context. I don’t need my model to be opus level. I need it to do the tasks I want it to do without being an overprotective nanny.
- jatora 2mo agoNo, you cant. I challenge anyone who claims this to show me an actual project built only by SLM's and not using opus, sonnet, sol, or terra. Spoiler: you can't.