3 ms·
I have really been trying to get local models to work. I have tried different harnesses, tooling, skills, prompts, etc. But when I compare claude code with anth
by ecshafer 5mo ago
I have really been trying to get local models to work. I have tried different harnesses, tooling, skills, prompts, etc. But when I compare claude code with anthropic models or codex with gpt 5.5, vs qwen, glm or gemma and the same harnesses, the frontier models come out massively ahead. I am at the point where I just don't see the point of the non-frontier models, they waste more time than they save.
- bee_rider 5mo agoThe hosted frontier models are massively subsidized, right? I think the point of local non-frontier models is just learning at this point, so you’ll be skilled if/when the market starts comparing the actual price of the two different models.
- datadrivenangel 5mo agoFor agentic coding I 100% agree with you, it's worse and slower and more expensive for LARGE coding with local models. Narrow coding (like writing a specific function) is slow but viable. Regular LLM chat usage on high-end consumer hardware is competitive except on cost though. 0 0 - https://www.williamangel.net/blog/2026/05/17/offline-llm-energy-use.html https://www.williamangel.net/blog/2026/05/17/offline-llm-ene...
- henry2023 5mo agolocal models are 3 to 6 months behind SOTA models with the huge benefit of not needing to send all your IP to a shady third party. If inference cost comes down (as it has been for the last few years) you’ll be able to run today’s SOTA in your laptop by the end of the year.
- ghrl 5mo agoI would say that is highly unlikely if by SOTA models you are not just referring to coding benchmarks but more general purpose ability and domain-specific knowledge. For example Kimi 2.6, which is comparable to Opus 4.6, is roughly 500+GB large, and I don't see how that would run on consumer hardware anytime soon. Besides, this is not just about the technical feasibility, but also economically not viable whatsoever. Why should consumer laptops be capable of running such models, when they would be massively underutilized most of the time, when inference providers can produce the same results faster, cheaper and a lot more viable economically?
- henry2023 5mo agoBecause privacy has perceived value.
- sourcecodeplz 5mo agoIt runs right now on 512gb RAM Macs and PCs.
- Our_Benefactors 5mo agoIt runs like shit though in terms of tokens/second and still has a reduced context window. Vs a single claude prompt can easily get into 300k tokens without breaking a sweat. I want local AI to be a thing but the hardware isn’t here yet, because the only options are a Mac Studio or DGX machines strapped together. RAM prices needs to crash before local AI has a chance at actually competing.
- zozbot234 5mo agoThe more recent Chinese models are no longer heavily limited by context size. It can easily fit in RAM on a prosumer laptop. (You can also use swap space to extemd that, since context is only written to once per inference, thus a relatively mild wear-and-tear concern.)
- Our_Benefactors 5mo agoClaude has 1M context window for the enterprise. 128k feels like a toy in comparison.
- sourcecodeplz 5mo agoDeepseek pro/flash both have 1m.
- ATMLOTTOBEER 5mo agoYou’re right, and it feels like these people saying otherwise either don’t use these tools professionally (and therefore can’t tell a difference between local/cloud models) or literally just haven’t tried running local models As soon as I can buy hardware for less than 5k that runs an opus 4.6+/5.5 model locally I will do it instantly
- lurking_swe 5mo agothe bigger issue is context windows. HUGE difference there.
- fg137 5mo ago"shady third party" If Claude hosted on AWS bedrock is not considered trustworthy, I have some bad news for you.
- henry2023 5mo agoAnthropic illegally downloaded virtually all copyrighted material in the world to train their models. What makes you think they will have even a little consideration for your IP?
- linkregister 5mo ago*tortiously downloaded
- fg137 5mo agoLet's say as a matter of fact that Anthropic seizes every opportunity to steal IP. How is that going to work on Bedrock, when they don't even manage the infrastructure?
- joka88xj 5mo ago[dead]
- koonsolo 5mo agoI came to the same conclusion. For the amount that a query costs, using Opus all the time is the cheapest option.
- myzek 5mo agoFor now. The prices for using those will increase massively at some point
- eikenberry 5mo agoThe point is to not indenture yourself to a corporation whose motives do not align with your own.
- illusive4080 5mo agoTheir motives are to make the best product to compete in a very competitive market.
- eikenberry 5mo agoRight.. and their definition of a "best product" is theirs, not mine and probably not yours.
- illusive4080 5mo ago…the best model for agentic coding, is the top goal right now.
- eikenberry 5mo agoGood example. AFAIK they are still focusing on doing this via larger models which is a bad call. They are also focusing to hard on fully-agentic coding which, while useful to a limited extent, is not the best way to use AI for most non-trivial coding tasks.
- illusive4080 5mo agoYeah, fully agentic coding just takes too much knowledge out of the engineer. It’s a lazy man’s button to do work and yield average results. Fun for vibe coding and not reliable enough for enterprise where quality matters. Maybe their focus will pay off in reliability, maybe not.
- kaeluka 5mo agoThey're trying to make the product sticky while they're still subsidizing the subscription price. The plan is to raise prices when you're addicted.
- WaylandYang 5mo ago[flagged]
- steve-atx-7600 5mo agoSame. It pains me to hear quibbling about spending or reimbursing $200/month for codex or Claude plans . These are virtually inexhaustible for me as a software engineer and seem like a steal given the gains in efficiency. And, who wants to screw around with harnesses or define agent orchestration when Claude/codex are good at this and getting better every month.