3 ms·

We need more models that optimize for coding and that can be cheaper than frontier models, like what SWE 1.7 and composer 2.5 are trying to do. I don't think th

We need more models that optimize for coding and that can be cheaper than frontier models, like what SWE 1.7 and composer 2.5 are trying to do. I don't think there's an effort to make something GLM-5.2 level but focused only on coding.

gsibble 3mo ago
And make it run locally!
UncleOxidant 3mo ago
Qwen was doing something like this with their coder models. But alas, they seem not to be releasing those anymore. Last one was Qwen3-coder-next.
- yousif_123123 3mo ago
  Its crazy that OpenAI and Anthropic themselves aren't doing that. No attempts at reducing inference cost for code as far as I know from them.
  - andai 3mo ago
    OpenAI do have codex models, which are half the price. I haven't used them enough to comment on the quality though. I remember them saying a few years ago that, they didn't think it was worth specializing models for code, because their general purpose models kept beating them. I guess they changed their mind? Since they did start making codex models again.
    - arcanemachiner 3mo ago
      They also stopped making them again.
  - andriy_koval 3mo ago
    My speculation is that frontier models are MoE, and they just have some number of experts for coding.
- gsibble 3mo ago
  I use this model. It's pretty good but not Opus 4.8 or Fable levels obviously. I'm really hoping we get more models like it (and better) soon. I run it locally and it's great that way.
  - UncleOxidant 3mo ago
    Qwen3-coder-next is very usable. But I don't think it's as good as Qwen3.6-27B (though it does run faster on my hardware). It would be great if we could get a Qwen3.7-coder, but I'm not going to hold my breath.
jstummbillig 3mo ago
Defining what "coding" means now, and how quickly we fall off the capability cliff seems increasingly important. Today my "coding" sessions often enough begin with real life problems, where I discuss domain or inter-domain things, ranging from business, economics, psychology, etc. Being able to do all of that with one model is something I am willing to pay a premium for. Of course not having to pay the premium, because the routing is smart or whatever, would be great. I just don't want to have to think about it.
- andriy_koval 3mo ago
  > Today my "coding" sessions often enough begin with real life problems intuition is that your sessions consists of 10% of domain related reasoning, and 90% of code plumbing. Those 90% could be moved to cheap and efficient specialized and focused model.
  - jstummbillig 3mo ago
    Possibly! It's just hard to reason about from the outside. When does the model benefit from all the ambient knowledge? Idk. Regardless, it's fairly obvious to me that none of what I do now will require "frontier models" for much longer. Models are getting better more quickly than my problems are getting harder.
  - RussianCow 3mo ago
    But that 10% is the most important part! Getting the plumbing wrong means you might have bugs or your code is brittle. Getting the domain-specific business logic wrong means your product doesn't fundamentally solve the correct problem.
    - andriy_koval 3mo ago
      [dead]
- tonyhart7 3mo ago
  but that is not model problem most agentic coding app can use powerful model for planning/reasoning then use "budget" model to do ground work
euleriancon 3mo ago
This isn't as easy as it sounds. Every ML model is struggling to balance between generalization and test performance. Taking a good model like GLM5.2 and just fine tuning it on coding can decrease real world performance due to mechanics like catastrophic forgetting. There is also other interesting behaviors were training on a broad training set can improve coding performance because there is positive transfer. There is 100% an effort to make solid coding focused models, but it is very hard to do that without including capabilities across a broad set of adjacent tasks.
- alansaber 3mo ago
  As you say- LLMs are fundamentally good because of their generalism. Distillation, ablation, ft all tend to be hacky and in some way hurt the model

gsibble 3mo ago

And make it run locally!

UncleOxidant 3mo ago

Qwen was doing something like this with their coder models. But alas, they seem not to be releasing those anymore. Last one was Qwen3-coder-next.

yousif_123123 3mo ago
Its crazy that OpenAI and Anthropic themselves aren't doing that. No attempts at reducing inference cost for code as far as I know from them.
- andai 3mo ago
  OpenAI do have codex models, which are half the price. I haven't used them enough to comment on the quality though. I remember them saying a few years ago that, they didn't think it was worth specializing models for code, because their general purpose models kept beating them. I guess they changed their mind? Since they did start making codex models again.
  - arcanemachiner 3mo ago
    They also stopped making them again.
- andriy_koval 3mo ago
  My speculation is that frontier models are MoE, and they just have some number of experts for coding.
gsibble 3mo ago
I use this model. It's pretty good but not Opus 4.8 or Fable levels obviously. I'm really hoping we get more models like it (and better) soon. I run it locally and it's great that way.
- UncleOxidant 3mo ago
  Qwen3-coder-next is very usable. But I don't think it's as good as Qwen3.6-27B (though it does run faster on my hardware). It would be great if we could get a Qwen3.7-coder, but I'm not going to hold my breath.

jstummbillig 3mo ago

Defining what "coding" means now, and how quickly we fall off the capability cliff seems increasingly important. Today my "coding" sessions often enough begin with real life problems, where I discuss domain or inter-domain things, ranging from business, economics, psychology, etc. Being able to do all of that with one model is something I am willing to pay a premium for. Of course not having to pay the premium, because the routing is smart or whatever, would be great. I just don't want to have to think about it.

andriy_koval 3mo ago
> Today my "coding" sessions often enough begin with real life problems intuition is that your sessions consists of 10% of domain related reasoning, and 90% of code plumbing. Those 90% could be moved to cheap and efficient specialized and focused model.
- jstummbillig 3mo ago
  Possibly! It's just hard to reason about from the outside. When does the model benefit from all the ambient knowledge? Idk. Regardless, it's fairly obvious to me that none of what I do now will require "frontier models" for much longer. Models are getting better more quickly than my problems are getting harder.
- RussianCow 3mo ago
  But that 10% is the most important part! Getting the plumbing wrong means you might have bugs or your code is brittle. Getting the domain-specific business logic wrong means your product doesn't fundamentally solve the correct problem.
  - andriy_koval 3mo ago
    [dead]
tonyhart7 3mo ago
but that is not model problem most agentic coding app can use powerful model for planning/reasoning then use "budget" model to do ground work

euleriancon 3mo ago

This isn't as easy as it sounds. Every ML model is struggling to balance between generalization and test performance. Taking a good model like GLM5.2 and just fine tuning it on coding can decrease real world performance due to mechanics like catastrophic forgetting. There is also other interesting behaviors were training on a broad training set can improve coding performance because there is positive transfer. There is 100% an effort to make solid coding focused models, but it is very hard to do that without including capabilities across a broad set of adjacent tasks.

alansaber 3mo ago
As you say- LLMs are fundamentally good because of their generalism. Distillation, ablation, ft all tend to be hacky and in some way hurt the model