3 ms·

> capabilities have largely converged across foundation models over the last 18 months For reference, in March '25 the models du jour were Sonnet 3.7, gpt o4 a

> capabilities have largely converged across foundation models over the last 18 months For reference, in March '25 the models du jour were Sonnet 3.7, gpt o4 and gemini 2.5 pro. GPT5 was in august '25. It's been a while since we've heard the old "models have stagnated". Oh well.

tancop 27d ago
It's not "models have stagnated" but "models released at the same time are on the same level". Improvements are still real but the relative gaps between OpenAI, Anthropic, Meta, Grok, Gemini and open models are closer than ever. That doesn't mean progress is slowing down, it's just more widely distributed.
- alephnerd 27d ago
  This, and depending on the workflow and usecase, you don't necessarily need the latest and greatest with the right kind of harness engineering. Like everything in engineering, it's about tradeoffs and what works best for your specific problem.
  - porridgeraisin 23d ago
    There is also the aspect of model generations being robust under a particular ctx mgmt strategy/tailored harness. Only since this year are most models robust in this way. In many open models, the problem is that successive generations are not that robust yet. i.e if you have a working setup, next generation of the same model family will need way too much reworking. So it's difficult to upgrade the model. Gemini is stellar in this respect (probably because, and I suspect, their flash models are distillations of the same larger model). OpenAI/Anthropic are too, but higher cost of newly released models is quite the blocker. Deepseek is especially hard to upgrade. Kimi doesnt target this segment, so people that can post train it do so. Qwen and llama are the stellar open ones, incredibly stable, although llama has stopped receiving upgrades in a while. It is still used a lot though.
- NitpickLawyer 27d ago
  Ah, I see. I misunderstood then. The thing about "gains come from the harness" made me think about it in that way.

tancop 27d ago

It's not "models have stagnated" but "models released at the same time are on the same level". Improvements are still real but the relative gaps between OpenAI, Anthropic, Meta, Grok, Gemini and open models are closer than ever. That doesn't mean progress is slowing down, it's just more widely distributed.

alephnerd 27d ago
This, and depending on the workflow and usecase, you don't necessarily need the latest and greatest with the right kind of harness engineering. Like everything in engineering, it's about tradeoffs and what works best for your specific problem.
- porridgeraisin 23d ago
  There is also the aspect of model generations being robust under a particular ctx mgmt strategy/tailored harness. Only since this year are most models robust in this way. In many open models, the problem is that successive generations are not that robust yet. i.e if you have a working setup, next generation of the same model family will need way too much reworking. So it's difficult to upgrade the model. Gemini is stellar in this respect (probably because, and I suspect, their flash models are distillations of the same larger model). OpenAI/Anthropic are too, but higher cost of newly released models is quite the blocker. Deepseek is especially hard to upgrade. Kimi doesnt target this segment, so people that can post train it do so. Qwen and llama are the stellar open ones, incredibly stable, although llama has stopped receiving upgrades in a while. It is still used a lot though.
NitpickLawyer 27d ago
Ah, I see. I misunderstood then. The thing about "gains come from the harness" made me think about it in that way.