3 ms·

It's not a smell. Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and va

It's not a smell. Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it.

vjsrinivas 3mo ago
Its almost a given considering how fast this field moves. Also, what kind of workflow structure would someone have that a single specific model is the only one that would perform acceptably?
- samuelknight 3mo ago
  Programs are tightly shaped to their dependencies and software frequently breaks when bumping a dependency versions. That's why we have package managers that are carefully built around lock files. Models should be thought of like a dependency, not just an endpoint to a workflow.
askvictor 3mo ago
It's kind of the same problem with cloud in general (though that moves much slower). If you want to be sure to be in control, then host it yourself
imhoguy 3mo ago
This is how development looks like for many years now, constant rewrite on the horizon. I think LLM development hype surpassed Blockchain and JS frameworks craze of decade ago.
recursive 3mo ago
> Why should these developers rebuild a core piece of their stack every few months? That's what they signed up for when established a hard dependency on an subscription online-only LLM model.
- BoorishBears 3mo ago
  If they were on open weights, at some point the provider would deprecate it, probably with worse notice And self-hosting would probably have been more expensive unless they had massive volume: Deepseek V3.x would have been the comparable open weight model for the performance and isn't that cost effective until hosted across multiple nodes with large batch sizes
  - recursive 3mo ago
    Sounds like something you'd need to evaluate before adopting then.
palmotea 3mo ago
> Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it. Because fuck you, that's way. /s There's a massive amount of contempt from a lot of software businesses for their customers, and it's just getting worse with LLM models.
Capricorn2481 3mo ago
These are AI models designed to be non deterministic. If upgrading from GPT 5.5 > 5.6 breaks your entire workflow, you shouldn't be using AI to begin with

vjsrinivas 3mo ago

Its almost a given considering how fast this field moves. Also, what kind of workflow structure would someone have that a single specific model is the only one that would perform acceptably?

samuelknight 3mo ago
Programs are tightly shaped to their dependencies and software frequently breaks when bumping a dependency versions. That's why we have package managers that are carefully built around lock files. Models should be thought of like a dependency, not just an endpoint to a workflow.

askvictor 3mo ago

It's kind of the same problem with cloud in general (though that moves much slower). If you want to be sure to be in control, then host it yourself

imhoguy 3mo ago

This is how development looks like for many years now, constant rewrite on the horizon. I think LLM development hype surpassed Blockchain and JS frameworks craze of decade ago.

recursive 3mo ago

> Why should these developers rebuild a core piece of their stack every few months? That's what they signed up for when established a hard dependency on an subscription online-only LLM model.

BoorishBears 3mo ago
If they were on open weights, at some point the provider would deprecate it, probably with worse notice And self-hosting would probably have been more expensive unless they had massive volume: Deepseek V3.x would have been the comparable open weight model for the performance and isn't that cost effective until hosted across multiple nodes with large batch sizes
- recursive 3mo ago
  Sounds like something you'd need to evaluate before adopting then.

palmotea 3mo ago

> Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it. Because fuck you, that's way. /s There's a massive amount of contempt from a lot of software businesses for their customers, and it's just getting worse with LLM models.

Capricorn2481 3mo ago

These are AI models designed to be non deterministic. If upgrading from GPT 5.5 > 5.6 breaks your entire workflow, you shouldn't be using AI to begin with