40 ms·

> Migrating to a different LLM is an afternoon's work at most, not nearly the complexity of porting an app between OS' or creating a robust hardware driver mode

> Migrating to a different LLM is an afternoon's work at most, not nearly the complexity of porting an app between OS' or creating a robust hardware driver model. I question this. Each vendor's offering has its own peculiar prompt quirks, does it not? Theoretically, switching RDBMS vendors (say Ora to Ingress) was also "an afernoon's work" but it never happened. The minutia is sticky with these sort of almost-but-not 'universal' interfaces.

mgh95 1y ago
> I question this. Each vendor's offering has its own peculiar prompt quirks, does it not? Theoretically, switching RDBMS vendors (say oracle to postgres) was also "an afernoon's work" but it never happened. The minutia is sticky with these sort of almost-but-not 'universal' interfaces. The bigger problem is that there was never a way to move data between oracle->postgres in pure data form (i.e. point pgsql at your oracle folder and it "just works"). Migration is always a pain, and thus there is a substantial degree of stickiness, due to the cost of moving databases both in terms of risk and effort. In contrast, vendors [1] are literally offering third party LLMS (such as claude) in addition to their own and offering one-click switching. This means users can try and if they desire switch with little friction. [1] https://blog.jetbrains.com/ai/2025/09/introducing-claude-agent-in-jetbrains-ides/ https://blog.jetbrains.com/ai/2025/09/introducing-claude-age...
- gretch 1y ago
  The current LLMs support data export/import inherently because the interface is pure text. All one needs to do is say something like “tell me all of personalization factors you have on me” and then just copy and paste that into the next LLM with “here’s stuff you should know about how to personalize output for me”
  - deleted 1y ago
    [deleted]
- RussianCow 1y ago
  Moreover, it's trivial to run several LLMs side-by-side for a while and measure the success of each, then migrate to the one that performs the best. And you can even migrate in-progress chats since all the context is passed on each call anyway.
iamflimflam1 1y ago
If you are doing things “properly” then you have good evals that let you test the behaviour of different LLMs and see if they work for your problem. The vendors have all standardised on OpenAIs API surface - you can use OpenAIs SDK with a number of providers - so switching is very easy. There are also quite a few services that offer this as a service. The real test is does a different LLM work - hence the need to evals to check.
kllrnohj 1y ago
> The minutia is sticky with these sort of almost-but-not 'universal' interfaces. True, but that's not really applicable here since LLMs themselves are not stable, and are certainly not stable within a vendors own product line. Like imagine if every time Oracle shipped a new version it was significantly behaviorally inconsistent with the previous one. Upgrading within a vendor and switching vendors ends up being the same task. So you quickly solidify on either 1) never upgrading, although with these being cloud services that's not necessarily feasible, and since LLMs are far from a local maxima in quality that'd quickly leave your stack obsolete or 2) being forced to be robust, which makes it easy to migrate to other vendors
jgalt212 1y ago
If you took any of current top 3 models from me, I would not miss the deleted one in the least. I run almost every non-trivial prompt through multiple models anyway.
BeetleB 1y ago
> Each vendor's offering has its own peculiar prompt quirks, does it not? That applies even when you switch models within a vendor, though.
impossiblefork 1y ago
It's reasonable to question it, but there's a fun Chinese paper (https://arxiv.org/abs/2507.15855 https://arxiv.org/abs/2507.15855) where they attempt to create a system of prompts that can be used with commercial LLMs to solve hard maths problems. It turns out that they can use the same prompt system for all of them, with no changes and still solve 5/6 IMO problems. I think this is possibly iffy, since people might have updated the models etc., but it's pretty obvious that this kind of thing is how OpenAI are doing their multi-stage thinking thing for maths internally. Consequently if prompt systems are this transferable for these hard problems, why wouldn't both they and individual prompts, be highly transferable in general?
psychoslave 1y ago
No, changing rdbms is a totally different challenge. They provide predictible reproducible output which are very sensitive and brittle to the slightest minor change in the input, while with LLM you expect the exact opposite. Changing the LLM backend in some IDE is as complicated as selecting an option in a dropbox for those who integrate such a feature. They are other scenarios where it might be a bit more complicated to transition of course, but that's it.

mgh95 1y ago

> I question this. Each vendor's offering has its own peculiar prompt quirks, does it not? Theoretically, switching RDBMS vendors (say oracle to postgres) was also "an afernoon's work" but it never happened. The minutia is sticky with these sort of almost-but-not 'universal' interfaces. The bigger problem is that there was never a way to move data between oracle->postgres in pure data form (i.e. point pgsql at your oracle folder and it "just works"). Migration is always a pain, and thus there is a substantial degree of stickiness, due to the cost of moving databases both in terms of risk and effort. In contrast, vendors [1] are literally offering third party LLMS (such as claude) in addition to their own and offering one-click switching. This means users can try and if they desire switch with little friction. [1] https://blog.jetbrains.com/ai/2025/09/introducing-claude-agent-in-jetbrains-ides/ https://blog.jetbrains.com/ai/2025/09/introducing-claude-age...

gretch 1y ago
The current LLMs support data export/import inherently because the interface is pure text. All one needs to do is say something like “tell me all of personalization factors you have on me” and then just copy and paste that into the next LLM with “here’s stuff you should know about how to personalize output for me”
- deleted 1y ago
  [deleted]
RussianCow 1y ago
Moreover, it's trivial to run several LLMs side-by-side for a while and measure the success of each, then migrate to the one that performs the best. And you can even migrate in-progress chats since all the context is passed on each call anyway.

iamflimflam1 1y ago

If you are doing things “properly” then you have good evals that let you test the behaviour of different LLMs and see if they work for your problem. The vendors have all standardised on OpenAIs API surface - you can use OpenAIs SDK with a number of providers - so switching is very easy. There are also quite a few services that offer this as a service. The real test is does a different LLM work - hence the need to evals to check.

kllrnohj 1y ago

> The minutia is sticky with these sort of almost-but-not 'universal' interfaces. True, but that's not really applicable here since LLMs themselves are not stable, and are certainly not stable within a vendors own product line. Like imagine if every time Oracle shipped a new version it was significantly behaviorally inconsistent with the previous one. Upgrading within a vendor and switching vendors ends up being the same task. So you quickly solidify on either 1) never upgrading, although with these being cloud services that's not necessarily feasible, and since LLMs are far from a local maxima in quality that'd quickly leave your stack obsolete or 2) being forced to be robust, which makes it easy to migrate to other vendors

jgalt212 1y ago

If you took any of current top 3 models from me, I would not miss the deleted one in the least. I run almost every non-trivial prompt through multiple models anyway.

BeetleB 1y ago

> Each vendor's offering has its own peculiar prompt quirks, does it not? That applies even when you switch models within a vendor, though.

impossiblefork 1y ago

It's reasonable to question it, but there's a fun Chinese paper (https://arxiv.org/abs/2507.15855 https://arxiv.org/abs/2507.15855) where they attempt to create a system of prompts that can be used with commercial LLMs to solve hard maths problems. It turns out that they can use the same prompt system for all of them, with no changes and still solve 5/6 IMO problems. I think this is possibly iffy, since people might have updated the models etc., but it's pretty obvious that this kind of thing is how OpenAI are doing their multi-stage thinking thing for maths internally. Consequently if prompt systems are this transferable for these hard problems, why wouldn't both they and individual prompts, be highly transferable in general?

psychoslave 1y ago

No, changing rdbms is a totally different challenge. They provide predictible reproducible output which are very sensitive and brittle to the slightest minor change in the input, while with LLM you expect the exact opposite. Changing the LLM backend in some IDE is as complicated as selecting an option in a dropbox for those who integrate such a feature. They are other scenarios where it might be a bit more complicated to transition of course, but that's it.