3 ms·
> The damage is done. You cannot build a business critical function on top of American SOTA frontier model. Especially not with the current crew in charge. The
by Aurornis 4mo ago
> The damage is done. You cannot build a business critical function on top of American SOTA frontier model. Especially not with the current crew in charge.
The switching costs of changing LLM providers is as low as it gets. All the individuals and startups I know try different models all of the time, even down to the level of choosing which provider to use based on the task. Bigger companies move slower but only because they have lawyers and teams negotiating contracts, not because there is a technical reason that it's hard to switch.
Companies have dealt with supply chain unpredictability by having multiple providers and switching between them since forever. It's infinitely easier to switch LLM providers than it is to deal with physical supply chain uncertainty.
- miki123211 4mo agoExactly! Even if you won't be able to use some model tomorrow, you can still make money by using it today! And in the age of limited compute, spiky workloads and constant outages, building a mechanism to fallback to a weaker model when your primary choice isn't available is smart anyway.
- PeterStuer 4mo agoFor real production I find the switching cost is not as trivial as you portray. Even going to a new model version in the same model family, say GPT-4o to GPT-5.2, a transition I just finished on a not too complicated application, requires extensive retesting and tweaking of prompts, guardrails and parameters.
- anonzzzies 4mo agoMaybe OP meant switching in a coding harness way? Not an application using AI? I had similar issues like you in the latter case, but in the former it's trivial.
- sshine 4mo agoI second this; even switching between minor versions of a model, you need to adjust prompts: the new model is better by implying a bunch of things that, when included in the prompt, will overdo that thing. Assessing quality of output is often not trivial, either. Typically, problems that are solved by offloading something to an LLM are super subjective, and customers “feel” something is different is vulnerable. We try to quantify output differences by many different similarity metrics. But a lot of energy goes into subjectively evaluating if something still works.
- Aurornis 4mo agoWe’re talking about SOTA models like Fable, though. If you’ve got a product where the budget allows for Fable level token costs, I doubt you wouldn’t have the budget to run your evals again on a cheaper model if Fable was unavailable. I mean it wouldn’t even take that much token volume to turn it into a money saving proposition to do the engineering work to switch to a cheaper model. Fable is primarily used for human in the loop tasks like coding or office work, not in some backend app unless the company has money to burn and doesn’t care about anything other than using the best model available at the time.
- jcims 4mo agoVendor diversity is a longstanding risk management principle. For it to work you need to invest in it as you build, not when the rug is pulled.
- PeterStuer 4mo ago100% agreed, and I do experiment, but it is atm just a question of where to put the effort first.
- 4mo ago
- throwaw12 4mo ago> The switching costs of changing LLM providers is as low as it gets Not trivial, you would need to do lots of evals and prompt tuning when you switch models. imagine what happens when you optimize your agent skills to the current model, and new model starts breaking. you would need to have versioning for your skills, serving different skills based on the model while you do A/B testing
- alfiedotwtf 4mo ago> Not trivial, you would need to do lots of evals and prompt tuning when you switch models. Couldn’t we just train smaller models to “translate” what the harness user wants to what the worker model expects? I mean, if models understand caveman, it seems like just a small stretch
- pornel 4mo agoIt's not switching costs, but trust. There's no congress. There's no policy (they've been making noises about not allowing AI regulation and now they're not-regulating it like a child paying with an on/off switch). The law is whatever Dear Leader's mood is today. It overrides any contract you sign with private companies, and they roll over and take it, because that's how oligarchies work.