5 ms·
Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start
by cogman10 4mo ago
Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them.
The UX is the same regardless the provider. You send in a prompt, it spits back an answer.
In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with. The transition is basically updating your current harnesses to know about the other models.
I think the comparison most apt is the rise of AMD. Sure, it never(?) achieved market dominance, but it did ultimately make a huge dent. And a big part of that was because AMD x86 was pretty close and pretty compatible with Intel x86 at a fraction of the cost.
- calebhwin 4mo agoHard disagree. Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel <> Libre but still not "cheap and easy".
- vmg12 4mo agoThis is just another example of the bitter lesson. In a year a model will come out that will make none of these model specific optimizations you made matter.
- pizlonator 4mo agoYeah But the point is that at any moment, there is friction in switching
- iknowstuff 4mo agoI don’t exactly see orgs lining up to switch (and train) their employees between claude desktop and codex and whatever copilot is doing. There’s probably some inertia to those harnesses/integrations on top of the llms themselves.
- mgambati 4mo agoThe inertia is legal and financial. People are paying Anthropic through AWS accounts because the simple reason of not dealing making new contract and legal agreements is enough of reason of the inertia. But, eventually, I’m quite sure that AWS will also provide open models with those contracts without any inertia. Copilot is already offering Kimi. My company has a deal with Devin and they provide new models all the time, and open models are becoming the most used ones by our internal metrics, especially because the company is very worried about cost.
- skeptic_ai 4mo agoAlso they pay for legal liability of code produced
- Yizahi 4mo agoMaybe for a fantasy of legal liability of output produced. I haven't heard of any LLM corpo being held liable for any output they generate. Even NYT lawsuit is going nowhere for 3 years in courts already, despite being the most grounded.
- skeptic_ai 4mo agoCopilot insures your company up to x if someone sues our company. I guess most companies do this. That’s why we can’t use anything else except copilot.
- zmgsabst 4mo agoAWS already supports Llama and GLM in its Bedrock service for hosted models. They’re much cheaper to run, eg, Llama 3.3 Instruct 70B is 5-10x cheaper than Sonnet 5. https://aws.amazon.com/bedrock/pricing/ https://aws.amazon.com/bedrock/pricing/ Say you have 20% of usecases that require the more expensive model — but in 80% you could just use Llama instead of Sonnet (eg, for basic queries of a document). That saves 80% of that 80%, or 65% of your total bill! That is the kind of “swap” that’s likely to occur in automated tooling as pricing pressure kicks in — “can you save 65% on our AI bill by switching Bedrock over in 80% of uses?”
- AussieWog93 4mo ago>switching out an LLM is both cheap an easy. Honestly, these days probably less friction switching out Redis or Elasticsearch (backend) than changing LLM provider (human facing). Fable is seriously good enough now to, in a 20k line project, take "replace Mongoengine with raw PyMongo" and not screw anything up.
- woeirua 4mo agoAgents will make all of these migrations trivial. I expect margin collapse across a lot of tech darlings.
- thedougd 4mo agoThis is the conversation I plan to have with Okta sales soon. Wait till you see how easy AI makes it to switch to Entra ID or anyone else. It’s tedium not even problem solving.
- jaxn 4mo agothough Okta is the first provider working on the enterprise mcp stuff.
- thedougd 4mo agoIt’s a new SKU. You’ll have to pay for that new type of app.
- itzprintz 4mo agoIf the price is the only incentive, I'd stay away from Entra ID
- vidarh 4mo agoIf the point was to move there'd be no point in having a conversation with Okta sales. The point is to get a discount.
- 4mo ago
- pants2 4mo agoIf you're developing on top of LLM APIs directly, this is definitely not true. There are differences in how context caching works, in what's available through native harnesses, the types of tools you're fine-tuned on (GPT uses apply_patch while Claude uses edit, with different formats), the API surface (Agents SDK, Responses API, Managed Agents), cost structures, and best-practice guidance all around. Not to mention the meta of account limits, billing, ZDR contracts, etc.
- tharkun__ 4mo agoExactly, as in, really, will they? Where and at what price, especially across an actual enterprise that needs to deploy them to lots of devs? There's much more than just the actual model. Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally with a bunch of open weight models. I can run super tiny models on my 8GB NVIDIA card. They all suck (I have to use <=~5GB models if I want "usable" ~250k context that doesn't need to use system RAM and CPU (which makes things super slow). I've also tried a GLM 4.7-flash, which even though it's super slow (in comparison) with ~250k context and it just doesn't cut it vs. the Claude Sonnet or Opus I get to use at work. All the while these are all touted as "totally usable, Claude/ChatGPT killer!" replacements. It's just not "there" with tool use or building software for that matter. Like, just a simple Claude "web search" fails with it. So I asked it to build itself its own "web search" functionality and it just couldn't. It made so many mistakes its just not funny any more. And it couldn't recover from them either. I retried a few times (as I didn't have python installed and it wanted to implement it using that - this happens to be new system - never mind other attempts). I spent as much time doing this (and failing) as I spent building an actual full feature at work last week w/ Sonnet. If it can't build itself a simple web search to .md file tool/skill, how am I supposed to trust this with actual coding? I'm used to being able to point Claude at our large code base and essentially work with it like a junior doing my bidding. Maybe 5.2 is a killer game changer vs. what I was able to try out (if slowly) but you really have to show me to convince me at this point. And not with synthetic benchmarks. In those, all of the models I tried are supposedly super awesome.
- Forgeties79 4mo ago> Unlike all your examples, switching out an LLM is both cheap an easy. For now
- andsoitis 4mo ago> So easy that every 3 months or so new models are released and people grab them and start using them Individuals perhaps, but not organizations.
- bag_boy 4mo agoSwitching an agent harness is more difficult, especially on the enterprise/teams level. Once your team gets settled with Claude teams, cowork, and the various plugins, it’s going to be a pain in the butt to switch.
- skissane 4mo agoThe irony is that Claude will help you migrate away from itself AI is possibly the first product in history that will eagerly help you replace it with one of its competitors.
- skeptic_ai 4mo agoWait till they will pop a warning: “using Claude to migrate away will suspend your account. “ Or even better just silently sabotage the migration so you can’t do it. Something we can definitively expect from Claude given past behavior
- danny_codes 4mo agoIs it? I switched to Kiro and it's essentially identical.. well a bit better because you get a better idea of what the harness is doing, but otherwise identical.
- bag_boy 4mo agoI should have clarified that it’s not difficult for technical workers to switch, but for your average knowledge worker…it’s not easy switching from Claude Cowork to Codex. I don’t think the switch from Cowork to open source is easy.
- Shorel 4mo agoI made my own. It's true I don't want to switch to another harness now. But switching models is just a command.
- walthamstow 4mo agoPlugins and skills are completely trivial to move and most work with any model. What is not trivial are Anthropic's new managed agents vendor lock-in offering.
- rhipitr 4mo agoSwitching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self host a larger model that requires beefier hardware to host, plus all the complexity to scale that out to a bit internal user-base
- KetoManx64 4mo agoMost of the AI companies have OpenAi compatible API's, so you just get a subscription from another provider and change the URL that your LLM Agent Harness uses to talk to the AI. I use OpenRoutet which lets you switch between providers (Anthropic, ChatGPT, Z-AI) whenever you want. Sometimes I'll have two different models from different providers evaluate each other's answers.
- onion2k 4mo agoUnlike all your examples, switching out an LLM is both cheap an easy. Rolling out AI access in a large business is still hard, especially if you're trying to do it safely e.g stopping people throwing all your company data including user PII into a chat for productivity reasons. It's more a staff training and guardrails issue than a choosing which LLM to use issue, but I imagine picking an open model like GLM would make it harder because the 'enterprise stuff' will be missing.