5 ms·

The cost to hire a human is highly predictable. The cost of AI isn't. I, as a human, need food and shelter, which puts a ceiling to my bargaining power. I can't

The cost to hire a human is highly predictable. The cost of AI isn't. I, as a human, need food and shelter, which puts a ceiling to my bargaining power. I can't withdraw my labour indefinitely. The power dynamics are also vastly against me. I represent a fraction of my employer's labour, but my employer represents 100% of my income. That dynamic is totally inverted with AI. You are a rounding error on their revenue sheet, they have a monopoly on your work throughput. How do you budget an workforce that could turn 20% more expensive overnight?

bornfreddy 6mo ago
By continuously testing competitors and local LLMs? The reason for rising prices is that they (Anthropic) probably realized that they have reached a ceiling of what LLMs are capable of, and while it's a lot, it is still not a big moat and it's definitely not intelligence.
- noosphr 6mo ago
  Anything but the simplest tooling is not transferable between model generations, let alone completely different families.
  - skissane 6mo ago
    > Anything but the simplest tooling is not transferable between model generations, let alone completely different families. It is transferable-yes, you will get issues if you take prompts and workflows tuned for one model and send them to another unchanged. But, most of the time, fixing it is just tinkering with some prompt templates People port solutions between models all the time. It takes some work, but the amount of work involved is tractable Plus: this is absolutely the kind of task a coding agent can accelerate The biggest risk is if your solution is at the frontier of capability, and a competing model (even another frontier model) just can’t do it. But a lot of use cases, that isn’t the case. And even if that is the case today, decent odds in a few more months it won’t be
    - jeffreygoesto 6mo ago
      Ha. Sounds a lot like the one 10x vs. predictable mediocre guys with a scaffolding of processes. Aim high and hit or miss or try to grind predictably and continuously. Same with humans and depends on the loss you can afford.
    - adi_kurian 6mo ago
      Yep. My approach has been, if I can’t reliably get something to 90+% with a flash / nano / haiku, then it’s not viable for any accuracy critical work. (I don’t know of or have the luck of having any other work.) Starting out with the pro / opus for any production classification work has always been a trick.
  - throwaway041207 6mo ago
    If you're talking about APIs and SDKs, whether direct API calls or driving tools like Claude code or codex with human out of the loop, I think that's actually fairly straightforward to switch between the various tools. If you're talking about output quality, then yeah, that's not as easy. But for product outputs (building a customer service agent or something like that), having a well-designed eval harness and doing testing and iteration can get you some degree of convergence between the models of similar generations. Coding is similar (iterate, measure), but less easy to eval.
  - vidarh 6mo ago
    It's not that hard to make it generic. It does take a little work, but really it boils down to figuring out how to make things work with the "dumbest" model in your set.
  - pizza 6mo ago
    For most tasks, at some future date, isn't there going to be some ambient baseline of capabilities you can get per $/tok, starting at ~0 for OSS models, such that eventually all tooling gets trivially transferable?
- NBJack 6mo ago
  Note that it is very likely this market can't sustain this level of competition for long. We are all still chasing the carrot of AGI, while hardware costs skyrocket.
zer00eyz 6mo ago
> The cost of AI isn't. This is why there are a ton of corps running the open source models in house... Known costs, known performance, upgrade as you see fit. The consumer backlash against 4o was noted by a few orgs, and they saw the writing on the wall... they didnt want to develop against a platform built on quicksand (see openweb, apps on Facebook and a host of other examples). There are people out there making smart AI business decisions, to have control over performance and costs.
alex_sf 6mo ago
The same way companies already deal with any cost.
kamma4434 6mo ago
> How do you budget an workforce that could turn 20% more expensive overnight? Like, say, oil or DRAMs?
- pmontra 6mo ago
  Exactly. Big headaches. It doesn't happen to the salaries of the employees of the companies affected by those price hikes. That's the point.
boredatoms 6mo ago
Its countered by competitors for inference. You could locally host a model and have your cost be fixed by your infra costs

bornfreddy 6mo ago

By continuously testing competitors and local LLMs? The reason for rising prices is that they (Anthropic) probably realized that they have reached a ceiling of what LLMs are capable of, and while it's a lot, it is still not a big moat and it's definitely not intelligence.

noosphr 6mo ago
Anything but the simplest tooling is not transferable between model generations, let alone completely different families.
- skissane 6mo ago
  > Anything but the simplest tooling is not transferable between model generations, let alone completely different families. It is transferable-yes, you will get issues if you take prompts and workflows tuned for one model and send them to another unchanged. But, most of the time, fixing it is just tinkering with some prompt templates People port solutions between models all the time. It takes some work, but the amount of work involved is tractable Plus: this is absolutely the kind of task a coding agent can accelerate The biggest risk is if your solution is at the frontier of capability, and a competing model (even another frontier model) just can’t do it. But a lot of use cases, that isn’t the case. And even if that is the case today, decent odds in a few more months it won’t be
  - jeffreygoesto 6mo ago
    Ha. Sounds a lot like the one 10x vs. predictable mediocre guys with a scaffolding of processes. Aim high and hit or miss or try to grind predictably and continuously. Same with humans and depends on the loss you can afford.
  - adi_kurian 6mo ago
    Yep. My approach has been, if I can’t reliably get something to 90+% with a flash / nano / haiku, then it’s not viable for any accuracy critical work. (I don’t know of or have the luck of having any other work.) Starting out with the pro / opus for any production classification work has always been a trick.
- throwaway041207 6mo ago
  If you're talking about APIs and SDKs, whether direct API calls or driving tools like Claude code or codex with human out of the loop, I think that's actually fairly straightforward to switch between the various tools. If you're talking about output quality, then yeah, that's not as easy. But for product outputs (building a customer service agent or something like that), having a well-designed eval harness and doing testing and iteration can get you some degree of convergence between the models of similar generations. Coding is similar (iterate, measure), but less easy to eval.
- vidarh 6mo ago
  It's not that hard to make it generic. It does take a little work, but really it boils down to figuring out how to make things work with the "dumbest" model in your set.
- pizza 6mo ago
  For most tasks, at some future date, isn't there going to be some ambient baseline of capabilities you can get per $/tok, starting at ~0 for OSS models, such that eventually all tooling gets trivially transferable?
NBJack 6mo ago
Note that it is very likely this market can't sustain this level of competition for long. We are all still chasing the carrot of AGI, while hardware costs skyrocket.

zer00eyz 6mo ago

> The cost of AI isn't. This is why there are a ton of corps running the open source models in house... Known costs, known performance, upgrade as you see fit. The consumer backlash against 4o was noted by a few orgs, and they saw the writing on the wall... they didnt want to develop against a platform built on quicksand (see openweb, apps on Facebook and a host of other examples). There are people out there making smart AI business decisions, to have control over performance and costs.

alex_sf 6mo ago

The same way companies already deal with any cost.

kamma4434 6mo ago

> How do you budget an workforce that could turn 20% more expensive overnight? Like, say, oil or DRAMs?

pmontra 6mo ago
Exactly. Big headaches. It doesn't happen to the salaries of the employees of the companies affected by those price hikes. That's the point.

boredatoms 6mo ago

Its countered by competitors for inference. You could locally host a model and have your cost be fixed by your infra costs