4 ms·

> I think we are starting to see the squeeze from the big players. I’m not convinced that these price increases represent an attempt to squeeze more profit out

> I think we are starting to see the squeeze from the big players. I’m not convinced that these price increases represent an attempt to squeeze more profit out of a saturated market. To me they look an awful lot like people realising that the sheer compute cost associated with modern models makes the historical zero-marginal cost model of software impossible. API calls to LLM models have far more in common with making calls to EC2 or Lambda for compute, than they do a standard API calls for SaSS. A lot of early LLM based business models seemed to assume that the historical near zero-marginal cost of delivery for software would somehow apply to hosted LLM models which clearly isn’t the case. You mix that in with rising datacenter costs, driven by lack of available electricity infrastructure to meet their demands, plus everyone trying to grab as much LLM land as possible, which requires more datacenters, more faster. And the result is rapidly increasing base costs for compute. Which we’re now seeing reflected in LLM pricing. For me the thing that stands out about LLMs, is that their compute costs are easily 100-10000x greater per API call than a traditional SaSS API call. That fact alone should be enough for people to realise that the historically bottomless VC money that normal funds this stuff, isn’t quite a bottomless as it needs to be to meaningfully subsidise consumer pricing.

patapong 1y ago
Very insightful. I think the payment model would have worked out just fine if the state of the art was the optimization of GPT-4 class models to bring down the cost over time, which would have made the services profitable eventually. Instead, newer models are getting larger and more resource heavy through reasoning, meaning costs per request are going up instead of down.
- avianlyric 1y ago
  I think we’ll start seeing people focus on optimisation, we already see companies like Apple focus on it. LLM are still to new, and still advancing to quickly for optimisation to take place. It’s like we’re back in the MHz wars of old between CPU manufacturers. The goal is just more performance, regardless of cost, because it was clear that even in the consumer space, people wanted more performance. Then we hit a kind of plateau in last 10 years, where basic compute is so powerful that your average consumer is not longer upgrading every year for better performance. A 5 year old machine has enough performance for most people. Then the focus on energy efficiency kicked in, because people didn’t want faster computers, they wanted battery life and cheaper computers. No doubt we’ll see the same with LLM, possibly quite soon. Claude Sonnet 4 and similar class models have enough reasoning performance, that agentic systems can be quite reliable. Which means we hit the base level of “reasoning” performance needed, and we can extend that “performance” in domain specific ways by lightly customising the agentic framework, with no need to fine tuning. The elimination of fine tuning to build domain specific agents is a huge game changer. But it also means that putting together a 10x or 100x efficient model, with “reasoning” performance equivalent to current gen LLM would also be a huge game changer. It opens up the possibility to apply this tech into spaces that currently require either lots of specialists knowledge to fine tune an LLM, or a huge amount of on tap compute to allow the agents to take enough turns to slowly “reason” they’re way through problems. But a Claude Sonnet 4 that runs on a iPhone for example. That would make Apple’s complete failure to improve Siri look like a genius level move. Why bother with small incremental improvements using current tech, when waiting a few years, and just stuffing a full fat LLM and agent system into an iPhone will basically give you the ultimate Siri.
  - tom_m 1y ago
    Supply and demand says otherwise. I think costs are actually going to go up. Right now they are trying to make sure the AI you run on your phone can't be used for coding or other tasks. They want to keep it expensive. They have no incentive to make it more efficient. No need. The models aren't designed for efficiency either. There was a little scare with DeepSeek, but it turned out to be nothing. At the end of the day you need a lot of hardware and juice to run these things at scale. The costs aren't coming down any time soon. Nor is the impact to the environment unfortunately either.
    - willseth 1y ago
      Who is they? Even if Anthropic doesn’t have incentive to run their models on your phone (but really why wouldn’t they?), Apple, Google, and Samsung do. And DeepSeek was very much a something. It was a something that demonstrated the barrier to entry can be at least an order of magnitude lower than previously thought. LLM technology is easily commoditized, and moat building will be difficult. Supply and demand is doing just what you would expect.
  - bobbob27 1y ago
    I truly believe you're right on that. Apple seems to be the most visible example of a company using LLMs in a way that will make long-term sense. The most logical future of AI is strictly on-device, capable enough to understand the user and making sense of the algorithmic-anarchy that is our phone. THAT is something exciting and practical, that doesn't require magical-thinking to desire.
barrell 1y ago
I don’t see any difference between your perception of events and “the squeeze,” except yours has more details. I think you and GP actually agree (Edited for tone)

patapong 1y ago

Very insightful. I think the payment model would have worked out just fine if the state of the art was the optimization of GPT-4 class models to bring down the cost over time, which would have made the services profitable eventually. Instead, newer models are getting larger and more resource heavy through reasoning, meaning costs per request are going up instead of down.

avianlyric 1y ago
I think we’ll start seeing people focus on optimisation, we already see companies like Apple focus on it. LLM are still to new, and still advancing to quickly for optimisation to take place. It’s like we’re back in the MHz wars of old between CPU manufacturers. The goal is just more performance, regardless of cost, because it was clear that even in the consumer space, people wanted more performance. Then we hit a kind of plateau in last 10 years, where basic compute is so powerful that your average consumer is not longer upgrading every year for better performance. A 5 year old machine has enough performance for most people. Then the focus on energy efficiency kicked in, because people didn’t want faster computers, they wanted battery life and cheaper computers. No doubt we’ll see the same with LLM, possibly quite soon. Claude Sonnet 4 and similar class models have enough reasoning performance, that agentic systems can be quite reliable. Which means we hit the base level of “reasoning” performance needed, and we can extend that “performance” in domain specific ways by lightly customising the agentic framework, with no need to fine tuning. The elimination of fine tuning to build domain specific agents is a huge game changer. But it also means that putting together a 10x or 100x efficient model, with “reasoning” performance equivalent to current gen LLM would also be a huge game changer. It opens up the possibility to apply this tech into spaces that currently require either lots of specialists knowledge to fine tune an LLM, or a huge amount of on tap compute to allow the agents to take enough turns to slowly “reason” they’re way through problems. But a Claude Sonnet 4 that runs on a iPhone for example. That would make Apple’s complete failure to improve Siri look like a genius level move. Why bother with small incremental improvements using current tech, when waiting a few years, and just stuffing a full fat LLM and agent system into an iPhone will basically give you the ultimate Siri.
- tom_m 1y ago
  Supply and demand says otherwise. I think costs are actually going to go up. Right now they are trying to make sure the AI you run on your phone can't be used for coding or other tasks. They want to keep it expensive. They have no incentive to make it more efficient. No need. The models aren't designed for efficiency either. There was a little scare with DeepSeek, but it turned out to be nothing. At the end of the day you need a lot of hardware and juice to run these things at scale. The costs aren't coming down any time soon. Nor is the impact to the environment unfortunately either.
  - willseth 1y ago
    Who is they? Even if Anthropic doesn’t have incentive to run their models on your phone (but really why wouldn’t they?), Apple, Google, and Samsung do. And DeepSeek was very much a something. It was a something that demonstrated the barrier to entry can be at least an order of magnitude lower than previously thought. LLM technology is easily commoditized, and moat building will be difficult. Supply and demand is doing just what you would expect.
- bobbob27 1y ago
  I truly believe you're right on that. Apple seems to be the most visible example of a company using LLMs in a way that will make long-term sense. The most logical future of AI is strictly on-device, capable enough to understand the user and making sense of the algorithmic-anarchy that is our phone. THAT is something exciting and practical, that doesn't require magical-thinking to desire.

barrell 1y ago

I don’t see any difference between your perception of events and “the squeeze,” except yours has more details. I think you and GP actually agree (Edited for tone)