5 ms·

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "over-use" of best/priciest models based on total-or-percent tokens/dollars, etc. All of this comes with direct developer engagement & standardized management escalation for holding it wrong. To me this customer behavior does not smell like a product you can 10x the pricing on to get profitable. We have exited the exploration phase and now ROI matters.

piker 4mo ago
I.e., the demand for programming tokens turns out to be quite elastic.
- steveBK123 4mo ago
  I would imagine it only gets worse in the face of good-enough open/chinese/local models too right? Microsoft adding Deepseek support already as I recall? That is - for any definition of "they are behind X months" then eventually they get to the point Claude was in January when the world freaked out, but at 1/10th the cost. A lot of firms are going to mandate that is good enough for their developers.
  - sdesol 4mo ago
    > Microsoft adding Deepseek support I believe this hasn't been confirmed yet but I think it speaks to a bigger problem for the AI companies which is, if you give capable developers a good reasoning LLM, they can make it work like it was a really expensive model. I believe we are 100% at the stage of good enough for the vast majority of tech companines. Fable and others will be more valuable for non-traditional tech companies. I read somewhere that the chinese AI companies are sharing knowledge and it would not surprise me if the government is applying pressure by saying work together or else. If they work together, they can truly commoditize LLMs and with China ramping up hardware support for AI, I see the future being inference speed and hardware being the moat.
    - thewebguyd 4mo ago
      If hardware becomes the moat, the US frontier labs are screwed. We have AWS, Azure, GCP. All three have or are making inference silicon. LLMs become just another service in the public cloud's large service catalog, and open weight wins. Which makes sense to me. Selling a chatbot interface/model access to the general public was never going to be a viable long term play. You still need developers to wrap the models into specialized tools. Queue the Jobs quote "It's a feature, not a product."
      - sdesol 4mo ago
        The big thing is, the western world has moved so much of the manufacturing to China and think a lot of people will not forgive Samsung and others, so I can see China owning a good portion of the supply chain.
        johnvanommen 4mo ago
        > The big thing is, the western world has moved so much of the manufacturing to China I built my career on Solaris and it got rugpulled by Linux. That wasn’t because of software, it was because of hardware. Linux’s cost advantage existed because Sun hardware had huge margins, because their software was basically free. AI will probably be a repeat of this. Whoever can come up with the hardware solution that minimizes the cost per token will win. I believe the 5090 still holds this crown, but someone certainly knows better than I do.
        fragmede 4mo ago
        but not all tokens are equal and vertical integration is the name of the game. Solaris did not lose to Linux, it lost to the LAMP stack on commodity x86 hardware. without the "AMP" part, Linux would've been dead in the water.
        fluoridation 4mo ago
        The L part of LAMP is kinda irrelevant at the application level, though. From the point of view of running the software, SAMP or LAMP doesn't make any difference. So yes, Solaris lost the competition with Linux to be the first letter of the acronym.
  - CuriouslyC 4mo ago
    100%. There will be strict quotas on the expensive models and day to day work will be done on the cheap models that are "good enough" with escalation to the metered models when the cheaper options are spinning their wheels. Eventually the US frontier lab APIs will only get the most heavily triaged work that multiple tiers of cheaper Chinese open weight models have failed on. And of course the C-suite will have unlimited access to Mythos tier models, which they'll use to summarize reports, while passing down mandates to rank and file to increase usage of less expensive models.
  - verdverm 4mo ago
    Yup, we are in the process of getting access to US hosted Chinese models. I've been petitioning Google and our rep, we will see but I suspect they will cave eventually. Gemini sucks and if they don't sell what their customers want, we go shopping around.
    - bloppe 4mo ago
      What you want is already available on OpenRouter and a million other services, but sure, you can wait 18 months for it to be on GCP.
      - verdverm 4mo ago
        > We are in the process... OpenRouter charges an extra 5.5%, Fireworks does not, Google is separate, but I doubt it will take 18 months. They are already aware they are losing business. OpenRouter is the wrong abstraction for enterprise, we only need one model provider, not everyone in the world. Nor do we want to have to worry about failover going to providers we don't want.
  - michaelchisari 4mo ago
    I'm set up to use Qwen 3.6 locally if needed. It's solid, it does what I need, it runs on my laptop and it's free. But that's because I never got on the "run three dozen agents in a ralph loop" trend or other high-token usage methods. The way I use AI is discrete and targeted and it seems that's how it will be for everyone once the economics settle.
  - Gigachad 4mo ago
    Our org straight up turned off our AI access after the GitHub copilot price increase blew right through the budget. At least on a personal I feel like I’ve been getting the same amount of work done but I have to think harder rather than sitting back and prompting and waiting.
  - wookmaster 4mo ago
    My company banned Chinese models but I’ve got opencode setup with some open source coding models just in case. Luckily our laptops have some RAM (Mac)
- jayd16 4mo ago
  If folks won't pay a higher price, doesn't that mean it's inelastic?
  - steveBK123 4mo ago
    Elastic demand means buyers are highly sensitive; a price hike causes a massive drop in purchases. Inelastic demand means buyers aren’t very sensitive; they keep buying regardless of price
    - jayd16 4mo ago
      Ah alright I have it backwards then.
  - unholiness 4mo ago
    "Elastic" in economics happens to refers to how elastic the supply/demand is when the price changes (not vice versa, as you're describing). So e.g. an inelastic demand means the quantity demanded changes very little when the price doubles.
dranudin 4mo ago
I can second this. Our company and department was all-in on AI. And since the token-based pricing came in, we got an email from IT that tried to explain that most developers don't know how to choose models and that the cheap models should be good enough for most of our work ..
- verdverm 4mo ago
  Have they built an internal ai enablement team?
  - dranudin 4mo ago
    Yes :D
- daemonologist 4mo ago
  Ha, developers know perfectly well how to choose models - they know that the company is footing the bill and Opus will give them the best result and/or require the least clean-up work. (We've received similar guidance. Even better is that our provider, GitHub Copilot, does not provide usage information to individual users if there is no per-user budget configured. So we just fire our requests into the black box and at the end of the month when IT gets the bill we maybe get a talking-to if it's excessive.)
burningChrome 4mo ago
I can give you some additional anecdotal evidence to support your comment. I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because teams were building the same tools for what they thought was nothing. Then they signed contracts with Anthropic and Google because I would assume they saw the token usage was through the roof. One month later? They completely cut off access to everybody for both Claude and Gemini. If you wanted access? Suddenly it was several forms, along with several approvals and a rock solid business case why you needed it. And before you got to the forms? You were added to a waiting list that was thousands of people long. The entire company is now in damage control after trying to get the genie back in the bottle. I'm guessing someone saw how much we would be paying for the tokens we'd been using and decided to shut the party down so to speak.
- sdesol 4mo ago
  Was there at least performance gains to be measured?
  - burningChrome 4mo ago
    AFAIK nobody was collecting analytics. The one team I was working on had put out a goal of "30% more efficient" using AI tools. Its about as subject as you can get. We never got around to what exactly that meant before everything got shut down. Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would affirm or deny any of this had a positive impact. Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks.
    - stingraycharles 4mo ago
      > Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks. What do you mean placebo effect? They thought things were created with AI while they actually weren’t?
ofjcihen 4mo ago
I do a lot of client work for fortune 100’s. Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more. Obviously this won’t be on an individual basis but some kind of unit. Either way, with how much I see these companies cutting back I have no idea how the big AI companies are going to be profitable.
- energy123 4mo ago
  The AI companies will be profitable if they ship the goods. Make AGI and companies will pay for it.
  - fluoridation 4mo ago
    Not if it costs more than a human, or if it performs worse (be that in quality or in time). After all, humans have NGI and companies don't just hire anyone with a pulse.
  - oblio 4mo ago
    > Make AGI It's easy, just: Make economically viable cold fusion Make flying cars Make a perpetual motion machine Just because AGI is 3 simple letters doesn't implicitly mean we will live to see it.
  - N_Lens 4mo ago
    Your comment reminds me of when a friend was watching me play counter strike 1.6 and said “this game looks so easy, just point at the enemies’ heads and left click?”
    - Der_Einzige 4mo ago
      He was right. Twitch shooters take far less skill than shooters with higher TTK and it’s not even close. Pros at “the finals” are much higher IQ than CS or valorant pros. Similar dynamic to how league might as well be candy land compared to dota2s checkers and star crafts chess.
Gigachad 4mo ago
Our company went from “AI AI AI” to “GitHub Copilot has been suspended due to exceeding the budget” with this month’s price increase.
- mkozlows 4mo ago
  If your company was giving you Copilot, they were never that far on board the AI train anyway.
  - Gigachad 4mo ago
    Github Copilot is a different product than the Windows / Office one. Horrible naming I know but it's basically just a UI and router to pick whatever model you want.
    - wookmaster 4mo ago
      Microsoft names everything copilot anymore so everyone will continue to be confused
    - mkozlows 4mo ago
      No, I know what it is. But it's not as good as Claude Code or Codex. Harnesses matter for coding agents, it's not just the model. There's a reason Microsoft had to make its developers stop using Claude Code, and it's because they all found that Copilot sucked in comparison.
  - xdertz 4mo ago
    Github Copilot lets you use Anthropic models. This is just a way for enterprises to fold this into their existing Microsoft contract instead of negotiating with a new vendor.
qurren 4mo ago
What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.
- aryehof 4mo ago
  They may be getting 3 times the work done, but what work? Quality effective code that meets requirements? Or something else?
  - oblio 4mo ago
    > Quality effective code that meets requirements... ... and that also improves the company's financials. Otherwise it's all fluff.
- steveBK123 4mo ago
  Most companies are not seeing 2x or 3x of value produced from developers with AI is the reason
  - brandensilva 4mo ago
    Yeah the cost doesn't justify the value. Spending x3 on tokens is meaningless if they aren't seeing the profit side scale with it. Going faster on product features doesn't mean it translates to more money. In fact I'm not even sure some customers can handle the speed at which a team could move with feature delivery. So scaling horizontally in different markets seems like an advantage if a product is already mature. Which is exactly what Anthropic and Open AI are doing because they want to put their tentacles in everything.
    - steveBK123 4mo ago
      > Going faster on product features doesn't mean it translates to more money. Exactly this. Unless the new features directly drive new users or new revenue from existing users, for many products iterating 2x faster does not mean 2x ROI. Additionally, from anecdotes here, at work, and in my network.. a lot of the unlocked developer velocity is going to fun/frivolous/extra things. I think part of it is developers have their own features they want for themselves that are the easiest and most direct thing to deploy LLMs against, in the absence of good direction. Yes it's cool you finally achieved 100% test coverage, or you wrote a new utility that makes your job easier, or cleared the 2 year old ticket that was 100 deep in your backlog, etc. But there were ROI reasons these things were not done previously. That's already putting aside the fact that development going 3x faster doesn't increase end to end output by 3x because <100% of a SWEs job is development.
whimsicalism 4mo ago
I think this panic around overuse is largely just managers talking amongst themselves and everyone trying to see what everyone else is doing. It is frankly hard to interpret in any other way
- bluGill 4mo ago
  Also talking to accountants who will sound alerts if the bottom line isn't looking good enough.
matheusmoreira 4mo ago
This is great to see. LLMs are great but the industry really needs to experience a correction.

piker 4mo ago

I.e., the demand for programming tokens turns out to be quite elastic.

steveBK123 4mo ago
I would imagine it only gets worse in the face of good-enough open/chinese/local models too right? Microsoft adding Deepseek support already as I recall? That is - for any definition of "they are behind X months" then eventually they get to the point Claude was in January when the world freaked out, but at 1/10th the cost. A lot of firms are going to mandate that is good enough for their developers.
- sdesol 4mo ago
  > Microsoft adding Deepseek support I believe this hasn't been confirmed yet but I think it speaks to a bigger problem for the AI companies which is, if you give capable developers a good reasoning LLM, they can make it work like it was a really expensive model. I believe we are 100% at the stage of good enough for the vast majority of tech companines. Fable and others will be more valuable for non-traditional tech companies. I read somewhere that the chinese AI companies are sharing knowledge and it would not surprise me if the government is applying pressure by saying work together or else. If they work together, they can truly commoditize LLMs and with China ramping up hardware support for AI, I see the future being inference speed and hardware being the moat.
  - thewebguyd 4mo ago
    If hardware becomes the moat, the US frontier labs are screwed. We have AWS, Azure, GCP. All three have or are making inference silicon. LLMs become just another service in the public cloud's large service catalog, and open weight wins. Which makes sense to me. Selling a chatbot interface/model access to the general public was never going to be a viable long term play. You still need developers to wrap the models into specialized tools. Queue the Jobs quote "It's a feature, not a product."
    - sdesol 4mo ago
      The big thing is, the western world has moved so much of the manufacturing to China and think a lot of people will not forgive Samsung and others, so I can see China owning a good portion of the supply chain.
      - johnvanommen 4mo ago
        > The big thing is, the western world has moved so much of the manufacturing to China I built my career on Solaris and it got rugpulled by Linux. That wasn’t because of software, it was because of hardware. Linux’s cost advantage existed because Sun hardware had huge margins, because their software was basically free. AI will probably be a repeat of this. Whoever can come up with the hardware solution that minimizes the cost per token will win. I believe the 5090 still holds this crown, but someone certainly knows better than I do.
        fragmede 4mo ago
        but not all tokens are equal and vertical integration is the name of the game. Solaris did not lose to Linux, it lost to the LAMP stack on commodity x86 hardware. without the "AMP" part, Linux would've been dead in the water.
        fluoridation 4mo ago
        The L part of LAMP is kinda irrelevant at the application level, though. From the point of view of running the software, SAMP or LAMP doesn't make any difference. So yes, Solaris lost the competition with Linux to be the first letter of the acronym.
- CuriouslyC 4mo ago
  100%. There will be strict quotas on the expensive models and day to day work will be done on the cheap models that are "good enough" with escalation to the metered models when the cheaper options are spinning their wheels. Eventually the US frontier lab APIs will only get the most heavily triaged work that multiple tiers of cheaper Chinese open weight models have failed on. And of course the C-suite will have unlimited access to Mythos tier models, which they'll use to summarize reports, while passing down mandates to rank and file to increase usage of less expensive models.
- verdverm 4mo ago
  Yup, we are in the process of getting access to US hosted Chinese models. I've been petitioning Google and our rep, we will see but I suspect they will cave eventually. Gemini sucks and if they don't sell what their customers want, we go shopping around.
  - bloppe 4mo ago
    What you want is already available on OpenRouter and a million other services, but sure, you can wait 18 months for it to be on GCP.
    - verdverm 4mo ago
      > We are in the process... OpenRouter charges an extra 5.5%, Fireworks does not, Google is separate, but I doubt it will take 18 months. They are already aware they are losing business. OpenRouter is the wrong abstraction for enterprise, we only need one model provider, not everyone in the world. Nor do we want to have to worry about failover going to providers we don't want.
- michaelchisari 4mo ago
  I'm set up to use Qwen 3.6 locally if needed. It's solid, it does what I need, it runs on my laptop and it's free. But that's because I never got on the "run three dozen agents in a ralph loop" trend or other high-token usage methods. The way I use AI is discrete and targeted and it seems that's how it will be for everyone once the economics settle.
- Gigachad 4mo ago
  Our org straight up turned off our AI access after the GitHub copilot price increase blew right through the budget. At least on a personal I feel like I’ve been getting the same amount of work done but I have to think harder rather than sitting back and prompting and waiting.
- wookmaster 4mo ago
  My company banned Chinese models but I’ve got opencode setup with some open source coding models just in case. Luckily our laptops have some RAM (Mac)
jayd16 4mo ago
If folks won't pay a higher price, doesn't that mean it's inelastic?
- steveBK123 4mo ago
  Elastic demand means buyers are highly sensitive; a price hike causes a massive drop in purchases. Inelastic demand means buyers aren’t very sensitive; they keep buying regardless of price
  - jayd16 4mo ago
    Ah alright I have it backwards then.
- unholiness 4mo ago
  "Elastic" in economics happens to refers to how elastic the supply/demand is when the price changes (not vice versa, as you're describing). So e.g. an inelastic demand means the quantity demanded changes very little when the price doubles.

dranudin 4mo ago

I can second this. Our company and department was all-in on AI. And since the token-based pricing came in, we got an email from IT that tried to explain that most developers don't know how to choose models and that the cheap models should be good enough for most of our work ..

verdverm 4mo ago
Have they built an internal ai enablement team?
- dranudin 4mo ago
  Yes :D
daemonologist 4mo ago
Ha, developers know perfectly well how to choose models - they know that the company is footing the bill and Opus will give them the best result and/or require the least clean-up work. (We've received similar guidance. Even better is that our provider, GitHub Copilot, does not provide usage information to individual users if there is no per-user budget configured. So we just fire our requests into the black box and at the end of the month when IT gets the bill we maybe get a talking-to if it's excessive.)

burningChrome 4mo ago

I can give you some additional anecdotal evidence to support your comment. I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because teams were building the same tools for what they thought was nothing. Then they signed contracts with Anthropic and Google because I would assume they saw the token usage was through the roof. One month later? They completely cut off access to everybody for both Claude and Gemini. If you wanted access? Suddenly it was several forms, along with several approvals and a rock solid business case why you needed it. And before you got to the forms? You were added to a waiting list that was thousands of people long. The entire company is now in damage control after trying to get the genie back in the bottle. I'm guessing someone saw how much we would be paying for the tokens we'd been using and decided to shut the party down so to speak.

sdesol 4mo ago
Was there at least performance gains to be measured?
- burningChrome 4mo ago
  AFAIK nobody was collecting analytics. The one team I was working on had put out a goal of "30% more efficient" using AI tools. Its about as subject as you can get. We never got around to what exactly that meant before everything got shut down. Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would affirm or deny any of this had a positive impact. Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks.
  - stingraycharles 4mo ago
    > Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks. What do you mean placebo effect? They thought things were created with AI while they actually weren’t?

ofjcihen 4mo ago

I do a lot of client work for fortune 100’s. Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more. Obviously this won’t be on an individual basis but some kind of unit. Either way, with how much I see these companies cutting back I have no idea how the big AI companies are going to be profitable.

energy123 4mo ago
The AI companies will be profitable if they ship the goods. Make AGI and companies will pay for it.
- fluoridation 4mo ago
  Not if it costs more than a human, or if it performs worse (be that in quality or in time). After all, humans have NGI and companies don't just hire anyone with a pulse.
- oblio 4mo ago
  > Make AGI It's easy, just: Make economically viable cold fusion Make flying cars Make a perpetual motion machine Just because AGI is 3 simple letters doesn't implicitly mean we will live to see it.
- N_Lens 4mo ago
  Your comment reminds me of when a friend was watching me play counter strike 1.6 and said “this game looks so easy, just point at the enemies’ heads and left click?”
  - Der_Einzige 4mo ago
    He was right. Twitch shooters take far less skill than shooters with higher TTK and it’s not even close. Pros at “the finals” are much higher IQ than CS or valorant pros. Similar dynamic to how league might as well be candy land compared to dota2s checkers and star crafts chess.

Gigachad 4mo ago

Our company went from “AI AI AI” to “GitHub Copilot has been suspended due to exceeding the budget” with this month’s price increase.

mkozlows 4mo ago
If your company was giving you Copilot, they were never that far on board the AI train anyway.
- Gigachad 4mo ago
  Github Copilot is a different product than the Windows / Office one. Horrible naming I know but it's basically just a UI and router to pick whatever model you want.
  - wookmaster 4mo ago
    Microsoft names everything copilot anymore so everyone will continue to be confused
  - mkozlows 4mo ago
    No, I know what it is. But it's not as good as Claude Code or Codex. Harnesses matter for coding agents, it's not just the model. There's a reason Microsoft had to make its developers stop using Claude Code, and it's because they all found that Copilot sucked in comparison.
- xdertz 4mo ago
  Github Copilot lets you use Anthropic models. This is just a way for enterprises to fold this into their existing Microsoft contract instead of negotiating with a new vendor.

qurren 4mo ago

What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.

aryehof 4mo ago
They may be getting 3 times the work done, but what work? Quality effective code that meets requirements? Or something else?
- oblio 4mo ago
  > Quality effective code that meets requirements... ... and that also improves the company's financials. Otherwise it's all fluff.
steveBK123 4mo ago
Most companies are not seeing 2x or 3x of value produced from developers with AI is the reason
- brandensilva 4mo ago
  Yeah the cost doesn't justify the value. Spending x3 on tokens is meaningless if they aren't seeing the profit side scale with it. Going faster on product features doesn't mean it translates to more money. In fact I'm not even sure some customers can handle the speed at which a team could move with feature delivery. So scaling horizontally in different markets seems like an advantage if a product is already mature. Which is exactly what Anthropic and Open AI are doing because they want to put their tentacles in everything.
  - steveBK123 4mo ago
    > Going faster on product features doesn't mean it translates to more money. Exactly this. Unless the new features directly drive new users or new revenue from existing users, for many products iterating 2x faster does not mean 2x ROI. Additionally, from anecdotes here, at work, and in my network.. a lot of the unlocked developer velocity is going to fun/frivolous/extra things. I think part of it is developers have their own features they want for themselves that are the easiest and most direct thing to deploy LLMs against, in the absence of good direction. Yes it's cool you finally achieved 100% test coverage, or you wrote a new utility that makes your job easier, or cleared the 2 year old ticket that was 100 deep in your backlog, etc. But there were ROI reasons these things were not done previously. That's already putting aside the fact that development going 3x faster doesn't increase end to end output by 3x because <100% of a SWEs job is development.

whimsicalism 4mo ago

I think this panic around overuse is largely just managers talking amongst themselves and everyone trying to see what everyone else is doing. It is frankly hard to interpret in any other way

bluGill 4mo ago
Also talking to accountants who will sound alerts if the bottom line isn't looking good enough.

matheusmoreira 4mo ago

This is great to see. LLMs are great but the industry really needs to experience a correction.