7 ms·

LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place o

• snek_case 6mo ago
  They're also getting closer to IPO and have a growing user base. They can't justify losing a very large number of billions of other people's money in their IPO prospectus. So there's a push for them to increase revenue per user, which brings us closer to the real cost of running these models.
  • giwook 6mo ago
    I agree, and I'm also quite skeptical that Anthropic will be able to remain true to its initial, noble mission statement of acting for the global good once they IPO. At that point you are beholden to your shareholders and no longer can eschew profit in favor of ethics. Unfortunately, I think this is the beginning of the end of Anthropic and Modei being a company and CEO you could actually get behind and believe that they were trying to do "the right thing". It will become an increasingly more cutthroat competition between Anthropic and OpenAI (and perhaps Google eventually if they can close the gap between their frontier models and Claude/GPT) to win market share and revenue. Perhaps Amodei will eventually leave Anthropic too and start yet another AI startup because of Anthropic's seemingly inevitable prioritization of profit over safety.
    • devmor 6mo ago
      Skeptical is a light way to put it. It is essentially a forgone conclusion that once a company IPOs, any veil that they might be working for the global good is entirely lifted. A publicly traded company is legally obligated to go against the global good.
      • giwook 6mo ago
        Fair point. Call me an optimist, but I'm still holding out hope that Amodei is and still can do the right thing. That hope is fading fast though.
        • thibauts 6mo ago
          « Don’t be evil »
          • abirch 6mo ago
            If no one can buy your soul, what's its value? Every Management Consulting Firm
      • WarmWash 6mo ago
        The problem is that people equate money to power and power to evil. So no matter what, if you do something lots of people like (and hence compensate you for), you will be evil. It's a very interesting quirk of human intuition.
        • arcanemachiner 6mo ago
          A reasonable conclusion, considering that money and power seem to have their own gravity, so people with more of both end up getting even more of both, and vice versa. Can't blame someone who comes to such a conclusion about money and power.
          • WarmWash 6mo ago
            The unreasonable part automatically labeling power as evil.
    • snek_case 6mo ago
      I think the pivot to profit over good has been happening for a long time. See Dario hyping and salivating over all programming jobs disappearing in N months. He doesn't care at all if it's true or not. In fact he's in a terrible position to even understand if this is possible or not (probably hasn't coded for 10+ years). He's just in the business of selling tokens.
      • bombcar 6mo ago
        And worse, he (eventually) has to sell tokens above cost - which may have so much "baggage" (read: debt to pay Nvidia) that it'll be nearly impossible; or a new company will come to play with the latest and greatest hardware and undercut them. Just how if Boeing was able to release a supersonic plane that was also twice as efficient tomorrow; it'd destroy any airline that was deep in debt for its current "now worthless" planes.
        • outofpaper 6mo ago
          That's why open models are going to win in the long run.
          • otterley 6mo ago
            I think the key question is “when”? In a highly competitive business environment, companies are going to naturally be attracted to the most capable model if it leads to a competitive advantage and the switching costs are low. This suggests that “open” (giving away inference despite ever-higher training costs) may not win for a very long time, if ever.
            • rapind 6mo ago
              When frontier models plateau and efficiency increases sufficiently that it becomes a commodity like other cloud compute. One driver of open models might be foreign actors. With the entire US economy being held up by AI, it's a crucial vulnerability for a capable foreign actor (guess who) to exploit if they wanted to.
    • sumedh 6mo ago
      > At that point you are beholden to your shareholders No not really, you can issue two types of shares, the company founders can control a type of shares which has more voting power while other shareholders can get a different type of shares with less voting power. Facebook, Google has something similar.
      • what 6mo ago
        No, they still have to act in the interest of shareholders even if they have no voting power.
        • wsun19 6mo ago
          As a PBC, the intent of the company is not only profit, but it's hard to analyze the counterfactuals of if Anthropic were a pure for-profit or a non-profit
          • hiroboto 6mo ago
            thats the benefit of a pbc
        • sumedh 6mo ago
          What will happen if they don't because the founders control the voting powe
  • ljm 6mo ago
    They're also getting into cloud compute given you can use the desktop app to work in a temporary sandbox that they provision for you. I was about to call it reselling but so many startups with their fingers in the tech startup pie offer containerised cloud compute akin to a loss leader. Harking back to the old days of buying clock time on a mainframe except you're getting it for free for a while.
    • lelanthran 6mo ago
      > Harking back to the old days of buying clock time on a mainframe except you're getting it for free for a while. I submitted this yesterday but it got no traction (I did not write it): https://www.mjeggleton.com/blog/AIs-mainframe-moment https://www.mjeggleton.com/blog/AIs-mainframe-moment
  • zozbot234 6mo ago
    The "real cost" of running near-SOTA models is not a secret: you can run local models on your own infrastructure. When you do, you quickly find out that typical agentic coding incurs outsized costs by literal orders of magnitude compared to the simple Q&A chat most people use AI for. All tokens are very much not created equal, and the typical coding token (large model, large noisy context) costs a lot even under best-case caching scenarios.
• louiereederson 6mo ago
  I meant reference Toby Ord's work here. I think his framing of the performance/cost frontier hasn't gotten enough attention https://www.tobyord.com/writing/hourly-costs-for-ai-agents https://www.tobyord.com/writing/hourly-costs-for-ai-agents
  • fragmede 6mo ago
    That post doesn't address the human factor of cost, and I don't mean that in a good way. Even if AI costs more than a human, it's tireless, doesn't need holidays, is never going to have to go to HR for sexual harassment issues, won't show up hungover or need an advance to pay for a dying relative's surgery. It can be turned on and off with the flip of a switch. Hire 30 today, fire 25 of them next week. Spin another 5 up just before the trade show demo needs to go out and fire them with no remorse afterwards.
    • pona-a 6mo ago
      I think the word you're looking for is contractors. But yes, you still have to treat those with _some_ human decency.
    • piker 6mo ago
      That was a great promise before the models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities and defaults without transparency or recourse.
      • mh- 6mo ago
        I still haven't seen any statistically sound data supporting that this is happening on the API (per-token pricing.) If you've got something to share I'd love to see it.
        • jessermejia 6mo ago
          There's an interesting analysis here: https://github.com/anthropics/claude-code/issues/42796 https://github.com/anthropics/claude-code/issues/42796 >The most striking row is user prompts: 5,608 in February vs 5,701 in March. The human put in the same effort. But the model consumed 80x more API requests and 64x more output tokens to produce demonstrably worse results.
• paulddraper 6mo ago
  > The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is at the cost of dramatically higher operating costs. Or they are just not willing to burn obscene levels of capital like OpenAI.
  • deleted 6mo ago
    [deleted]
• ethin 6mo ago
  I mean, the signs have been there that the costs to run and operate these models wasn't as simple as inference costs. And the signs were there (and, arguably, are still there) that it costs way, way more than many people like to claim on the part of Anthropic. So to me this price hike is not at all surprising. It was going to come eventually, and I suspect it's nowhere near over. It wouldn't surprise me if in 2-3 years the "max" plan is $800 or $2000 even.
  • ezst 6mo ago
    > It wouldn't surprise me if in 2-3 years the "max" plan is $800 or $2000 even. I'd rather be surprised if they are still doing business by then.
    • QuiEgo 6mo ago
      I would not be surprised at all, a $1,000/mo tool that makes your $20,000/mo engineer a lot more productive is an easy sell. I’m guessing we’re gonna have a world like working on cars - most people won’t have expensive tools (ex a full hydraulic lift) for personal stuff, they are gonna have to make do with lesser tools.
      • cyanydeez 6mo ago
        noway. i bought a $3k AMD395+ under the Sam Altman price hike and its got a local model that readily accomplishes medial tasks. theres a ceiling to these price hikes because open weights will keep popping up as competitors tey to advertise their wares. sure, we POV different capabilities but theres definitely not that much cash in propfietary models for their indererminance
        • benjiro3000 6mo ago
          [dead]
      • selfmodruntime 6mo ago
        No engineer will cost 20.000 bucks a month at this point in time. Offshoring is still happening aggressively.
• Aurornis 6mo ago
  > It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. I think we're reaching the point where more developers need to start right-sizing the model and effort level to the task. It was easy to get comfortable with using the best model at the highest setting for everything for a while, but as the models continue to scale and reasoning token budgets grow, that's no longer a safe default unless you have unlimited budgets. I welcome the idea of having multiple points on this curve that I can choose from. depending on the task. I'd welcome an option to have an even larger model that I could pull out for complex and important tasks, even if I had to let it run for 60 minutes in the background and made my entire 5-hour token quota disappear in one question. I know not everyone wants this mental overhead, though. I predict we'll see more attempts at smart routing to different models depending on the task, along with the predictable complaints from everyone when the results are less than predictable.
  • Salgat 6mo ago
    The problem is that without a lot of human involvement (which is even more expensive), even the best still make a lot of mistakes.
  • KaiShips 6mo ago
    [flagged]
  • jpalawaga 6mo ago
    Except developers can’t even do that. Estimation of any not-small task that hasn’t been done before is essentially a random guess.
    • nilkn 6mo ago
      I don't completely agree. Estimation is nontrivial, but not necessarily a random guess. Teams of human engineers have been doing this for decades -- not always with great success, but better than random. Deciding whether to put an intern or your best staff engineer on a problem is a challenge known to any engineering manager and TPM.
• Lihh27 6mo ago
  heh adaptive thinking is letting the meter run itself. they make more when it runs longer.
• iainmerrick 6mo ago
  That sounds very plausible. But it implies they could offer even higher performance models at much higher costs if they chose to; and presumably they would if there were customers willing to pay. Is that the case? Surely there are a decent number of customers who’d be willing to pay more, much more, to get the very best LLMs possible. Like, Apple computers are already quite pricey -- $1000 or $2000 or so for a decent one. But you can spec up one that’s a bit better (not really that much better) and they’ll charge you $10K, $20K, $30K. Some customers want that and many are willing to pay for it. Is there an equivalent ultra-high-end LLM you can have if you’re willing to pay? Or does it not exist because it would cost too much to train?
  • criemen 6mo ago
    > Is there an equivalent ultra-high-end LLM you can have if you’re willing to pay? Or does it not exist because it would cost too much to train? I guess at the time that was GPT-4.5. I don't think people used it a lot because it was crazy expensive, and not that much better than the rest of the crop.
    • foobar10000 6mo ago
      The issue is not better - it’s better _AND_ fast enough. An agentic loop is essentially [think,verify] in a loop - i.e. [t1,v1,t2,v2,t3,v3,…] A model that does [t1,t2,t3,t4] in 40 minutes, if verify takes 10 min, will most likely do MUCH worse that a model that does t1 (decently worse) in 10 mins, v1 in 10 mins, t2 now based on t1 and v1 in 10 mins, v2 in 10 mins, etc.. So, for agentic workflows - ones where the model gets feedback from tools, etc…, fast enough is important.
  • AshamedBadger56 6mo ago
    > As of right now that's basically Mythos. If the rumors are true, it's a 10 trillion parameter model which is QUITE a bit larger than other models. If so, I'd suspect the main reason it has a super limited roll out is that they can't afford to run it at scale yet.
• conductr 6mo ago
  Yeah. Combine this with much of Corpos right now using a “burn as many tokens as you need” policy on AI, the incentive is there for them to raise price and find an equilibrium point or at least reduce the bleed.
• amelius 6mo ago
  Once they implement their models directly in silicon, the cost will come down and the speed will go up. See Taalas.
  • aaronblohowiak 6mo ago
    taalas is amazing. id gladly spend 5-15k on something that matched that performance with opus 4.6 quality
• orangecar 6mo ago
  [dead]
• nl 6mo ago
  This is a bad take. It's not really wrong in the sense that yes higher performance does cost more. But it ignores completely the fact that the same intelligence is dropping by an order of magnitude (at least) every 12 months. GPT o1 launched at $600/M output tokens and GPT4.5 launched at $150/M. Opus 4.7 is $25/M for more intelligence
• jimiljojo 6mo ago
  What a well thought and written comment. I totally agree.
  • jimiljojo 6mo ago
    Also appreciate the author for doing the analysis and sharing the results
• atoav 6mo ago
  For me it was pretty clear from the start that costs will have to increase. It is the classical drug dealer model: first you hook them with cheap supply, maybe even free, then you slowly jack the prize up to a level that can (just) be sustained. Then you decrease the quality of the product by diluting it so you get more bucks for each gram you bought. You could also call it enshittification if you like. The goal of every company that needs to make ever more money for investors is to earn more money while spending less. There are many ways of doing this without reducing the quality of the product, e.g. using less staff to do more, getting more compute out of same the energy, using cheaper or free energy, optimizing algorithms in ways that do not degrade quality or you grow because you gain more customers and break into new markets etc. And once you made all these optimizations and the market is saturated, then the only optimizations left are the ones where the quality goes down or the risk is increased. Quality in that sense, is what you can get away with without customers jumping ship. So you will also work on locking customers in and make jumping ship look very hard and complicated.
• svantana 6mo ago
  FWIW, Artificial Analysis has a "Intelligence vs Cost" plot on their front page that shows models' score vs cost to run the benchmark, which should be more fair in this sense. According to that one, Opus 4.7 (max) is slightly cheaper than 4.6 (though still very expensive).