24 ms·

Measuring Claude 4.7's tokenizer costs

• bcjdjsndon 5mo ago
  Because those braniacs added 20-30% more system prompt
• uberman 5mo ago
  On actual code, I see what you see a 30% increase in tokens which is in-line with what they claim as well. I personally don't tend to feed technical documentation or random pros into llms. Given that Opus 4.6 and even Sonnet 4.6 are still valid options, for me the question is not "Does 4.7 cost more than claimed?" but "What capabilities does 4.7 give me that 4.6 did not?" Yesterday 4.6 was a great option and it is too soon for me to tell if 4.7 is a meaningful lift. If it is, then I can evaluate if the increased cost is justified.
  • pier25 5mo ago
    haven't people been complaining lately about 4.6 getting worse?
    • ed_elliott_asc 5mo ago
      No we increased our plans
    • solenoid0937 5mo ago
      People complain about a lot of things. Claude has been fine: https://marginlab.ai/trackers/claude-code-historical-performance/ https://marginlab.ai/trackers/claude-code-historical-perform...
      • Majromax 5mo ago
        While that's a nice effort, the inter-run variability is too high to diagnose anything short of catastrophic model degradation. The typical 95% confidence interval runs from 35% to 65% pass rates, a full factor of two performance difference. Moreover, on the companion codex graphs (https://marginlab.ai/trackers/codex-historical-performance/ https://marginlab.ai/trackers/codex-historical-performance/), you can see a few different GPT model releases marked yet none correspond to a visual break in the series. Either GPT 5.4-xhigh is no more powerful than GPT 5.2, or the benchmarking apparatus is not sensitive enough to detect such changes.
        • yorwba 5mo ago
          Yes, MarginLab only tests 50 tasks a day, which is too few to give a narrower confidence interval. On the other hand, this really calls into question claims of performance degradation that are based on less intensive use than that. Variance is just so high that long streaks of bad luck are to be expected and plausibly the main source of such complaints. Similarly, it's unlikely you can measure a significant performance difference between models like GPT 5.4-xhigh and GPT 5.2 unless you have a task where one of them almost always fails or one almost always succeeds (thus guaranteeing low variance), or you make a lot of calls (i.e. probably through the API and not in interactive mode.)
          • Majromax 5mo ago
            > Similarly, it's unlikely you can measure a significant performance difference between models like GPT 5.4-xhigh and GPT 5.2 unless you have a task where one of them almost always fails or one almost always succeeds That feels like a concession to the limited benchmarking framework. 5.4-xhigh is supposed to be (and is widely believe to be) a better model than 5.2, so if that's invisible in the benchmarking scores then the protocol has problems. The test probably should include cases that should be 'easy passes' or 'near always failures', and then paired testing could offer greater precision on improvements or degradations. Conversely, if model providers also don't do this then they could be accidentally 'benchmaxxing' if they use protocols like this to set dynamic quantization levels for inference. All you really need for a credible observation of problems from 'less intensive use' is a problem domain that isn't well-covered by the measured and monitored benchmark.
            • yorwba 5mo ago
              Here's a sample-size calculator that may help illustrate the issue: https://sample-size.net/sample-size-proportions/ https://sample-size.net/sample-size-proportions/ Put in the benchmark score of one model as p₀ and of the other model as p₁ (as a fraction between 0 and 1) and observe what kind of sample size you need to reliably observe a significant difference. The largest change between GPT 5.2 and 5.4 highlighted in https://openai.com/index/introducing-gpt-5-4/ https://openai.com/index/introducing-gpt-5-4/ is OSWorld-Verified going from 47.3% to to 75.0%. That's quite the difference, right? So plug in 0.473 and 0.75 and note that the required sample size per model is 55. For the software engineering tasks in SWE-Bench Pro, the change from 55.6% to 57.7% is a whopping 2.1 percentage points, which you can detect with a mere 8836 samples. I'm sure someone in charge of benchmarking at OpenAI knows how statistics work and always makes sure to take a sufficiently large number of samples when comparing different models, but for most other people who want to know which model is better, the answer is unlikely to be worth the cost of measuring it precisely enough to find out.
      • cbg0 5mo ago
        That performance monitor is super easy to game if you cache responses to all the SWE bench questions.
        • solenoid0937 5mo ago
          You dramatically overestimate how much time engineers at hypergrowth startups have on their hands
          • cbg0 5mo ago
            Caching some data is time consuming? They can just ask Claude to do it.
          • dns_snek 5mo ago
            There's a direct business incentive to game/cheat benchmarks, it wouldn't even be difficult to do, and besides, they have workforce-replacing AI to do it for them.
            • cindyllm 5mo ago
              [dead]
      • addisonj 5mo ago
        I will be the first to acknowledge that humans are a bad judge of performance and that some of the allegations are likely just hallucinations... But... Are you really going to completely rely on benchmarks that have time and time again be shown to be gamed as the complete story? My take: It is pretty clear that the capacity crunch is real and the changes they made to effort are in part to reduce that. It likely changed the experience for users.
      • sumedh 5mo ago
        Your link shows there have been huge drops. How is it fine?
      • jofzar 5mo ago
        Matrix also found that Claude was AB testing 4.6 vs 4.7 in production for the last 12 days. https://matrix.dev/blog-2026-04-16 https://matrix.dev/blog-2026-04-16
  • grim_io 5mo ago
    How long will they host 4.6? Maybe longer for enterprise, but if you have a consumer subscription, you won't have a choice for long, if at all anymore.
    • nfredericks 5mo ago
      Opus 4.5 is still available
      • grim_io 5mo ago
        Wow, they hosted it for 6 months. Truly LTS territory :)
    • Jeremy1026 5mo ago
      I was trying to figure out earlier today how to get 4.6 to run in Claude Code, as part of the output it included "- Still fully supported — not scheduled for retirement until Feb 2027." Full caveat of, I don't know where it came up with this information, but as others have said, 4.5 is still available today and it is now 5, almost 6 months old.
    • hypercube33 5mo ago
      I'm still using 4.5 because it gets the niche work I'm using it for where 4.6 would just fight me.
  • tetha 5mo ago
    Yeah that was an interesting discovery in a development meeting. Many people were chasing after the next best model and everything, though for me, Sonnet 4.6 solves many topics in 1-2 rounds. I mainly need some focus on context, instructions and keeping tasks well-bounded. Keeping the task narrow also simplifies review and staying in control, since I usually get smaller diffs back I can understand quickly and manage or modify later. I'll look at the new models, but increasing the token consumptions by a factor of 7 on copilot, and then running into all of these budget management topics people talk about? That seems to introduce even more flow-breakers into my workflow, and I don't think it'll be 7 times better. Maybe in some planning and architectural topics where I used Opus 4.6 before.
    • snoman 5mo ago
      I wonder if there are different use cases. You sound like you’re using an LLM in a similar way to me. I think about the problem and solution, describe what I need implemented, provide references in the context (“the endpoint should be structured like this one…”) and then evaluate the output. It sounds like other folks are more throwing an LLM at the problem to see what it comes up with. More akin to how I delegate a problem to one of my human engineers/architects. I understand, conceptually, why they might be doing that but I know that I stopped trying that because it didn’t produce quality. I wonder if the newer models are better at handling that ambiguity better.
  • jstummbillig 5mo ago
    I don't understand how people measure how much more or less work they need to do. It's not that gpt-4o was incapable of exuding enormous amounts of code quickly, it's that the tokens were relativ garbage. How do you have an opinion on 4.6/4.7 here? It's less clear but I could totally see that 4.7 or beyond leads to project completion 20% faster, by removing dead ends, foot guns, less backtracking, etc. How to tell / measure effectively? No clue.
    • uberman 5mo ago
      My personal opinion here based on observations not empirical tested. 4.5 could generate code, but I often ran out of context and the results were regularly incomplete. The result was that I had to spend as much time proofing and debugging as I did making direct progress. 4.6 has what in practice seems to an almost unlimited context window and rarely produces incomplete or flat out wrong results. That is a big step forward though i do burn through quota much faster. I have not formed an option yet how what 4.7 does for me other than to say I have observed my quota being consumed faster. To be fair, I have not put 4.7 to a challenging task yet. It honestly surprises me that someone who regularly uses Claude would not have an opion about 4.6 or even Opus vs Sonnet at this point. The lift at least for me was obvious.
• dallen33 5mo ago
  I'm still using Sonnet 4.6 with no issues.
  • risyachka 5mo ago
    How does this solve the issue? 4.6 will be disabled after one or more release like any other legacy model.
    • gadflyinyoureye 5mo ago
      Won't the thing that replaces 4.6 come down in token cost?
• iknowstuff 5mo ago
  Interesting because I already felt like current models spit out too much garbage verbose code that a human would write in a far more terse, beautiful and grokable way
  • aray07 5mo ago
    yeah opus 4.7 feels a lot more verbose - i think they changed the system prompt and removed instructions to be terse in its responses
  • QuercusMax 5mo ago
    I had a case yesterday where Claude wrote me a series of if/elses in python. I asked it if it could use some newer constructs instead, and it told me that I was on a new enough python version that I could use match/case. Great! And then it proceeded to rewrite the block with a dict lookup plus if-elses, instead of using match/case. I had to nag it to actually rewrite the code the way it said it would!
• louiereederson 5mo ago
  LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is at the cost of dramatically higher operating costs. Their gross margins in this past quarter will be an important data point on this. I think the tendency for graphs of model assessment to display the log of cost/tokens on the x axis (i.e. Artificial Analysis' site) has obscured this dynamic.
  • snek_case 5mo ago
    They're also getting closer to IPO and have a growing user base. They can't justify losing a very large number of billions of other people's money in their IPO prospectus. So there's a push for them to increase revenue per user, which brings us closer to the real cost of running these models.
    • giwook 5mo ago
      I agree, and I'm also quite skeptical that Anthropic will be able to remain true to its initial, noble mission statement of acting for the global good once they IPO. At that point you are beholden to your shareholders and no longer can eschew profit in favor of ethics. Unfortunately, I think this is the beginning of the end of Anthropic and Modei being a company and CEO you could actually get behind and believe that they were trying to do "the right thing". It will become an increasingly more cutthroat competition between Anthropic and OpenAI (and perhaps Google eventually if they can close the gap between their frontier models and Claude/GPT) to win market share and revenue. Perhaps Amodei will eventually leave Anthropic too and start yet another AI startup because of Anthropic's seemingly inevitable prioritization of profit over safety.
      • devmor 5mo ago
        Skeptical is a light way to put it. It is essentially a forgone conclusion that once a company IPOs, any veil that they might be working for the global good is entirely lifted. A publicly traded company is legally obligated to go against the global good.
• xd1936 5mo ago
  And what about with Caveman[1]? 1. https://github.com/juliusbrussee/caveman https://github.com/juliusbrussee/caveman
  • Majromax 5mo ago
    Caveman doesn't and cannot change the tokenizer, so the relative token count differences by input category will remain unchanged.
  • brokencode 5mo ago
    Can we have one thread about Claude without people trying to shovel Caveman? Much of the token usage is in reasoning, exploring, and code generation rather than outputs to the user. Does making Claude sound like a caveman actually move the needle on costs? I am not sure anymore whether people are serious about this. To me, caveman sounds bad and is not as easy to understand compared to normal English.
  • aray07 5mo ago
    isn’t caveman a joke? why would you use it for real work?
    • wetpaws 5mo ago
      [dead]
• atonse 5mo ago
  Just yesterday I was happy to have gotten my weekly limit reset [1]. And although I've been doing a lot of mockup work (so a lot of HTML getting written), I think the 1M token stuff is absolutely eating up tokens like CRAZY. I'm already at 27% of my weekly limit in ONE DAY. https://news.ycombinator.com/item?id=47799256 https://news.ycombinator.com/item?id=47799256
  • aray07 5mo ago
    yeah similar for me - it uses a bunch more tokens and I haven’t been able to tell the ROI in terms of better instruction following it seems to hallucinate a bit more (anecdotal)
    • titaniumtown 5mo ago
      I had it hallucinate a tool that didn't exist, it was very frustrating!
      • dminik 5mo ago
        Anthropic intruduces fake tool calls to prevent distillation of their models. Others still distill. Anthropic distils third party models. Claude now hallucinates tools. Brilliant.
  • jabart 5mo ago
    I'm seeing the opposite. With Opus 4.7 and xhigh, I'm seeing less session usage , it's moving faster, and my weekly usage is not moving that much on a Team Pro account.
  • cbm-vic-20 5mo ago
    Four day workweek!
  • sreekanth850 5mo ago
    Iam at 22%, just two task. A bug fixing and a Scalar integration.
  • 5mo ago
• jmward01 5mo ago
  Yeah. I just did a day with 4.7 and I won't be going back for a while. It is just too expensive. On top of the tokenization the thinking seems like it is eating a lot more too.
  • aray07 5mo ago
    yeah i am still not clear why there are 5 effort modes now on top of more expensive tokenization
    • jddj 5mo ago
      Once you've seen a few results of an LLM given too much sway over product decisions, 5 effort modes expressed as various english adjectives is pretty much par for the course
    • jmward01 5mo ago
      choice is often a great dark-pattern (lack of choice is too but...). Choices generally grow cost to discover optimality in an np way. This means if the entity giving choice has more ability to compute the value prop than the entity deciding the choice you can easily create an exploitive system. Just create a bunch of choices, some actually do save money with enough thought but most don't, and you will gain: People that think they got what they wanted, the feature is there!, so they can't complain but... People that end up essentially randomly picking so the average value of the choices made by customers is suboptimal.
  • JimmaDaRustla 5mo ago
    What was your level methodology and results? Can't just post "too expensive" and not explain how you went about it.
• rafram 5mo ago
  Pretty funny that this article was clearly written by Claude.
• markrogersjr 5mo ago
  4.7 one-shot rate is at least 20-30% higher for me
  • ChicagoBoy11 5mo ago
    How are you able to track this as you use it? A bit stumped atm
    • markrogersjr 5mo ago
      Purely empirical
  • deleted 5mo ago
    [deleted]
• CodingJeebus 5mo ago
  The fundamental problem with these frontier model companies is that they're incentivized to create models that burn through more tokens, full stop. It's a tale as old as capitalism: you wake up every day and choose to deliver more value to your customers or your shareholders, you cannot do both simultaneously forever. People love to throw around "this is the dumbest AI will ever be", but the corollary to that is "this is the most aligned the incentives between model providers and customers will ever be" because we're all just burning VC money for now.
  • NickC25 5mo ago
    > but the corollary to that is "this is the most aligned the incentives between model providers and customers will ever be" because we're all just burning VC money for now. Please say this louder for everyone to hear. We are still at the stage where it is best for Anthropic's product to be as consumer aligned (and cost-friendly) as possible. Anthropic is loosing a lot of money. Both of those things will not be true in the near future.
  • BosunoB 5mo ago
    Their bigger incentive is to deliver the best product in the cheapest way, because there is tight competition with at least 2 other companies. I know we all love to hate on capitalism but it's actually functioning fine in this situation, and the token inflation is their attempt to provide a better product, not a worse one.
  • HarHarVeryFunny 5mo ago
    > The fundamental problem with these frontier model companies is that they're incentivized to create models that burn through more tokens That's one market segment - the high priced one, but not necessarily the most profitable one. Ferrari's 2025 income was $2B while Toyota's was $30B. Maybe a more apt comparison is Sun Microsystems vs the PC Clone market. Sun could get away with high prices until the PC Clones became so fast (coupled with the rise of Linux) that they ate Sun's market and Sun went out of business. There may be a market for niche expensive LLMs specialized for certain markets, but I'll be amazed if the mass coding market doesn't become a commodity one with the winners being the low cost providers, either in terms of API/subscriptions costs, or licensing models for companies to run on their own (on-prem or cloud) servers.
• stefan_ 5mo ago
  I don't know anything about tokens. Anthropic says Pro has "more usage*", Max has 5x or 20x "more usage*" than Pro. The link to "usage limits" says "determines how many messages you can send". Clearly no one is getting billed for tokens.
  • aray07 5mo ago
    anthropic’s pricing is all based on token usage https://platform.claude.com/docs/en/about-claude/pricing https://platform.claude.com/docs/en/about-claude/pricing So if you are generating more tokens, you are eating up your usage faster
• _pdp_ 5mo ago
  IMHO there is a point where incremental model quality will hit diminishing returns. It is like comparing an 8K display to a 16K display because at normal viewing distance, the difference is imperceptible, but 16K comes at significant premium. The same applies to intelligence. Sure, some users might register a meaningful bump, but if 99% can't tell the difference in their day-to-day work, does it matter? A 20-30% cost increase needs to deliver a proportional leap in perceivable value.
  • snek_case 5mo ago
    It probably depends what you're using the models for. If you use them for web search, summarizing web pages, I can imagine there's a plateau and we're probably already hitting it. For coding though, there is kind of no limit to the complexity of software. The more invariants and potential interactions the model can be aware of, the better presumably. It can handle larger codebases. Probably past the point where humans could work on said codebases unassisted (which brings other potential problems).
    • Bolwin 5mo ago
      > summarizing web pages For summarizing creative writing, I've found Opus and Gemini 3 pro are still only okay and actively bad once it gets over 15K tokens or so. A lot of long context and attention improvements have been focused on Needle in a Haystack type scenarios, which is the opposite of what summarization needs.
  • aray07 5mo ago
    yeah thats is my biggest issue - im okay with paying 20-30% more but what is the ROI? i dont see an equivalent improvement in performance. Anthropic hasnt published any data around what these improvements are - just some vague “better instruction following"
    • margorczynski 5mo ago
      The other thing is most people don't really care about price per token or whatever but how much it will cost to execute (successfully) a task they want. It doesn't matter if a model is e.g. 30% cheaper to use than another (token-wise) but I need to burn 2x more tokens to get the same acceptable result.
• mikert89 5mo ago
  The compute is expensive, what is with this outrage? People just want free tools forever?
  • rvz 5mo ago
    > The compute is expensive, what is with this outrage? Gamblers (vibe-coders) at Anthropic's casino realising that their new slot machine upgrade (Claude Opus) is now taking 20%-30% more credits for every push of the spin button. Problem is, it advertises how good it is (unverified benchmarks) and has a better random number generator but it still can be rigged (made dumber) by the vendor (Anthropic). The house (Anthropic) always wins. > People just want free tools forever? Using local models are the answer to this if you want to use AI models free forever.
  • aray07 5mo ago
    are you okay with paying more for your services without any perceived improvement in the service itself?
    • schmookeeg 5mo ago
      That's been a constant for my entire adult life.
    • deleted 5mo ago
      [deleted]
  • Jweb_Guru 5mo ago
    I'm mostly surprised that people found the output quality of Opus 4.6 good enough... 4.7 so far is a pretty sizable improvement for the stuff I care about. I don't really care how cheap 4.6 was per task when 90% of the tasks weren't actually being done correctly. Or maybe it's that people like the LLM agreeing with them blindly while sneakily doing something else under the hood? Did people enjoy Claude routinely disregarding their instructions? Not really sure I understand, I truly found 4.6 immensely frustrating (from the getgo, not just the "pre-nerf" version, whatever that means). 4.7 is a buggy mess, it's slow, and it costs a lot per token. It's also a huge breath of fresh air because it actually seems to make a good faith effort at doing the thing you asked it to do, and doesn't waste your time with irrelevant nonsense just to make it look busy or because it thinks you want that nonsense (I mean, it still does all of these things to some extent, but so far it seems like it does them much less than 4.6 did). Disclaimer: I'm always running on max and don't really have token limits so I am in a position not to care about cost per token. But I am not surprised by the improved benchmark results at all, 4.6 was really not nearly as strong of a model as people seem to remember it being.
• sipsi 5mo ago
  I tried to do my usual test (similar to pelican but a bit more complex) but it ran out of 5 hour limit in 5 minutes. Then after 5 hours I said "go on" and the results were the worst I've ever seen.
• qq66 5mo ago
  This is the backdoor way of raising prices... just inflate the token pricing. It's like ice cream companies shrinking the box instead of raising the price
  • Bridged7756 5mo ago
    No, you're forgetting the never ending world shattering models being released every couple of months. Each one with 2X token costs of course, for a vague performance gain and that will deprecate the previous ones.
    • therobots927 5mo ago
      It’s nice to see comments like this. It makes me feel less crazy. Something very weird is going on behind the scenes at Anthropic.
    • captn3m0 5mo ago
      https://platform.claude.com/docs/en/about-claude/model-deprecations#model-status https://platform.claude.com/docs/en/about-claude/model-depre... Retirement date for Opus 4.6 is marked as "Not sooner than February 5, 2027"
• Yukonv 5mo ago
  Some broad assumptions are being made that plans give you a precise equivalent to API cost. This is not the case with reverse engineering plan usage showing cached input is free [0]. If you re-run the math removing cached input the usage cost is ~5-34% more. Was the token plan budget increase [1] proportional to account for this? Can’t say with certainty. Those paying API costs though the price hike is real. [0] https://she-llac.com/claude-limits https://she-llac.com/claude-limits [1] https://xcancel.com/bcherny/status/2044839936235553167 https://xcancel.com/bcherny/status/2044839936235553167
• throwaway613746 5mo ago
  [dead]
• encoderer 5mo ago
  In my “repo os” we have an adversarial agent harness running gpt5.4 for plan and implementation and opus4.6 for review. This was the clear winner in the bake-off when 5.4 came out a couple months ago. Re-ran the bake-off with 4.7 authoring and… gpt5.4 still clearly winning. Same skills, same prompts, same agents.md.
• lacoolj 5mo ago
  This is probably an adjacent result of this (from anthropic launch post): > In Claude Code, we’ve raised the default effort level to xhigh for all plans. Try changing your effort level and see what results you get
  • aray07 5mo ago
    effort level is separate from tokenization. Tokenization impacts you the same regardless. I find 5 thinking levels to be super confusing - I dont really get why they went from 3 -> 5
• curioussquirrel 5mo ago
  Claude's tokenizers have actually been getting less efficient over the years (I think we're at the third iteration at the least since Sonnet 3.5). And if you prompt the LLM in a language other than English, or if your users prompt it or generate content in other languages, the costs go higher even more. And I mean hundreds of percent more for languages with complex scripts like Tamil or Japanese. If you're interested in the research we did comparing tokenizers of several SOTA models in multiple languages, just hit me up.
  • arcanemachiner 5mo ago
    I would encourage you to post a link here, and also to submit to HN if you haven't already. :)
    • curioussquirrel 5mo ago
      Will do! Thanks for the encouragement
    • curioussquirrel 5mo ago
      Here you go! https://news.ycombinator.com/item?id=47847282 https://news.ycombinator.com/item?id=47847282
• varispeed 5mo ago
  Don't forget that the model doesn't have an incentive to give right solution the first time. At least with Opus 4.6 after it got nerfed, it would go round in circles until you tell it to stop defrauding you and get to correct solution. That not always worked though. I found starting session again and again until less nerfed model was put on the request. Still all points to artificially make customer pay more.
• namnnumbr 5mo ago
  The title is a misdirection. The token counts may be higher, but the cost-per-task may not be for a given intelligence level. Need to wait to see Artificial Analysis' Intelligence Index run for this, or some other independent per-task cost analysis. The final calculation assumes that Opus 4.7 uses the exact same trajectory + reasoning output as Opus 4.6. I have not verified, but I assume it not to be the case, given that Opus 4.7 on Low thinking is strictly better than Opus 4.6 on Medium, etc., etc.
  • watsonL1F7 5mo ago
    [dead]
  • aray07 5mo ago
    im running some experiments on this but based on what i have seen on my own personal data - I dont think this is true "given that Opus 4.7 on Low thinking is strictly better than Opus 4.6 on Medium, etc., etc.” Opus 4.7 in general is more expensive for similar usage. Now we can argue that is provides better performance all else being equal but I haven’t been able to see that
  • unpwn 5mo ago
    Very unlikely that the article is wrong. the 4.7 intelligence bump is not that big, plus most of the token spend is in inputs/tool calls etc, much of which won't change even with this bump.
    • namnnumbr 5mo ago
      IMO, you're incorrect: 1. In my own use, since 1 Apr this month, very heavy coding: > 472.8K Input Tokens +299.3M cached > 2.2M Output Tokens My workloads generate ~5x more output than input, and output tokens cost 5x more per token... output dominates my bill at roughly 25x the cost of input. (Even more so when you consider cache hits!) If Opus 4.7 was more efficient with reasoning (and thus output), I'd likely save considerable money (were I paying per-token). 2. Anthropic's benchmarks DO show strictly-better (granted they are Anthropic's benchmarks, so salt may be needed) https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2Fff97ab0f2a5f3a243da02398f97dec1ac99b526a-3840x2160.png&w=3840&q=75 https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-...
• _fat_santa 5mo ago
  A question I've been asking alot lately (really since the release of GPT-5.3) is "do I really need the more powerful model"? I think a big issue with the industry right now is it's constantly chasing higher performing models and that comes at the cost of everything else. What I would love to see in the next few years is all these frontier AI labs go from just trying to create the most powerful model at any cost to actually making the whole thing sustainable and focusing on efficiency. The GPT-3 era was a taste of what the future could hold but those models were toys compare to what we have today. We saw real gains during the GPT-4 / Claude 3 era where they could start being used as tools but required quite a bit of oversight. Now in the GPT-5 / Claude 4 era I don't really think we need to go much further and start focusing on efficiency and sustainability. What I would love the industry to start focusing on in the next few years is not on the high end but the low end. Focus on making the 0.5B - 1B parameter models better for specific tasks. I'm currently experimenting with fine-tuning 0.5B models for very specific tasks and long term I think that's the future of AI.
  • fkealy 5mo ago
    I agree, and yet here i am using it... However, I think the industry IS going multiple directions all at once with smaller models, bigger models etc. I need to try out Google's latest models but alas what can one person do in the face of so many new models...
  • minimaxir 5mo ago
    Many people were hoping that Sonnet 4.6 was "Opus 4.5 quality but with Sonnet speed/cost" but unfortunately that didn't pan out.
    • malfist 5mo ago
      You can already see people here saying the same stuff about opus 4.7, saw a comment claiming that Opus 4.7 on low thinking was better than 4.6 on high. I'm not seeing that in my testing, but these opinions are all vibe based anyway.
  • rambojohnson 5mo ago
• beej71 5mo ago
  News like this always makes me wonder about running my own model, something I've never done. A couple thousand bucks can get you some decent hardware, it looks like, but is it good for coding? What is your all's experience? And if it's not good enough for coding, what kind of money, if any, would make it good enough?
  • aray07 5mo ago
    i think the new qwen models are supposed to be good based on some the articles that i read
  • hleszek 5mo ago
    The latest Qwen3.6 model is very impressive for its size. Get an RTX 3090 and go to https://www.reddit.com/r/LocalLLaMA/ https://www.reddit.com/r/LocalLLaMA/ to see the latest news on how to run models locally. Totally fine for coding.
  • bakugo 5mo ago
    You should be aware that any model you can run on less than $10k worth of hardware isn't going to be anywhere close to the best cloud models on any remotely complex task. Many providers out there host open weights models for cheap, try them out and see what you think before actually investing in hardware to run your own.
  • arcanemachiner 5mo ago
    I want to give give you realistic expectations: Unless you spend well over $10K on hardware, you will be disappointed, and will spend a lot of time getting there. For sophisticated coding tasks, at least. (For simple agentic work, you can get workable results with a 3090 or two, or even a couple 3060 12GBs for half the price. But they're pretty dumb, and it's a tease. Hobby territory, lots of dicking around.) Do yourself a favor: Set up OpenCode and OpenRouter, and try all the models you want to try there. Other than the top performers (e.g. GLM 5.1, Kimi K2.5, where required hardware is basically unaffordable for a single person), the open models are more trouble than they're worth IMO, at least for now (in terms of actually Getting Shit Done).
• adaptive_loop 5mo ago
  Every time a new model comes out, I'm left guessing what it means for my token budget in order to sustain the quality of output I'm getting. And it varies unpredictably each time. Beyond token efficiency, we need benchmarks to measure model output quality per token consumed for a diverse set of multi-turn conversation scenarios. Measuring single exchanges is not just synthetic, it's unrealistic. Without good cost/quality trade-off measures, every model upgrade feels like a gamble.
  • therobots927 5mo ago
    That’s the joy of purchasing an intangible and non-deterministic product. The profit margin is completely within the vendor’s control and quality is hard for users to measure.
  • bityard 5mo ago
    The company I work for provides all engineering employees with a Claude subscription. My job isn't writing (much) code, and we have Copilot with MS Office, plus multiple internal AI tools on top of that. So I'm free to do low-stakes experiments on Claude without having to worry about hitting my monthly usage limit. I am finding that for complex tasks, Claude's quality of output varies _tremendously_ with repeated runs of the same model and prompt. For example, last week I wrote up (with my own brain and keyboard) a somewhat detailed plain english spec of a work-related productivity app that I've always wanted but never had the time to write. It was roughly the length of an average college essay. The first thing I asked Claude to do was not write any code, but come up with a more formal design and implementation plan based on the requirements that I gave. The idea was to then hand _that_ to Claude and say, okay, now build it. I used Opus 4.6 with High reasoning for all of this and did not change any model settings between runs. The first run was overall _amazing_. It was detailed, well-written, contained everything that I asked for. The only drawback was that I was ambiguous on a couple of points which meant that the model went off and designed something in a way that I wasn't expecting and didn't intend. So I cleared that up in my prompt, and instead of keeping the context and building on what was already there, I started a new chat and had it start again from scratch. What it wrote the second time was _far_ less impressive. The writing was terse, there was a lot less detail, the pretty dependency charts and various tables it made the first time were all gone. Lots of stuff was underspecified or outright missing. New chat, start again. Similar results as the second run, maybe a bit worse. It also started _writing code_ which was something I told it NOT to do. At this point I'm starting to panic a little because I'm sure I didn't add, "oh, and make it crappy" to the prompt and I was a little angry about not saving the first iteration since it was fairly close to what I had wanted anyway. I decided to try one last time and it finally gave me back something within about 95% of the first run in terms of quality, but with all the problems fixed. So, I was (finally) happy with that, and it used that to generate the application surprisingly well, with only a few issues that should not be too hard to fix after the fact. So I guess 4th time was a charm, and the fare was about $7 in tokens to get there.
• taosx 5mo ago
  Claude seems so frustrating lately to the point where I avoid and completely ignore it. I can't identify a single cause but I believe it's mostly the self-righteousness and leadership that drive all the decisions that make me distrust and disengage with it.
  • estearum 5mo ago
    using dumber models to own the libs
    • testbjjl 5mo ago
      Definitely experimenting with less expensive ones. I have a few versions of my settings.json I also wonder if token utilization has or will ever find its way to employee performance reviews as these models go up in price.
  • QuercusMax 5mo ago
    What do you mean by this? What are you frustrated by? You're offended by their political beliefs, so you don't like the way the model works?
• sysmax 5mo ago
  Well, LLMs are priced per token, and most of the tokens are just echoing back the old code with minimal changes. So, a lot of the cost is actually paying for the LLM to echo back the same code. Except, it's not that trivial to solve. I tried experimenting with asking the model to first give a list of symbols it will modify, and then just write the modified symbols. The results were OK, but less refined than when it echoes back the entire file. The way I see it is that when you echo back the entire file, the process of thinking "should I do an edit here" is distributed over a longer span, so it has more room to make a good decision. Like instead of asking "which 2 of the 10 functions should you change" you're asking it "should you change method1? what about method2? what about method3?", etc., and that puts less pressure on the LLM. Except, currently we are effectively paying for the LLM to make that decision for *every token*, which is terribly inefficient. So, there has to be some middle ground between expensively echoing back thousands of unchanged tokens and giving an error-ridden high-level summary. We just haven't found that middle ground yet.
  • gruez 5mo ago
    >and most of the tokens are just echoing back the old code with minimal changes I thought coding harnesses provided tools to apply diffs so the LLM didn't have to echo back the entire file?
    • sysmax 5mo ago
      They can, but this reduces the quality. The LLM has a harder time picking the first edit, and then all subsequent work is influenced by that one edit. Like first creating an unnecessary auxiliary type, and then being stuck modifying the rest of the code to work with it. So, in practice, many tools still work on the file level.
  • mmastrac 5mo ago
    I think the ideal way for these LLMs to work will be using AST-level changes instead of "let me edit this file". grit.io was working on this years ago, not sure if they are still alive/around, but I liked their approach (just had a very buggy transformer/language).
• ricardobeat 5mo ago
  I can’t stand reading this. One article. Many words. Not written by a human. Feels like LLMs are devolving into having a single, instantly recognizable and predictable writing style.
• aliljet 5mo ago
  This is the reality I'm seeing too. Does this mean that the subscriptions (5x, 10x, 20x) are essentially reduced in token-count by 20-30%?
  • aray07 5mo ago
    yeah thats the part that is unclear to me as well - if our usage capacity is now going to run out faster.
    • AndyNemmity 5mo ago
      The same thing I've been doing all the time, now has used up 1/3rd of my week in one day on max20. So yes, for the same tasks, usage runs out faster (currently)
  • cbg0 5mo ago
    Boris said on Twitter that they've increased rate limits for everyone.
• texttopdfnet 5mo ago
  [dead]
• texttopdfnet 5mo ago
  [dead]
• Bingolotto 5mo ago
  Talked to Claude earlier today and Opus 4.7 cost up to 35% more.
• technotony 5mo ago
  Not only that but they seem to have cut my plan ability to use Sonnet too. I have a routine that used to use about 40% of my 5 hour max plan tokens, then since yesterday it gets stopped because it uses the whole 100%. Anyone else experience this?
  • mfro 5mo ago
    yeah it seems like sonnet 4.6 burns thru tokens crazy fast. I did one prompt, sonnet misunderstood it as 'generate an image of this' and used all of my free tokens.
• jmward01 5mo ago
  Claude code seems to be getting worse on several fronts and better on others. I suspect product is shifting from 'make it great' to 'make it make as much money for us as possible and that includes gathering data'. Recently it started promoting me for feedback even though I am on API access and have disabled this. When I did a deep dive of their feedback mechanism in the past (months ago so probably changed a lot since then) the feedback prompt was pushing message ids even if you didn't respond. If you are on API usage and have told them no to training on your data then anything pushing a message id implies that it is leaking information about your session. It is hard to keep auditing them when they push so many changes so I am now 'default they are stealing my info' instead of believing their privacy/data use policy claims. Basically, my level of trust is eroding fast in their commitment to not training on me and I am paying a premium to not have that happen.
• speedgoose 5mo ago
  The "multiplier" on Github Copilot went from 3 to 7.5. Nice to see that it is actually only 20-30% and Microsoft wanting to lose money slightly slower. https://docs.github.com/fr/copilot/reference/ai-models/supported-models#model-multipliers https://docs.github.com/fr/copilot/reference/ai-models/suppo...
  • Someone1234 5mo ago
    Yep, and I just made a recommendation that was essentially "never enable Opus 4.7" to my org as a direct result. We have Opus 4.6 (3x) and Opus 4.5 (3x) enabled currently. They are worth it for planning. At 7.5x for 4.7, heck no. It isn't even clear it is an upgrade over Opus 4.6.
    • bwat49 5mo ago
      in copilot I find it hard to justify using opus at even 3x vs just using GPT 5.4 high at 1x
      • d0100 5mo ago
        I went from plan with opus, implement with claude, to simply plan and implement with GPT 5.4 It's a very good model for a very good price
        • WithinReason 5mo ago
          What is "claude"?
    • GaryBluto 5mo ago
      Microsoft are going to be removing Opus 4.5 and 4.6 from Copilot soon so I'd enjoy the lower cost while it lasts.
      • deleted 5mo ago
        [deleted]
• thibran 5mo ago
  For me there is no point in using Claude Opus 4.7, it's too expensive since it does not do 100% of the job. Since AI can anyway only do 90% of most tasks, I can use another model and do the remaining 15-30% myself.
• therobots927 5mo ago
  As a regular listener of Ed Zitron this comes as absolutely no surprise. Once you understand the levels of obfuscation available to anthro / OAI you will realize that they have almost certainly hit a model plateau ~1 year ago. All benchmark improvements since have come at a high compute cost. And the model used when evaluating said benchmarks is not the same model you get with your subscription. This is already becoming apparent as users are seeing quality degrade which implies that anthropic is dropping performance across the board to minimize financial losses.
• montjoy 5mo ago
  It appears that they are testing using Max. For 4.7 Anthropic recognizes the high token usage of max and recommends the new xhigh mode for most cases. So I think the real question is whether 4.7 xhigh is “better” than 4.6 max. > max: Max effort can deliver performance gains in some use cases, but may show diminishing returns from increased token usage. This setting can also sometimes be prone to overthinking. We recommend testing max effort for intelligence-demanding tasks. > xhigh (new): Extra high effort is the best setting for most coding and agentic use cases Ref: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices https://platform.claude.com/docs/en/build-with-claude/prompt...
  • dcrazy 5mo ago
    Inserting an xhigh tier and pushing max way out has very “these go to 11” vibes.
• omega3 5mo ago
  Contrary to people here who feel the price increases, reduction of subscription limits etc are the result of the Anthropic models being more expensive to run than the API & subscription revenue they generate I have a theory that Anthropic has been in the enshittification & rent seeking phase for a while in which they will attempt to extract as much money out of existing users as possible. Commercial inference providers serve Chinese models of comparable quality at 0.1x-0.25x. I think Anthropic realised that the game is up and they will not be able to hold the lead in quality forever so it's best to switch to value extraction whilst that lead is still somewhat there.
  • CharlesW 5mo ago
    > Commercial inference providers serve Chinese models of comparable quality… "Comparable" is doing some heavy lifting there. Comparable to Anthropic models in 1H'25, maybe.
    • omega3 5mo ago
      Benchmarks suggests they are comparable: https://artificialanalysis.ai/?models=claude-opus-4-6-adaptive%2Cminimax-m2-7%2Ckimi-k2-5%2Cglm-5-1#artificial-analysis-intelligence-index https://artificialanalysis.ai/?models=claude-opus-4-6-adapti... But let's say for the sake of discussion Opus is much better - still doesn't justify the price disparity especially when considering that other models are provided by commercial inference providers and anthropics is inhouse.
      • xienze 5mo ago
        > Benchmarks suggests they are comparable The problem here is people think AI benchmarks are analogous to say, CPU performance benchmarks. They're not: * You can't control all the variables, only one (the prompt). * The outputs, BY DESIGN, can fluctuate wildly for no apparent reason (i.e., first run, utter failure, second run, success). * The biggest point, once a benchmark is known, future iterations of the model will be trained on it. Trying to objectively measure model performance is a fool's errand.
      • 5mo ago
• ndom91 5mo ago
  `/model claude-opus-4-6`
• rbren 5mo ago
  Good reminder to choose model-agnostic tooling!
• yuanzhi1203 5mo ago
  We noticed this two weeks ago where we found some of our requests are unexpected took more tokens than measured by count_tokens call. At the end they were Anthropic's A/B testing routing some Opus 4.6 calls to Opus 4.7. https://matrix.dev/blog-2026-04-16.html https://matrix.dev/blog-2026-04-16.html (We were talking to Opus 4.7 twelve days ago)
  • ec109685 5mo ago
    Wonder what they do for their token cache if they swap mid-session like that.
    • yuanzhi1203 5mo ago
      Exactly! In https://matrix.dev/blog-2026-04-04-2.html#questions-this-raises https://matrix.dev/blog-2026-04-04-2.html#questions-this-rai..., we raised exactly the same concerns. In particular, we actually saw that a hot swap caused a 100% cache miss. If it's a session filled with 800k tokens, rebuilding the cache is very expensive. Also looking back at their claim: "Token counts may include tokens added automatically by Anthropic for system optimizations. You are not billed for system-added tokens. Billing reflects only your content." A/B testing sounds a bit different. Do they really count it as "system-added tokens" and not charge for this extra cost? If you consider the model you're requesting as the baseline, then yes. But technically it's an A/B test of a different model, so they might secretly charge 130% as "we didn't add any system prompt, we just routed you to a better model."
• khalic 5mo ago
  Just hit my quota with 20x for the first time today…
• DiscourseFan 5mo ago
  Yeah I noticed today, I had it work up a spreadsheet for me and I only got 3 or 4 turns in the conversation before it used up all my (pro) credits. It wasn't even super-complicated or anything, only moderately so.
• margorczynski 5mo ago
  It doesn't look good for Anthropic, especially considering they are burning billions in investor money. Looks like they lost the mandate of heaven, if Open AI plays it right it might be their end. Add to that the open source models from China.
  • therobots927 5mo ago
    OpenAI is dealing with exactly the same energetic and financial constraints as Anthropic. That will become apparent soon.
  • throwaway041207 5mo ago
    I work at a company that has gone all in on Anthropic, and we're just shoveling money at them. I suspect there are a more enterprises than we realize that are doing this. When I read these comments on Hacker News, I see a lot of people miffed about their personal subscription limits. I think this is a viewpoint that is very consumer focused, and probably within Anthropic they're seeing buckets of money being dumped on them from enterprises. They probably don't really care as much about the individual subscription user, especially power users.
  • solenoid0937 5mo ago
    1. HN is so unrepresentative of real life. You have people on their $20/$200 subscriptions complaining about usage limits. They are a tiny fraction of Anthropic's revenue. API billing and enterprise is where the money is. 2. Anthropic and OpenAI's financials are totally different. The former has nearly the same RRR and a fraction of the cash burn. There is a reason Anthropic is hot on secondary and OAI isn't
• kburman 5mo ago
  Anthropic must be loving it. It's free money.
• redml 5mo ago
  It does cost more but I found the quality of output much higher. I prefer it over the dumbing of effort/models they were doing for the last two months. They have to get users used to picking the appropriate model for their task (or have an automatic mode - but still let me force it to a model).
• rambojohnson 5mo ago
  So intelligence has turned into a utility per Sam Altman et al., and now the same companies get to hike the price of accessing it by 20–30%, right as it’s becoming the backbone of how teams actually ship work. People are pushing out so much, so fast that last week’s output is already a blur. I’ve got colleagues who refuse to go back to writing any of this stuff by hand. And now maintaining that pace means absorbing arbitrary price increases, shrugged off with “we were operating at a loss anyway.” It stops being “pay to play” and starts looking more like pay just to stay in the ring, while enterprise players barely feel the hit and everyone else gets squeezed out. Market maturing my butthole... it’s obviously a dependency being priced in real time. Tech is an utter shit show right now, compounded by the disaster of the unemployment market still reeling from the overhiring of 2020. save up now and career pivot. pick up gardening.
  • colechristensen 5mo ago
    Like every startup ever, they were selling it to you at a loss to compete for market share and are slowly increasing pricing. Duh.
    • wslh 5mo ago
      But the unique thing about AI is that the "world" is depending on it like water, oil, gas, etc. Not just a specific use case.
      • colechristensen 5mo ago
        So it should be free? What's your point exactly?
        • wslh 5mo ago
          No, that's not my point. My point is that AI looks like something fairly unique in today's landscape: a resource that almost everyone is starting to depend on. It's a bit like the Internet, except usage is metered, and paying more can improve the quality of the result for the same underlying task, such as cybersecurity. In this context I also imagine we will have greater and greater local models, and the (dependency) ending game is completely unclear.
• saltyoldman 5mo ago
  I was sort of hoping that the peak is something like $15 per hour of vibe help (yes I know some of you burn $15 in 12milliseconds), and that you can have last year's best or the current "nano/small" model at $1 per hour. But it looks like it's just creeping up. Probably because we're paying for construction, not just inference right now.
• deleted 5mo ago
  [deleted]
• bugsense 5mo ago
  I would use a service like Straion.com to avoid the forths and back. It increases token consumption but I can get things right the first time.
• JohnMakin 5mo ago
  30% more token use, but even by their benchmarks, don't appear to have any real big successes there, and some regressions. What's the point? It doesn't do any better on the suite of obedience/compliance tests I've written for 4.6, and in some tests, got worse, despite their claim there it is better. Anecdotally, it was gobbling so many tokens on even the simplest queries I immediately shut it off and went back to 4.5. Why release this?
• JimmaDaRustla 5mo ago
  Am I dumb, or are they not explaining what level thinking they're using? We all read the Anthropic blog post yesterday - 4.7 max consumes/produces an incredible number of tokens and it's not equivalent to 4.6 max; xhigh is the new "max".
• synergy20 5mo ago
  that's what i feel, going to use codex more
• avereveard 5mo ago
  Well yeah it was disclosed here https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2Fff97ab0f2a5f3a243da02398f97dec1ac99b526a-3840x2160.png&w=3840&q=75 https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-... high is the new xhigh
• storytellera 5mo ago
  [dead]
• wartywhoa23 5mo ago
  Seeing this big crowd of people trying to persuade themselves or others that the ever growing hole in their pockets is totally justified and beneficial is pretty hilarious!
• jstummbillig 5mo ago
  "One session" is not a very interesting unit of work. What I am interested in is how much less work I am required to do, to get the results I want. This is not so much about my instructions being followed more closely. It's the LLM being smarter about what's going on and for example saving me time on unnecessary expeditions. This is where models have been most notably been getting better to my experience. Understanding the bigger picture. Applying taste. It's harder to measure, of course, but, at least for my coding needs, there is still a lot of room here. If one session costs an additional 20% that's completely fine, if that session gets me 20% closer to a finished product (or: not 20% further away). Even 10% closer would probably still be entirely fine, given how cheap it is.
• climike 5mo ago
  [dead]
• AIrtemis 5mo ago
  here comes the rug-pull
• AIrtemis 5mo ago
  here comes the rug-pull to justify the enterprise pricing...
• motbus3 5mo ago
  I've been using 4.6 models since each of them launched. Same for 4.5. 4.6 performers worse or the same in most of the tasks I have. If there is a parameter that made me use 4.6 more frequently is because 4.5 get dumber and not because 4.6 seemed smarter.
  • nwienert 5mo ago
    Agree on all counts, 4.5 was a monster, 4.6 a clear regression, and then 4.5 was dumbed down so I moved on.
• admiralrohan 5mo ago
  In Kolkata, sweet sellers was struggling with cost management after covid due to increased prices of raw materials. But they couldn't increase the price any further without losing customers. So they reduced the size of sweets instead, and market slowly reduced expectations. And this is the new normal now. Human psychology is surprisingly similar, and same pattern comes across domains.
  • hirako2000 5mo ago
    It's not just in Kolkata, worldwide packs of biscuits etc remained the same size but less inside. I didn't buy Springles chips in years, even the box now is nothing like it was. Thinner. Shorter. I imagine how far from the top the slices stack up.
  • steelbrain 5mo ago
    See also: Shrinkflation (https://en.wikipedia.org/wiki/Shrinkflation https://en.wikipedia.org/wiki/Shrinkflation)
    • admiralrohan 5mo ago
      Wow, wasn't aware that this has a official term. Thanks for sharing.
      • 32rewr 5mo ago
        [flagged]
  • deleted 5mo ago
    [deleted]
• outlore 5mo ago
  I can manage session cost effectively myself if forking and rewinds were first class features
• socratic_weeb 5mo ago
  This is good news. It means the bubble is popping. Bye bye VC subsidies...
• epistasis 5mo ago
  Anybody else having problem getting Opus 4.7 to write code? I had it pick up a month-old project, some small one off scripts that I want to modify, and it refused to even touch the code. So far it costs a lot less, because I'm not going to be using it.
  • deleted 5mo ago
    [deleted]
  • mrtesthah 5mo ago
    No, see, we have to leave writing code to fully identity-verified individuals working on behalf of only the largest institutions now because what if they decided to write malware?
  • apelapan 5mo ago
    On the contrary, I threw a multi-threading optimization task on it, that 4.5 and 4.6 have been pretty useless at handling. 4.7 bested my hand-tuned solution by almost 2x on first attempt. This was what I thought was my best moat as a senior dev. No other model has been able to come close to the throughput I could achieve on my own before. Might be a fluke of course, and they've picked up a few patterns in training that applies to this particular problem and doesn't generalize. We'll see.
    • epistasis 5mo ago
      Good to hear! My experience with code and 4.7 is still "I won't touch your python scripts because of my malware system instruction." With other chats the tool usage is through the roof with Opus 4.7 with mediocre results after much longer latency. I'll try again in a few days...
  • GeoAtreides 5mo ago
    > it refused to even touch the code. ??? please i beg post the prompt and the refusal I literally can not imagine a model refusing to do something
    • 5mo ago
• chakintosh 5mo ago
  Yeah one PRD request of a small scope app cost me 70%
• SamuelBraude 5mo ago
  [dead]
• ericol 5mo ago
  I did some work yesterday with Opus and found it amazing. Today we are almost on non-speaking terms. I'm asking it to do some simple stuff and he's making incredible stupid mistakes: This is the third time that I have to ask you to remove the issue that was there for more than 20 hours. What is going on here? and at the same time the compacting is firing like crazy. (What adds ~4 minute delays every 1 - 15 minutes) | # | Time | Gap before | Session span | API calls | |---|----------|-----------|--------------|-----------| | 1 | 15:51:13 | 8s | <1m | 1 | | 2 | 15:54:35 | 48s | 37m | 51 | | 3 | 16:33:33 | 2s | 19m | 42 | | 4 | 16:53:44 | 1s | 9m | 30 | | 5 | 17:04:37 | 1s | 17m | 30 | # — sequential compaction event number, ordered by time. Time — timestamp of the first API call in the resumed session, i.e. when the new context (carrying the compaction summary) was first sent to the model. Gap before — time between the last API call of the prior session and the first call of this one. Includes any compaction processing time plus user think time between the two sessions. Session span — how long this compaction-resumed session ran, from its first API call to its last before the next compaction (or end of session). API calls — total number of API requests made during this resumed session. Each tool use, each reply, each intermediate step = one request. Bottomline, I will probably stay on Sonnet until they fix all these issues.
  • whalesalad 5mo ago
    I am having a shit experience lately. Opus 4.7, max effort. > You're right, that was a shit explanation. Let me go look at what V1 MTBL actually is before I try again. > Got it — I read the V1 code this time instead of guessing. Turns out my first take was wrong in an important way. Let me redo this in English. :facepalm:
    • al_borland 5mo ago
      This seems like the experience I've had with every model I've tried over the last several years. It seems like an inherent limitation of the technology, despite the hyperbolic claims of those financially invested in all of this paying off.
• tabbott 5mo ago
  I find it interesting that folks are so focused on cost for AI models. Human time spent redirecting AI coding agents towards better strategies and reviewing work, remains dramatically more expensive than the token cost for AI coding, for anything other than hobby work (where you're not paying for the human labor). $200/month is an expensive hobby, but it's negligible as a business expense; SalesForce licenses cost far more. The key question is how well it a given model does the work, which is a lot harder to measure. But I think token costs are still an order of magnitude below the point where a US-based developer using AI for coding should be asking questions about price; at current price points, the cost/benefit question is dominated by what makes the best use of your limited time as an engineer.
  • hyraki 5mo ago
    Yes 200 as a business expense is really not that bad. But a hobby is hard to justify.
    • scuff3d 5mo ago
      It's not gonna stay that way. Token cost is being massively subsidized right now. Prices will have to start increasing at some point.
      • Gigachad 5mo ago
        Seems like the real costs and numbers are very hidden right now. It’s all private companies and secret info how much anything costs and if anything is profitable.
        • davikr 5mo ago
          Some say margins could be up to 90% on API inference. The house always wins?
          • Gigachad 5mo ago
            Some could say anything when there’s no proof.
• sageframe 5mo ago
  [dead]
• tornikeo 5mo ago
  Good lord. Reading all these comments makes me feel so much better for dumping anthropic the first time their opus started becoming dumber (circa Month ago). It feels like most people in this thread are somehow bound to Claude, even though it is alread fully enshittfied.
  • therobots927 5mo ago
    Given that they haven’t even gone public yet, doesn’t that seem like putting the cart before the horse a bit? And if they’re already enshittifying it won’t be long until the other placers start doing so as well. Have we passed peak LLM intelligence and are we now watching it degrade as they fail to roll these new advanced models out to their increasing user base? Are the finances not adding up? Lots of questions.
    • 32rewr 5mo ago
      Its quite possible there's some tacit collusion going on - it benefits both OAI and Anthropic to make moves that benefit both if they both intend to go public.
• dionian 5mo ago
  I noticed it was compacting more aggressively which i actually like, because i was letting sessions get really long and using them uncached (parallel sessions)
• TomGarden 5mo ago
  Asked Opus 4.7 to extend an existing system today. After thorough exploration and a long back and forth on details it came up with a plan. Then proceeded to build a fully parallel, incompatible system from scratch with the changes I wanted but everything else incompatible and full of placeholders
• 2001zhaozhao 5mo ago
  To me, all of this seems to be pointing to the future solution being some sort of diffusion-based LLM that can process multiple tokens per pass, while keeping the benefits of more "verbose" token encoding.
• mianzubair 5mo ago
  [flagged]
• memcoder 5mo ago
  depends if you're running Opus for everything vs tiering. my pipeline: Haiku 4.5 for ~70% of implementation, Sonnet 4 for one review step, Opus 4.5 only for planning and final synthesis claude code on opus continuously = whole bill. different measurement. haiku 4.5 is good enough for fanout. opus earns it on synthesis where you need long context + complex problem solving under constraints
• zeronone 5mo ago
  > Only one instruction type moved materially: change_case:english_capital (0/1 → 1/1). Everything else tied. So the new tokenizer costs for English/code is to support SHOUTING in English?
• kevinten10 5mo ago
  [flagged]
• clbrmbr 5mo ago
  How can they change the tokenizer without a wholesale pre-train?
• Frannky 5mo ago
  Give it a try to opencode + mimo V2 pro...
• noisy_boy 5mo ago
  At this point, as an experienced developer, unless they can promise consistent very high quality, which they can't, I would rather lean towards almost as good but faster. At this point, that compromise is Codex. I would rather steer quickly, get ideas because I'm moving quickly, do course correction quickly - basically I'm not happy blocking my chain of thought/concentration and fall prey to distractions due to Claude's slowness and compaction cycles. Sometimes I don't even notice that Codex has compacted. For architectural discussions, sure I'll pick Claude. I'm mentally prepared for that. But once we are in the thick of things, speed matters. I would they rather focus on improving Sonnet's speed.
• joewongg 5mo ago
  [dead]
• olq_plo 5mo ago
  That blog post is full of AI slop. Repeats the same argument a gazillion times. It's not X, it's Y. Awful to read.
• 2001zhaozhao 5mo ago
  FYI: Anthropic increased people's subscription quotas to counteract the token cost change. In classic Anthropic fashion this is only announced via X post and not any official announcement. However, if you are using API costs then I guess you're left holding the bag.
• blurbleblurble 5mo ago
  4.7 has been incredibly frustrating vs 4.6. Not sure what's going on but it keeps dropping stuff and getting stuck in weird side quests. Hope it gets fixed cause 4.6 was awesome.
• e1ghtSpace 5mo ago
  Do they ever make AIs that are super rediculously expensive to run but get really good scores on tests, and aren't for consumers? Like drag racing for AI?
  • m00x 5mo ago
    Mythos is basically this
• foreman_ 5mo ago
  [flagged]
• sho 5mo ago
  Taking the article's 5% accuracy improvement at face value: if true, then it's more than worth the token inflation IMO. That's because of tool call chains, where errors compound and accumulate, and small improvements in accuracy get greatly magnified. Again, the article's numbers are likely a rather crude approximation, but taking 85% accuracy (claude 4.6) vs 90% (4.7) as inputs: 4.6 1 iteration 85% 4.7 1 iteration 90% 4.6 5 iterations 44.37% 4.7 5 iterations 59.85% 4.6 10 iterations 19.69% 4.7 10 iterations 34.87% Compounded, small improvements really move the needle downstream. 1.4x doesn't seem worth it for 5% better, but 10 calls in, that's more than a 40% improvement.
  • rohansood15 5mo ago
    You're assuming errors cannot be retried/recovered. They can.
    • sho 5mo ago
      You're assuming errors are a clear failure that can be identified and retried, rather than a silent drift from user intent that simply feeds bad but well-formed results into the next step. They're not. Well, of course sometimes they are, but the much more insidious failure mode is doing the wrong thing in the right way.
• Unsponsoredio 5mo ago
  [dead]
• EthanFrostHI 5mo ago
  [flagged]
• greatgib 5mo ago
  What annoys me the most with the proprietary side of Gemini and Claude is that you used to have the tokenizer (standard) and open sourced. So you could understand what was going on, how the model would understand/split the tokens. Now it is trade secret only usable through the api!
• bilekas 5mo ago
  > The model will not silently generalize an instruction from one item to another. I am clearly missing something but wouldn't this be an ideal thing to do? Surely if it was optimised it would use less tokens while not losing anything from the instructions ?
• sergiopreira 5mo ago
  [dead]
• Sergey777 5mo ago
  [dead]
• kinnth 5mo ago
  It feels like a dedicated orchestration/planning agent needs to be much clearer on costs now as part of the tast plan. Multiple models used at different stages depending on the task.
• SpyCoder77 5mo ago
  This begs the question: should we translate our prompts into CJK and translate the output back into English?
• shtofadhor 5mo ago
  [dead]
• sarpdag 5mo ago
  Since the Opus 4.7 release. I hit my 5 hour window limit second time on claude code max plan, which never happened before. I am not happy for sure.
• ardline 5mo ago
  This is the kind of thing that looks simple until you're three layers deep in edge cases.
• Futurmix 5mo ago
  [flagged]
• Abderahmane 5mo ago
  Bonjour