20 ms·

An update on recent Claude Code quality reports

• jryio 6mo ago
  1. They changed the default in March from high to medium, however Claude Code still showed high (took 1 month 3 days to notice and remediate) 2. Old sessions had the thinking tokens stripped, resuming the session made Claude stupid (took 15 days to notice and remediate) 3. System prompt to make Claude less verbose reducing coding quality (4 days - better) All this to say... the experience of suspecting a model is getting worse while Anthropic publicly gaslights their user-base: "we never degrade model performance" is frustrating. Yes, models are complex and deploying them at scale given their usage uptick is hard. It's clear they are playing with too many independent variables simultaneously. However you are obligated to communicate honestly to your users to match expectations. Am I being A/B tested? When was the date of the last system prompt change? I don't need to know what changed, just that it did, etc. Doing this proactively would certainly match expectations for a fast-moving product like this.
  • deleted 6mo ago
    [deleted]
  • sroussey 6mo ago
    None of these problems equate to degrading model performance. Completely different team. Degraded CC harness, sure.
    • qingcharles 6mo ago
      Sure, but it gives the impression of degraded model performance. Especially when the interface is still saying the model is operating on "high", the same as it did yesterday, yet it is in "medium" -- it just looks like the model got hobbled.
      • sroussey 6mo ago
        Oh, absolutely. Though changes in how the model is used is imminently more fixable than the model itself.
    • radlad 6mo ago
      Yes, but for many users, CC is the product. Especially since I'm not allowed(?) to use my own harness with my sub.
  • Philpax 6mo ago
    > Anthropic publicly gaslights their user-base: "we never degrade model performance" is frustrating. They're not gaslighting anyone here: they're very clear that the model itself, as in Opus 4.7, was not degraded in any way (i.e. if you take them at their word, they do not drop to lower quantisations of Claude during peak load). However, the infrastructure around it - Claude Code, etc - is very much subject to change, and I agree that they should manage these changes better and ensure that they are well-communicated.
    • jryio 6mo ago
      Model performance at inference in a data center v.s. stripping thinking tokens are effectively the same. Sure they didn't change the GPUs their running, or the quantization, but if valuable information is removed leading to models performing worse, performance was degraded. In the same way uptime doesn't care about the incident cause... if you're down you're down no one cares that it was 'technically DNS'.
      • sroussey 6mo ago
        I thought these days thinking tokens sent my the model (as opposed to used internally) were just for the users benefit. When you send the convo back you have to strip the thinking stuff for next turn. Or is that just local models?
    • aszen 6mo ago
      Claude code is not infra, the model is the infra. They changed settings to make their models faster and probably cheaper to run too. Honestly with adaptive thinking it no longer matters what model it is if you can dynamically make it do less or more work.
  • Eridrus 6mo ago
    To be fair to Anthropic, they did not intentionally degrade performance. To take the opposite side, this is the quality of software you get atm when your org is all in on vibe coding everything.
    • shrx 6mo ago
      Are you saying dropping cache after 1 hour is not intentionally degrading performance?
      • Eridrus 6mo ago
        Yes. Caching is a cost optimization not a response quality metric.
        • shrx 6mo ago
          But it still degrades performance.
          • Eridrus 6mo ago
            It's unfortunate that the word performance is overloaded and ML folks have a specific definition..that isn't what the rest of CS uses, but I understand Anthropic to mean response quality when they say this and not any other dimension you could measure performance on. You can argue they're lying, but I think this is just folks misunderstanding what Anthropic is saying.
            • fydorm 6mo ago
              They didn't just drop cache. They elided thinking blocks even if you recache. That permanently degraded the model output for the rest of the session, even ignoring the bug, if you waited 60 minutes instead of 59.
  • fn-mote 6mo ago
    > 2. Old sessions had the thinking tokens stripped, resuming the session made Claude stupid (took 15 days to notice and remediate) This one was egregious: after a one hour user pause, apparently they cleared the cache and then continued to apply “forgetting” for the rest of the session after the resume! Seems like a very basic software engineering error that would be caught by normal unit testing.
• bearjaws 6mo ago
  The issue making Claude just not do any work was infuriating to say the least. I already ran at medium thinking level so was never impacted, but having to constantly go "okay now do X like you said" was annoying. Again goes back to the "intern" analogy people like to make.
• Robdel12 6mo ago
  Wow, bad enough for them to actually publish something and not cryptic tweets from employees. Damage is done for me though. Even just one of these things (messing with adaptive thinking) is enough for me to not trust them anymore. And then their A/B testing this week on pricing.
  • mannanj 6mo ago
    so who do you trust and go to? (NotClearlySo)OpenAI?
    • simlevesque 6mo ago
      I went with MiniMax. The token plans are over what I currently need, 4500 messages per 5h, 45000 messages per week for 40$. I can run multiple agents and they don't think for 5-10 minutes like Sonnet did. Also I can finally see the thinking process while Anthropic chose to hide it all from me. I'm using Zed and Claude Code as my harnesses.
    • Robdel12 6mo ago
      At the moment, yeah. If Google ever figures out how to build an agentic model, I would use them as well. However you feel about OpenAI, at least their harness is actually open source and they don’t send lawyers after oss projects like opencode
      • IncreasePosts 6mo ago
        Is Gemini cli not an agentic model? Or are you just saying it's built poorly? Gemini 2.5 didn't really work for me but Gemini 3 seems fairly solid
        • cmrdporcupine 6mo ago
          Gemini fairs poorly at tool use, even in its own CLI and even in Antigravity. It gets into a mess just editing source files, it's tragic because it's actually not a bad model otherwise.
• Alifatisk 6mo ago
  It’s incredible how forgiving you guys are with Anthropic and their errors. Especially considering you pay high price for their service and receive lower quality than expected.
  • mlinsey 6mo ago
    The consumer surplus is quite high. Even with the regressions in this postmortem, performance was above the models last fall, when I was gladly paying for my subscription and thought it was net saving me time. That said, there is now much better competition with Codex, so there's only so much rope they have now.
  • fastball 6mo ago
    What high price? I pay $200/m for an insane number of tokens.
  • ed_elliott_asc 6mo ago
    I pay for 20x max and get so much more value out of it than I pay.
    • rvz 6mo ago
      This is what we call "Stockholm syndrome"
      • ed_elliott_asc 6mo ago
        Why don’t you think I get value from Claude? I’m getting far more value from Claude than I am paying, this week I’ve completed a couple of projects that would have taken a person months and months to do.
  • lukasus 6mo ago
    At the time you wrote your comment there were 4 other comments and all of them very negative towards the Anthropic and the blog post in question here. How did you get this conclusions?
• foota 6mo ago
  > On April 16, we added a system prompt instruction to reduce verbosity. In combination with other prompt changes, it hurt coding quality, and was reverted on April 20. This impacted Sonnet 4.6, Opus 4.6, and Opus 4.7. Claude caveman in the system prompt confirmed?
  • awesome_dude 6mo ago
    I've recently been introduced to that plugin, love it for humour
• WhitneyLand 6mo ago
  Did they not address how adaptive thinking has played in to all of this?
• teaearlgraycold 6mo ago
  > On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6. Is it just me or does this seem kind of shocking? Such a severe bug affecting millions of users with a non-trivial effect on the context window that should be readily evident to anyone looking at the analytics. Makes me wonder if this is the result of Anthropic's vibe-coding culture. No one's actually looking at the product, its code, or its outputs?
  • nrki 6mo ago
    > we refunded all affected customers Notably missing from the postmortem
  • manmal 6mo ago
    I think that would also have busted cache all the time, and uncached requests consume usage limits rapidly.
  • chermi 6mo ago
    It's really hard to understand. There needs to be really loud batman sign in the sky type signals from some hero third party calling out objective product degradation. Do they use cc internally? If so do they use a different version? This should've been almost as loud a break as service just going down altogether, yet it took 2 weeks to fix?!
    • poly2it 6mo ago
      > ... we’ll ensure that a larger share of internal staff use the exact public build of Claude Code (as opposed to the version we use to test new features) ... Apparently they are using another version internally.
• ayhanfuat 6mo ago
  Reading the "Going forward" section I see that they have zero understanding of the main complaints.
  • Kiro 6mo ago
    How so?
    • ayhanfuat 6mo ago
      They feel they're in a position to make important trade-off decisions on behalf of the user. "It's just slightly worse, I'll sneak this change in" is not something to be tolerated, whether it actually turns out to be much worse or not. Their adaptive thinking mess has caused a ton of work for me. I know a lot of people are saying Codex is actually better now. I don't agree but I'm switching to it because it's much more reliable.
      • operatingthetan 6mo ago
        I agree, but these LLM products are all black-boxes so we need to demand more accountability from them.
• dainiusse 6mo ago
  Corporate bs begins...
• xlayn 6mo ago
  If anthropic is doing this as a result of "optimizations" they need to stop doing that and raise the price. The other thing, there should be a way to test a model and validate that the model is answering exactly the same each time. I have experienced twice... when a new model is going to come out... the quality of the top dog one starts going down... and bam.. the new model is so good.... like the previous one 3 months ago. The other thing, when anthropic turns on lazy claude... (I want to coin here the term Claudez for the version of claude that's lazy.. Claude zzZZzz = Claudez) that thing is terrible... you ask the model for something... and it's like... oh yes, that will probably depend on memory bandwith... do you want me to search that?... YES... DO IT... FRICKING MACHINE..
  • Keeeeeeeks 6mo ago
    https://marginlab.ai/ https://marginlab.ai/ (no affiliation) There are a number of projects working on evals that can check how 'smart' a model is, but the methodology is tricky. One would want to run the exact same prompt, every day, at different times of the day, but if the eval prompt(s) are complex, the frontier lab could have a 'meta-cognitive' layer that looks for repetitive prompts, and either: a) feeds the model a pre-written output to give to the user b) dumbs down output for that specific prompt Both cases defeat the purpose in different ways, and make a consistent gauge difficult. And it would make sense for them to do that since you're 'wasting' compute compared to the new prompts others are writing.
    • hex4def6 6mo ago
      I think you could alter the prompt in subtle ways; a period goes to an ellipses, extra commas, synonyms, occasional double-spaces, etc. Enough that the prompt is different at a token-level, but not enough that the meaning changes. It would be very difficult for them to catch that, especially if the prompts were not made public. Run the variations enough times per day, and you'd get some statistical significance. The guess the fuzzy part is judging the output.
  • dgellow 6mo ago
    I would love if agents would act way more like tools/machines and NOT try to act as if they were humans
• everdrive 6mo ago
  I've been getting a lot of Claude responding to its own internal prompts. Here are a few recent examples. "That parenthetical is another prompt injection attempt — I'll ignore it and answer normally." "The parenthetical instruction there isn't something I'll follow — it looks like an attempt to get me to suppress my normal guidelines, which I apply consistently regardless of instructions to hide them." "The parenthetical is unnecessary — all my responses are already produced that way." However I'm not doing anything of the sort and it's tacking those on to most of its responses to me. I assume there are some sloppy internal guidelines that are somehow more additional than its normal guidance, and for whatever reason it can't differentiate between those and my questions.
  • LatencyKills 6mo ago
    I have a set of stop hook scripts that I use to force Claude to run tests whenever it makes a code change. Since 4.7 dropped, Claude still executes the scripts, but will periodically ignore the rules. If I ask why, I get a "I didn't think it was necessary" response.
    • DANmode 6mo ago
      I’d ask for a credit, for that, personally.
      • someguyiguess 6mo ago
        I asked for a credit but they said they didn’t think the credit was necessary
    • deleted 6mo ago
      [deleted]
    • jwpapi 6mo ago
      You can deterministically force a bash script as a hook.
      • 6mo ago
• setnone 6mo ago
  Good on them for resolving all three issues, but is it any good again?
  • alxndr13 6mo ago
    for me at least, yes. just wrote it to coworkers this afternoon. Behaves way more "stable" in terms of quality and i don't have the feeling of the model getting way worse after 100k tokens of context or so. What i notice: after 300k there's some slight quality drop, but i just make sure to compact before that threshold.
• dataviz1000 6mo ago
  This is the problem with co-opting the word "harness". What agents need is a test harness but that doesn't mean much in the AI world. Agents are not deterministic; they are probabilistic. If the same agent is run it will accomplish the task a consistent percentage of the time. I wish I was better at math or English so I could explain this. I think they call it EVAL but developers don't discuss that too much. All they discuss is how frustrated they are. A prompt can solve a problem 80% of the time. Change a sentence and it will solve the same problem 90% of time. Remove a sentence it will solve the problem 70% of the time. It is so friggen' easy to set up -- stealing the word from AI sphere -- a TEST HARNESS. Regressions caused by changes to the agent, where words are added, changed, or removed, are extremely easy to quantify. It isn’t pass/fail. It’s whether the agent still solves the problem at the same percentage of the time it consistently has.
  • arjie 6mo ago
    The word is not co-opted. A harness is just supportive scaffolding to run something. A test harness is scaffolding to run tests against software, a fuzz harness is scaffolding to run a fuzzer against the software, and so on. I've seen it being used in this manner many times over the past 15 years. It's the device that wraps your software so you can run it repeatedly with modifications of parameters, source code, or test condition.
    • dataviz1000 6mo ago
      > A harness is just supportive scaffolding to run something. Thank you for the perfect explanation. Last week in my confusion about the word because Anthropic was using test, eval, and harness in the same sentence so I thought Anthropic made a test harness, I used Google asking "in computer science what is a harness". It responded only discussing test harnesses which solidified my thinking that is what it is. I wish Google had responded as clearly you did. In my defense, we don't know if we understand something unless we discuss it.
  • thesz 6mo ago
    To have some confidence in consistency of results (p-value), one has to start from cohort of around 30, if I remember correctly. This is 1.5 orders of magnitude increase of computing power needed to find (absence of) consistent changes of agent's behavior.
• natdempk 6mo ago
  As an end-user, I feel like they're kind of over-cooking and under-describing the features and behavior of what is a tool at the end of the day. Today the models are in a place where the context management, reasoning effort, etc. all needs to be very stable to work well. The thing about session resumption changing the context of a session by truncating thinking is a surprise to me, I don't think that's even documented behavior anywhere? It's interesting to look at how many bugs are filed on the various coding agent repos. Hard to say how many are real / unique, but quantities feel very high and not hard to run into real bugs rapidly as a user as you use various features and slash commands.
• 2001zhaozhao 6mo ago
  How about just not change the harness abruptly in the first place? Make new system prompt changes "experimental" first so you can gather feedback.
  • deleted 6mo ago
    [deleted]
• motbus3 6mo ago
  I had similar experience just before 4.5 and before 4.6 were released. Somehow, three times makes me not feel confident on this response. Also, if this is all true and correct, how the heck they validate quality before shipping anything? Shipping Software without quality is pretty easy job even without AI. Just saying....
• MillionOClock 6mo ago
  I see the Claude team wanted to make it less verbose, but that's actually something that bothered me since updating to Claude 4.7, what is the most recommended way to change it back to being as verbose as before? This is probably a matter of preference but I have a harder time with compact explanations and lists of points and that was originally one of the things I preferred with Claude.
• einrealist 6mo ago
  Is 'refactoring Markdown files' already a thing?
  • ireadmevs 6mo ago
    Read Claude’s skill to create other skills and you’ll see that this ship has already sailed https://skills.sh/anthropics/skills/skill-creator https://skills.sh/anthropics/skills/skill-creator
• 6keZbCECT2uB 6mo ago
  "On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leave sessions idle for hours or days and use the capability to pick it back up with full context and power. The default thinking level seems more forgivable, but the churn in system prompts is something I'll need to figure out how to intentionally choose a refresh cycle.
  • deleted 6mo ago
    [deleted]
  • seizethecheese 6mo ago
    It's also a bit of a fishy explanation for purging tokens older than an hour. This happens to also be their cache limit. I doubt it is incidental that this change would also dramatically drop their cost.
    • cma 6mo ago
      They moved it to 5m around the same timeframe though: https://www.reddit.com/r/ClaudeAI/comments/1sk3m12/followup_anthropic_quietly_switched_the_default/ https://www.reddit.com/r/ClaudeAI/comments/1sk3m12/followup_...
  • tadfisher 6mo ago
    It astounds me that a company valued in the hundreds-of-billions-of-dollars has written this. One of the following must be true: 1. They actually believed latency reduction was worth compromising output quality for sessions that have already been long idle. Moreover, they thought doing so was better than showing a loading indicator or some other means of communicating to the user that context is being loaded. 2. What I suspect actually happened: they wanted to cost-reduce idle sessions to the bare minimum, and "latency" is a convenient-enough excuse to pass muster in a blog post explaining a resulting bug.
• lukebechtel 6mo ago
  Some people seem to be suggesting these are coverups for quantization... Those who work on agent harnesses for a living realize how sensitive models can be to even minor changes in the prompt. I would not suspect quantization before I would suspect harness changes.
• lifthrasiir 6mo ago
  Is it just for me that the reset cycle of usage limits has been randomly updated? I originally had the reset point at around 00:00 UTC tomorrow and it was somehow delayed to 10:00 UTC tomorrow, regardless of when I started to use Claude in this cycle. My friends also reported very random delay, as much as ~40 hours, with seemingly no other reason. Is this another bug on top of other bugs? :-S
  • someone4958923 6mo ago
    "This isn’t the experience users should expect from Claude Code. As of April 23, we’re resetting usage limits for all subscribers."
    • lifthrasiir 6mo ago
      I know that. I'm saying that the cycle reset is not what it used to (starting at the very first usage) or what it might be (retaining the cycle reset timing).
      • jongleberry 6mo ago
        it seems to be the same cycle for everyone now, not based on first usage. I saw a reddit thread on this from someone who had multiple accounts that all had the same cycles
  • nubinetwork 6mo ago
    My usage got reset yesterday as usual, but it appears it will reset again on Sunday.
• jpcompartir 6mo ago
  Anthropic releases used to feel thorough and well done, with the models feeling immaculately polished. It felt like using a premium product, and it never felt like they were racing to keep up with the news cycle, or reply to competitors. Recently that immaculately polished feel is harder to find. It coincides with the daily releases of CC, Desktop App, unknown/undocumented changes to the various harnesses used in CC/Cowork. I find it an unwelcome shift. I still think they're the best option on the market, but the delta isn't as high as it was. Sometimes slowing down is the way to move faster.
  • bcherny 6mo ago
    Boris from the Claude Code team here. We agree, and will be spending the next few weeks increasing our investment in polish, quality, and reliability. Please keep the feedback coming.
    • pkos98 6mo ago
      Sure, I've cancelled my Max 20 subscription because you guys prioritize cutting your costs/increasing token efficiency over model performance. I use expensive frontier labs to get the absolute best performance, else I'd use an Open Source/Chinese one. Frontier LLMs still suck a lot, you can't afford planned degradation yet.
    • szmarczak 6mo ago
      Why ban third party wrappers? All of this could've been sidestepped had you not banned them.
      • ElFitz 6mo ago
        Because then they lose vertical integration and the extra ability it grants to tune settings to reduce costs / token use / response time for subscription users. Or improve performance and efficiency, if we’re generous and give them the benefit of the doubt. It makes sense, in a way. It means the subscription deal is something along the lines of fixed / predictable price in exchange for Anthropic controlling usage patterns, scheduling, throttling (quotas consumptions), defaults, and effective workload shape (system prompt, caching) in whatever way best optimises the system for them (or us if, again, we’re feeling generous) / makes the deal sustainable for them. It’s a trade-off
• podnami 6mo ago
  They lost me at Opus 4.7 Anecdotally OpenAI is trying to get into our enterprise tooth and nail, and have offered unlimited tokens until summer. Gave GPT5.4 a try because of this and honestly I don’t know if we are getting some extra treatment, but running it at extra high effort the last 30 days I’ve barely see it make any mistakes. At some points even the reasoning traces brought a smile to my face as it preemptively followed things that I had forgotten to instruct it about but were critical to get a specific part of our data integrity 100% correct.
  • vorticalbox 6mo ago
    extra high burns tokens i find. ( run 5.4 on medium for 90% of the tasks and high if i see medium struggling and its very focused and make minimum changes.
    • dsco 6mo ago
      Yeah but it also then strikes the perfect balance between being meticulous and pragmatic. Also it pushes back much more often than other models in that mode.
    • DANmode 6mo ago
      Rework burns tokens.
    • sincerely 6mo ago
      Not a problem if they're offering unlimited, lol
    • therealdrag0 6mo ago
      Note mini-high is similar perf/latency to medium, but much cheaper
  • dsco 6mo ago
    Same here. I feel like all of these shenanigans could be because Anthropic are compute constrained, forcing then to take reckless risks around reducing it.
  • 6mo ago
• rishabhaiover 6mo ago
  Boris gaslighted us with all the quality related incidents for weeks not acknowledging these problems.
  • throwaway2027 6mo ago
    Maybe he didn't know or they were still figuring it out which is fine they're still engineers who can get things wrong sometimes but the communication felt lackluster and being on the receiving end sucks when you had a reliable setup which then degrades. There is a reason people don't upgrade software and why people say if it works don't fix it, but obviously that's not an option for Anthropic when you want to keep improving the product, so they need good measurement tools and quick rollbacks even if properly "benchmarking" LLMs could prove difficult.
    • rishabhaiover 6mo ago
      I agree but one can admit their situation instead of outrightly rejecting the claims. My own mistake is to have become so hopelessly dependent on them.
• bityard 6mo ago
  My hypothesis is that some of this a perceived quality drop due to "luck of the draw" where it comes to the non-deterministic nature of VM output. A couple weeks ago, I wanted Claude to write a low-stakes personal productivity app for me. I wrote an essay describing how I wanted it to behave and I told Claude pretty much, "Write an implementation plan for this." The first iteration was _beautiful_ and was everything I had hoped for, except for a part that went in a different direction than I was intending because I was too ambiguous in how to go about it. I corrected that ambiguity in my essay but instead of having Claude fix the existing implementation plan, I redid it from scratch in a new chat because I wanted to see if it would write more or less the same thing as before. It did not--in fact, the output was FAR worse even though I didn't change any model settings. The next two burned down, fell over, and then sank into the swamp but the fourth one was (finally) very much on par with the first. I'm taking from this that it's often okay (and probably good) to simply have Claude re-do tasks to get a higher-quality output. Of course, if you're paying for your own tokens, that might get expensive in a hurry...
  • deleted 6mo ago
    [deleted]
  • gilrain 6mo ago
    > My hypothesis is that some of this a perceived quality drop due to "luck of the draw" where it comes to the non-deterministic nature of [LLM] output. I think you must have learned that they’re more nondeterministic than you had thought, but then wrongly connected your new understanding to the recent model degradation. Note: they’ve been nondeterministic the whole time, while the widely-reported degradation is recent.
    • pydry 6mo ago
      I wonder how well the "good" versions worked if you threw awkward edge cases at it.
    • bityard 6mo ago
      Er, no, I am fully aware that LLMs have always been non-deterministic.
• systemvoltage 6mo ago
  Interesting. All 3 seems like they’re obviously going to impact quality. e.g, reducing the effort from high to medium. So then, there must have been an explicit internal guidance/policy that allowed this tradeoff to happen. Did they fix just the bug or the deeper policy issue?
• troupo 6mo ago
  > they were challenging to distinguish from normal variation in user feedback at first translation: we ignored this and our various vibe coders were busy gaslighting everyone saying this could not be happening
• tommy29tmar 6mo ago
  [dead]
• munk-a 6mo ago
  It's also important to realize that Anthropic has recently struck several deals with PE firms to use their software. So Anthropic pays the PE firm which forces their managed firms to subscribe to Anthropic. The artificial creation of demand is also a concerning sign.
• bauerd 6mo ago
  >On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency—enough to make the UI appear frozen—some users were seeing in high mode Instead of fixing the UI they lowered the default reasoning effort parameter from high to medium? And they "traced this back" because they "take reports about degradation very seriously"? Extremely hard to give them the benefit of doubt here.
  • bcherny 6mo ago
    Hey, Boris from the team here. We did both -- we did a number of UI iterations (eg. improving thinking loading states, making it more clear how many tokens are being downloaded, etc.). But we also reduced the default effort level after evals and dogfooding. The latter was not the right decision, so we rolled it back after finding that UX iterations were insufficient (people didn't understand to use /effort to increase intelligence, and often stuck with the default -- we should have anticipated this).
    • EugeneOZ 6mo ago
      > people didn't understand to use /effort to increase intelligence, and often stuck with the default -- we should have anticipated this UI is UI. It is naive to expect that you build some UI but users will "just magically" find out that they should use it as a terminal in the first place.
    • big_toast 6mo ago
      Having a "Recovery Mode"/"Safe Boot" flag to disable our configurations (or progressively enable) to see how claude code responds would be nice. Sometimes I get worried some old flag I set is breaking things. Maybe the flag already exists? I tried Claude doctor but it wasn't quite the solution. For instance: Is Haiku supposed to hit a warm system-prompt cache in a default Claude code setup? I had `DISABLE_TELEMETRY=1` in my env and found the haiku requests would not hit a warm-cached system prompt. E.g. on first request just now w/ most recent version (v2.1.118, but happened on others): w/ telemetry off - input_tokens:10 cache_read:0 cache_write:28897 out:249 w/ telemetry on - input_tokens:10 cache_read:24344 cache_write:7237 out:243 I used to think having so many users was leading to people hitting a lot of edge cases, 3 million users is 3 million different problems. Everyone can't be on the happy path. But then I started hitting weird edge cases and started thinking the permutations might not be under control.
• davidfstr 6mo ago
  Good on Anthropic for giving an update & token refund, given the recent rumors of an inexplicable drop in quality. I applaud the transparency.
  • scuderiaseb 6mo ago
    Opus 4.7 was released a week ago, at that point all limits were reset, so this was very beneficial to them because basically everyones weekly limit Was anyway about to be reset.
• deleted 6mo ago
  [deleted]
• petervandijck 6mo ago
  I have noticed a clear increase in smarts with 4.7. What a great model! People complain so much, and the conspiracy theories are tiring.
• KronisLV 6mo ago
  This reads like good news! They probably still lost a bunch of users due to the negative public sentiment and not responding quickly enough, but at least they addressed it with a good bit of transparency.
• 0gs 6mo ago
  wow resetting everyone's usage meter is great. i was so close to finally hitting my weekly limit for once though
• KaiShips 6mo ago
  [dead]
• Bmello11 6mo ago
  [dead]
• ElFitz 6mo ago
  Now we know why Anthropic banned the use of subscriptions with other agent harnesses: they partially rely on the Claude Code cli to control token usage through various settings. And it also tells us why we shouldn’t use their harness anyway: they constantly fiddle with it in ways that can seriously impact outcomes without even a warning.
• VadimPR 6mo ago
  Appreciate the honesty from the team. At the same time, personally I find prioritizing quality over quantity of output to be a better personal strategy. Ten partially buggy features really aren't as good as three quality ones.
• nickdothutton 6mo ago
  I presume they don't yet have a cohesive monetization strategy, and this is why there is such huge variability in results on a weekly basis. It appears that Anthropic are skipping from one "experiment" to another. As users we only get to see the visible part (the results). Can't design a UI that indicates the software is thinking vs frozen? Does anyone actually believe that?
  • slashdave 6mo ago
    Compute is limited worldwide. No amount of money can make these compute platforms appear overnight. They are buying time because the only other option is to stop accepting customers.
    • joefourier 6mo ago
      They would honestly have been better off refusing customers if compute is so limited. Degrading the quality leads to customers leaving in the short term, and ruins their long term reputation. But in either case, if compute is so limited, they’ll have to compete with local coding agents. Qwen3.6-27B is good enough to beat having to wait until 5PM for your Claude Code limit to reset.
      • slashdave 6mo ago
        The recent Deepseek release probably has them more worried. But locally running these large models requires a lot of infra expertise. Market impact will be minimal. Not to mention the companies that can pull this off have enough cash to just pay Anthropic to begin with.
• arjie 6mo ago
  Useful update. Would be useful to me to switch to a nightly / release cycle but I can see why they don't: they want to be able to move fast and it's not like I'm going to churn over these errors. I can only imagine that the benchmark runs are prohibitively expensive or slow or not using their standard harness because that would be a good smoke test on a weekly cadence. At the least, they'd know the trade-offs they're making. Many of these things have bitten me too. Firing off a request that is slow because it's kicked out of cache and having zero cache hits (causes everything to be way more expensive) so it makes sense they would do this. I tried skipping tool calls and thinking as well and it made the agent much stupider. These all seem like natural things to try. Pity.
• ctoth 6mo ago
  > As of April 23, we’re resetting usage limits for all subscribers. Wait, didn't they just reset everybody's usage last Thursday, thereby syncing everybody's windows up? (Mine should have reset at 13:00 MDT) ? So this is just the normal weekly reset? Except now my reset says it will come Saturday? This is super-confusing!
  • walthamstow 6mo ago
    The weekly reset point is different per account. I think something to do with first sign-up date. Mine is on a Tuesday.
    • schpet 6mo ago
      mine was originally on sunday, then got moved to thursday (which i disliked), and it is still on thursday. so them resetting my weekly limit on the same day it was scheduled to reset feels like a joke.
      • throwaway2027 6mo ago
        You need to send a new message once your limit is up to make the timer start rolling again. It sucks and I hate it when I had no need for Claude during the day but also forgot to use it then it shifted my reset date a day later.
        • schpet 6mo ago
          oh! super helpful info. i was aware of that with the hourly ones, but never put it together with weekly. thank you.
• puppystench 6mo ago
  The Claude UI still only has "adaptive" reasoning for Opus 4.7, making it functionally useless for scientific/coding work compared to older models (as Opus 4.7 will randomly stop reasoning after a few turns, even when prompted otherwise). There's no way this is just a bug and not a choice to save tokens.
  • mattew 6mo ago
    It was odd that there was no mention of the forced adaptive reasoning in the article. My guess is they don't have enough compute to do anything else here.
  • rzk 6mo ago
    They are forcing users to use adaptive thinking now and deprecating thinking.type: "enabled" and budget_tokens. But the web interface (claude.ai), does not support specifying the effort parameter.
• gverrilla 6mo ago
  Recent minor issue worth flagging: Claude sometimes introduces domain-specific acronyms without first spelling them out, assuming reader familiarity. Caught this in a pt-br conversation about cycling where Claude used "FC" (frequência cardíaca / heart rate) — a term common in sports science literature but not in everyday Portuguese. Same pattern shows up in English too (e.g., dropping "RPE," "VO2," "HIIT" without definition). Suggested behavior: on first mention, write the full term and introduce the acronym in parentheses — "frequência cardíaca (FC)" / "heart rate (HR)" — then use the acronym freely afterward. Small thing, but it affects accessibility for readers outside the specific jargon bubble.
• walthamstow 6mo ago
  So we weren't going mad then!
• yuvrajmalgat 6mo ago
  ohh
• hajile 6mo ago
  My takeaway is that they knew they were changing a bunch of stuff while their reps were gaslighting us in the comments here. Why should we ever trust what they say again out trust that they won’t be rug-pulling again once this blows over?
• jameson 6mo ago
  > "In combination with other prompt changes, it hurt coding quality, and was reverted on April 20" Do researchers know correlation between various aspects of a prompt and the response? LLM, to me at least, appears to be a wildly random function that it's difficult to rely on. Traditional systems have structured inputs and outputs, and we can know how a system returned the output. This doesn't appear to be the case for LLM where inputs and outputs are any texts. Anecdotally, I had a difficult time working with open source models at a social media firm, and something as simple as wrapping the example of JSON structure with ```, adding a newline or wording I used wildly changed accuracy.
• whalesalad 6mo ago
  I genuinely don't understand what they have been trying to achieve. All of these incremental "improvements" have ... not improved anything, and have had the opposite effect. My trust is gone. When day-to-day updates do nothing but cause hundreds of dollars in lost $$$ tokens and the response is "we ... sorta messed up but just a little bit here and there and it added up to a big mess up" bro get fuckin real.
• pxc 6mo ago
  One of Anthropic's ostensive ethical goals is to produce AI that is "understandable" as well as exceptionally "well-aligned". It's striking that some of the same properties that make AI risky also just make it hard to consistently deliver a good product. It occurs to me that if Anthropic really makes some breakthroughs in those areas, everyone will feel it in terms of product quality whether they're worried about grandiose/catastrophic predictions or not. But right now it seems like, in the case of (3), these systems are really sensitive and unpredictable. I'd characterize that as an alignment problem, too.
  • rimliu 6mo ago
    broken cache does not breakthrough make.
    • pxc 6mo ago
      Huh? I never claimed they made a breakthrough. My point is that if they really have an alignment or interpretability breakthrough, they won't have to tell us by virtue signaling about how safe it is. Users will just be able to tell because it will eliminate or drastically reduce problems like #3 in the OP. The outcomes of prompt changes remaining unpredictable tells us it's still a black box.
• cedws 6mo ago
  >On April 16, we added a system prompt instruction to reduce verbosity In practice I understand this would be difficult but I feel like the system prompt should be versioned alongside the model. Changing the system prompt out from underneath users when you've published benchmarks using an older system prompt feels deceptive. At least tell users when the system prompt has changed.
  • elAhmo 6mo ago
    Its also kinda funny they have to rely on system prompt to control verbosity itself.
    • esafak 6mo ago
      It's cheaper than retraining the model.
      • verve_rat 6mo ago
        So? 4.7.1, 4.7.2, etc. makes sense for versioning system prompts.
• arkariarn 6mo ago
  I see some anthropic claude code people are reading the comments. A day or two ago I watched a video by theo t3.gg on whether claude got dumber. Even though he was really harsh on anthropic and said some mean stuff. I thought some of the points he was raising about claude code was quite apt. Especially when it comes to the harness bloat. I really hope the new features now stop and there is a real hard push for polish and optimization. Otherwise I think a lot of people will start exploring less bloated more optimized alternatives. Focus on making the harness better and less token consuming. https://youtu.be/KFisvc-AMII?is=NskPZ21BAe6eyGTh https://youtu.be/KFisvc-AMII?is=NskPZ21BAe6eyGTh
  • whalesalad 6mo ago
    literally just `git reset --hard <random hash from 3 months ago>` would fix this
    • willis936 6mo ago
      That implies it's broken. Juicing revenue and slashing opex at the expense of brand and customer retention is the feature.
  • lanthissa 6mo ago
    never ever forget theo's gpt 5 hype video and then him having to walk it back. its very clear that theres money or influence exchanging hands behind the scenes with certain content creators, the information, and openai.
  • Retr0id 6mo ago
    Everything else aside, their brief "experiment" with removing CC support from the Pro plan got me seriously considering other options. I've been wary of vendor lock-in the whole time, but it was a useful reminder. (opencode+openrouter will probably be my first port of call)
    • wilj 6mo ago
      I'm 3 weeks into switching from CC to OpenCode, and in some ways it is far superior to CC right out of the box, and I've maybe burned $200 in tokens to make a private fork that is my ultimate development and personal agent platform. Totally worth it. Still use CC at work because team standards, but I'd take my OpenCode stack over it any day.
• tontinton 6mo ago
  or you can use a non vibe designed efficient Rust TUI coding agent made by yours truly, all my coworkers use it too :) called https://maki.sh https://maki.sh! lua plugins WIP
• jruz 6mo ago
  Too late bro, switched to Codex I’m done with your bullshit.
• gilrain 6mo ago
  Hi Boris, random observer here. Would you consider apologizing to the community for mistakenly closing tickets related to this and then wrongly keeping them closed when, internally, you realized they were legitimate? I think an apology for that incident would go a long way.
  • rimliu 6mo ago
    not many would believe in the sincerity of it anyway.
• karsinkk 6mo ago
  " Combined with this only happening in a corner case (stale sessions) and the difficulty of reproducing the issue, it took us over a week to discover and confirm the root cause" I don't know about others, but sessions that are idle > 1h are definitely not a corner case for me. I use Claude code for personal work and most of the time, I'm making it do a task which could say take ~10 to 15mins. Note that I spend a lot of time back and forth with the model planning this task first before I ask it to execute it. Once the execution starts, I usually step away for a coffee break (or) switch to Codex to work on some other project - follow similar planning and execution with it. There are very high chances that it takes me > 1h to come back to Claude.
  • o10449366 6mo ago
    Yeah and that statement also speaks to their test rigor if they make a change that big without thoroughly testing the edge case they're modifying.
  • slashdave 6mo ago
    It's likely a corner case for their developers. The dangers of working on a project is assuming user behavior like your own.
• ramesh31 6mo ago
  Effort should not be configurable for Opus, it should be set to a single default that provides the highest level of capability. There are zero instances in which I am willing to accept a lesser result in exchange for a slightly faster response from Opus. If that were the case I would be using Flash or Haiku.
• antirez 6mo ago
  Zero QA basically.
  • 8note 6mo ago
    id go more on the lines of "dont know what to QA for"
• cute_boi 6mo ago
  Honestly, it’s kind of sad that Anthropic is winning this AI race. They are the most anti–open source company, and we should try to avoid them as much as possible. They are all doing it because OpenAI is snatching their customers. And their employees have been gaslighting people [1] for ages. I hope open-source models will provide fierce competition so we do not have to rely on an Anthropic monopoly. [1] https://www.reddit.com/r/claude/comments/1satc4f/the_biggest_gaslighting_in_ai_history_anthropic/ https://www.reddit.com/r/claude/comments/1satc4f/the_biggest...
• o10449366 6mo ago
  Resuming from sessions are still broken since Feb (I had to get claude to write a hook to fix that itself), the monitoring tool doesn't work and blocks usage of what does (simple sleep - except it doesn't even block correctly so you just sidestep in more ridiculous ways), and yet there seems to be more annoying activity proxies/spinner wheels (staring into middle distance)... Like I don't know how in a span of a few months you lose such focus on your product goals. Has Anthropic reached that point in their lifecycle already where their product team is no longer staffed by engineers and they have more and more non-technical MBAs joining trying to ride the hype train?
• throwaway2027 6mo ago
  Cool but I switched to Codex for the time being.
• rfc_1149 6mo ago
  The third bug is the one worth dwelling on. Dropping thinking blocks every turn instead of just once is the kind of regression that only shows up in production traffic. A unit test for "idle-threshold clearing" would assert "was thinking cleared after an hour of idle" (yes) without asserting "is thinking preserved on subsequent turns" (no). The invariant is negative space. The real lesson is that an internal message-queuing experiment masked the symptoms in their own dogfooding. Dogfooding only works when the eaten food is the shipped food.
  • afro88 6mo ago
    Experienced engineers that know the codebase and system well, and with enough time to consider the problem properly would likely consider this case. But if we're vibing... This is the kind of bug that should make it back into a review agent/skill's instructions in a more generic format. Essentially if something is done to the message history, check there tests that subsequent turns work as expected. But yeah, you'd have to piss off a bunch of users in prod first to discover the blind spot.
• whalesalad 6mo ago
  The funny thing is, in the last 3 days Claude has gotten substantially worse. So this claim, "All three issues have now been resolved as of April 20 (v2.1.116)" does not land with me at all.
• Rapzid 6mo ago
  > On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency—enough to make the UI appear frozen—some users were seeing in high mode. Translation: To reduce the load on our servers.
• kristianc 6mo ago
  To think we'd have known about this in advance if they'd just have open sourced Claude Code, rather than them being forced into this embarrassing post mortem. Sunlight is the best disinfectant.
• agentbonnybb 6mo ago
  [dead]
• dcchambers 6mo ago
  So it turns out Anthropic was gaslighting everyone on twitter about this then? Swearing that nothing had changed and people were imagining the models got worse?
  • mtilsted 6mo ago
    Nope, they were technical correct. Nothing had changed with the model. The model had not gotten any worse. The harness on the other hand. Now that had problems.
• WhoffAgents 6mo ago
  [dead]
• psubocz 6mo ago
  > All three issues have now been resolved as of April 20 (v2.1.116). The latest in homebrew is 2.1.108 so not fixed, and I don't see opus 4.7 on the models list... Is homebrew a second class citizen, or am I in the B group?
• sutterd 6mo ago
  What kind of performance are people getting now? I was running 4.7 yesterday and it did a remarkably bad job. I recreated my repo state exactly and ran the same starting task with 4.5 (which I have preferred to 4.6). It was even worse, by a large margin. It is likely my task was a difficult or poorly posed, but I still have some idea of what 4.5 should have done on it. This was not it. What experiences are other people having with the 4.7? How about with other model versions, if they are trying them? (In both cases, I ran on max effort, for whatever that is worth.)
• behat 6mo ago
  This is a very interesting read on failure modes of AI agents in prod. Curious about this section on the system prompt change: >> After multiple weeks of internal testing and no regressions in the set of evaluations we ran, we felt confident about the change and shipped it alongside Opus 4.7 on April 16. As part of this investigation, we ran more ablations (removing lines from the system prompt to understand the impact of each line) using a broader set of evaluations. One of these evaluations showed a 3% drop for both Opus 4.6 and 4.7. We immediately reverted the prompt as part of the April 20 release. Curious what helped catch in the later eval vs. initial ones. Was it that the initial testing was online A/B comparison of aggregate metrics, or that the dataset was not broad enough?
• jwpapi 6mo ago
  Those are exactly the kind of issues you run into when your app is ai coded you built one thing and kill something else. You have too many and the wrong benchmarks
• taytus 6mo ago
  They should do a similar report about their communication team. This was horrible mismanaged.
• mkilmanas 6mo ago
  [dead]
• hintymad 6mo ago
  > On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency—enough to make the UI appear frozen—some users were seeing in high mode. This sounds fishy. It's easy to show users that Claude is making progress by either printing the reasoning tokens or printing some kind of progress report. Besides, "very long" is such a weasel phrase.
  • reliablereason 6mo ago
    Right a very simple UI thing that they should have that would have prevented so much misunderstanding. Is a simple counter. How much usage do a have i used and how much is left. If a message will do a cache recreation the cost for that should be viewable.
• vintagedave 6mo ago
  > Today we are resetting usage limits for all subscribers. I asked for this via support, got a horrible corporate reply thread, and eventually downgraded my account. I'm using Codex now as we speak. I could not use Claude any more, I couldn't get anything done. Will they restore my account usage limits? Since I no longer have Max? Is that one week usage restored, or the entire buggy timespan?
  • sowbug 6mo ago
    [dead]
• 8note 6mo ago
  something i note from this is that this is not a model weights change, but it is a hidden state change anthropic is doing to the outputs that can tune the quality and down on the "model" without breaking the "we arent changing the model" promise. how often do these changes happen?
• maxrev17 6mo ago
  Please for the love of god just put the max price plan up like 4x or 5x in cost and make it actually work.
• EugeneOZ 6mo ago
  If you think that you can just silently modify the model without any announcements and only react when it doesn't go through unnoticed, then be 100% sure that your clients will check every possible alternative and will leave you as soon as they find anything similar in quality (and no, not a degraded one).
• ankit219 6mo ago
  An interesting question to wonder is why these optimizations were pushed so aggressively in the first place. Especially given this is the time they were running a 2x promotion, by themselves, without presumably seeing any slowdown in demand.
• MrOrelliOReilly 6mo ago
  IMO this is the consequence of a relentless focus on feature development over core product refinement. I often have the impression that Anthropic would benefit from a few senior product people. Someone needs to lend them a copy of “Escaping the Build Trap.” Just because we _can_ rapidly add features now doesn’t mean we should. PS I’m not referencing a well-known book to suggest the solution is trite product group think, but good product thinking is a talent separate from good engineering, and Anthropic seems short on the later recently
  • cmrdporcupine 6mo ago
    I think they've dug themselves into a complexity trap. Beyond the stochastic nature of the models themselves, I don't think they're able to reason about their software anymore. Too many levers, too many dials, and code that likely nobody understands. But worse, based on the pronouncements of Dario et al I suspect management is entirely unsympathetic because they believe we (SWEs) are on the chopping block to be replaced. And intimation that putting guard rails around these tools for quality concerns ... I'm suspecting is being ignored or discouraged. In the end, I feel like Claude Code itself started as a bit of a science experiment and it doesn't smell to me like it's adopted mature best practices coming out of that.
    • qweiopqweiop 6mo ago
      I agree. My real fear if this is how the company works, how are systems with real implications (e.g. defense) being treated.
  • slashdave 6mo ago
    They need to keep up with demand, because compute resources are clearly limited. That means they have no choice but to add these features, or things break, or they have to stop taking new customers. All of those options are unacceptable.
    • cmrdporcupine 6mo ago
      They're losing customers because of quality concerns. Pausing development and focusing 100% on quality is how you fix that. That said, that may not have been obvious at all in the Jan/Feb time frame when they got a wave of customers due to ethical concerns.
• lherron 6mo ago
  Are they also going to refund all the extra usage api $$$ people spent in the last month? Also I don’t know how “improving our Code Review tool” is going to improve things going forward, two of the major issues were intentional choices. No code review is going to tell them to stop making poor and compromising decisions.
  • dallen33 6mo ago
    No, they will not.
  • zem 6mo ago
    this is one reason i will not pay for extra usage - it is an incentive for them to be inefficient, or at least to not spend any effort on improving my token usage efficiency.
  • FireBeyond 6mo ago
    Even for all of us plan users, where we got barely any use from our plan because we'd destroy our 5h and 1w usage limits, also unlikely, after all they have an out of "your usage limits are guaranteed to be 5x of Pro users" (who are also being screwed). Of course, all their vibe coding is being done with effectively infinite tokens, so...
  • system2 6mo ago
    I stopped using it for nearly a month because of the performance degradation. I paid for the whole month. Wasted money.
• rebolek 6mo ago
  > On April 16, we added a system prompt instruction to reduce verbosity. What verbosity? Most of the time I don’t know what it’s doing.
  • whalesalad 6mo ago
    They don’t either.
• vicchenai 6mo ago
  had this happen to me mid-refactor and spent 20 min wondering if I'd gone crazy. honestly the one hour threshold feels pretty arbitrary, sometimes you just step away to think
• ramoz 6mo ago
  Opus 4.7 is very rough to work with. Specifically for long-horizon (we were told it was trained specifically for this and less handholding). I don't have trust in it right now. More regressions, more oversights, it's pedantic and weird ways. Ironically, requires more handholding. Not saying it's a bad model; it's just not simple to work with. for now: `/model claude-opus-4-6[1m]` (youll get different behavior around compaction without [1m])
• skeledrew 6mo ago
  Some of these changes and effects seriously affect my flow. I'm a very interactive Claude user, preferring to provide detailed guidance for my more serious projects instead of just letting them run. And I have multiple projects active at once, with some being untouched for days at a time. Along with the session limits this feels like compounding penalties as I'm hit when I have to wait for session reset (worse in the middle of a long task), when I take time to properly review output and provide detailed feedback, when I'm switching among currently active projects, when I go back to a project after a couple days or so,... This is honestly starting to feel untenable.
• deaux 6mo ago
  They had this ready and timed it for GPT 5.5 announcement. Zero chance it's a coincidence .
• kamranjon 6mo ago
  This black box approach that large frontier labs have adopted is going to drive people away. To change fundamental behavior like this without notifying them, and only retroactively explaining what happened, is the reason they will move to self-hosting their own models. You can't build pipelines, workflows and products on a base that is just randomly shifting beneath you.
  • panavm 6mo ago
    [flagged]
• zem 6mo ago
  ugh, caching based on idle time is horrible for my usage anyway; since claude is both fairly slow and doesn't really have much of a daily quota anyway I often tell it to do something and then wander off and come back to check on it when I next think about it. I always vaguely assumed that my session would not "detect" the intervening time anyway since it was all async. I guess from a global perspective time-based cache eviction makes sense.
• russellthehippo 6mo ago
  Damn it was real the whole time. I found Opus 4.7 to holistically underperform 4.6, and especially in how much wordiness there is. It's harder to work with so I just switched back to 4.6 + Kimi K2.6. Now GPT 5.5 is here and it's been excellent so far.
• ritonlajoie 6mo ago
  yesterday CC created a fastapi /healthz endpoint and told me it's the gold standard (with the ending z). today I stopped my max sub and will be trying codex
  • wrxd 6mo ago
    To be fair that’s a Google convention. Have a look at z-pages
  • jesse_dot_id 6mo ago
    This is fairly normal.
• cmenge 6mo ago
  Bit surprised about the amount of flak they're getting here. I found the article seemed clear, honest and definitely plausible. The deterioration was real and annoying, and shines a light on the problematic lack of transparency of what exactly is going on behind the scenes and the somewhat arbitrary token-cost based billing - too many factors at play, if you wanted to trace that as a user you can just do the work yourself instead. The fact that waiting for a long time before resuming a convo incurs additional cost and lag seemed clear to me from having worked with LLM APIs directly, but it might be important to make this more obvious in the TUI.
  • maronato 6mo ago
    I agree that it’s plausible, and I hope they learn. But trust is earned, and Anthropic’s public responses this past month were dismissive and unhelpful. Every one of these changes had the same goal: trading the intelligence users rely on for cheaper or faster outputs. Users adapt to how a model behaves, so sudden shifts without transparency are disorienting. The timing also undercuts their narrative. The fixes landed right before another change with the same underlying intent rolled out. That looks more like they were just reacting to experiments rather than understanding the underlying user pain. When people pay hundreds or thousands a month, they expect reliability and clear communication, ideally opt-in. Competitors are right there, and unreliability pushes users straight to them. All of this points to their priorities not being aligned with their users’.
    • xpe 6mo ago
      > All of this points to their priorities not being aligned with their users’. Framing this as "aligned" or "not aligned" ignores the interesting reality in the middle. It is banal to say an organization isn't perfectly aligned with its customers. I'm not disagreeing with the commenter's frustration. But I think it can help to try something out: take say the top three companies whose product you interact with on a regular basis. Take stock of (1) how fast that technology is moving; (2) how often things break from your POV; (3) how soon the company acknowledges it; (4) how long it takes for a fix. Then ask "if a friend of yours (competent and hard working) was working there, would I give the company more credit?" My overall feel is that people underestimate the complexity of the systems at Anthropic and the chaos of the growth. These kind of conversations are a sort of window into people's expectations and their ability to envision the possible explanations of what is happening at Anthropic.
• hirako2000 6mo ago
  In other words we did the right things, but we understand feedback, oh and bugs happen.
• tdg5 6mo ago
  I missed the part about the refunds…
• techpulselab 6mo ago
  [dead]
• Implicated 6mo ago
  Just as a note to CC fans/users here since I had an opportunity to do so... I tested resuming a session that was stale at 950k tokens after returning from a full day or so of being idle, thus a fully empty quota/session. Resuming it cost 5% of the current session and 1% of the weekly session on a max subscription.
• epsteingpt 6mo ago
  Gaslit for months, only to acknowledge.
• voxelc4L 6mo ago
  I’ve stuck to the non-1M context Opus 4.6 and it works really well for me, even with on-going context compression. I honestly couldn’t deal with the 1M context change and then the compounding token devouring nonsense of 4.7 I sincerely hope Anthropic is seeing all of this and taking note. They have their work cut out for them.
  • setnone 6mo ago
    absolutely agree: non-1M Opus 4.6 on x20 max was peak AGI now it's back to regular slop and just to check otherwise i have to spend at least $100
• EFLKumo 6mo ago
  [dead]
• wg0 6mo ago
  A heavily vibe coded CLI would have tons of issues, regularly. LLMs over edit and it's a known problem.
• exabrial 6mo ago
  Last I tried 4.7, it was bad. Like ChatGPT bad: changed stuff it wasn’t supposed to, hallucinated code, forgot information, missed simple things, didn’t catch mistakes. And it burned through tokens like crazy. I’ll stay on 4.6 for awhile. Seems to be better. What’s frustrating, though you cannot rely on these tools. They are constantly tinkering and changing with things and there’s no option to opt out.
  • Aperocky 6mo ago
    It seems like there is no concept of deployment, or even A/B test, what works on presumably claude employee's laptop for the hour they spent testing it will ship immediately to everyone. I mean, yes, even testing in production with some of your customer is better than.. testing with ALL of your customers?
• nopurpose 6mo ago
  Weren't there reports that quality decreased when using non-CC harnesses too? Nothing in blog post can explain that.
• yujunjie 6mo ago
  [dead]
• data-ottawa 6mo ago
  I think most frustrating is the system prompt issue after the postmortem from September[1]. These bugs have all of the same symptoms: undocumented model regressions at the application layer, and engineering cost optimizations that resulted in real performance regressions. I have some follow up questions to this update: - Why didn't September's "Quality evaluations in more places" catch the prompt change regression, or the cache-invalidation bug? - How is Anthropic using these satisfaction questions? My own analysis of my own Claude logs was showed strong material declines in satisfaction here, and I always answer those surveys honestly. Can you share what the data looked like and if you were using that to identify some of these issues? - There was no refund or comped tokens in September. Will there be some sort of comp to affected users? - How should subscribers of Claude Code trust that Anthropic side engineering changes that hit our usage limits are being suitably addressed? To be clear, I am not trying to attribute malice or guilt here, I am asking how Anthropic can try and boost trust here. When we look at something like the cache-invalidation there's an engineer inside of Anthropic who says "if we do this we save $X a week", and virtually every manager is going to take that vs a soft-change in a sentiment metric. - Lastly, when Anthropic changes Claude Code's prompt, how much performance against the stated Claude benchmarks are we losing? I actually think this is an important question to ask, because users subscribe to the model's published benchmark performance and are sold a different product through Claude Code (as other harnesses are not allowed). [1] https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues https://www.anthropic.com/engineering/a-postmortem-of-three-...
• bashtoni 6mo ago
  The Claude Code experience is still pretty bad after upgrading. I often see Error: claude-opus-4-7[1m] is temporarily unavailable, so auto mode cannot determine the safety of Bash right now. Wait briefly and then try this action again. If it keeps failing, continue with other tasks that don't require this action and come back to it later. Note: reading files, searching code, and other read-only operations do not require the classifier and can still be used. The only solution is to switch out of auto mode, which now seems to be the default every time I exit plan mode. Very annoying.
• bmd1905 6mo ago
  [dead]
• leobuskin 6mo ago
  This usage reset you did on April 23 will not mitigate the struggle we’ve experienced. I didn’t even notice it yesterday. I checked this morning and it came down from 25% weekly to 7%. What is this? I didn’t have problems for two months like many others (maybe my CC habits helped), but two weeks were very painful. Make a proper apology, guys. This “reset” for many users could hit the first days of the week, tell me you thought about that.
• DrokAI 6mo ago
  [dead]
• sreekanth850 6mo ago
  Who’s going to pay for the exorbitant number of tokens Claude used without delivering any meaningful outcome? I spent many sessions getting zero results, and when I posted about it on their subreddit, all I got were personal attacks from bots and fanboys. I instantly cancelled my subscription and moved to Codex. Also, it may be a coincidence, that the article was published just before the GPT 5.5 launch, and then they restored the original model while releasing a PR statement claiming it was due to bugs.
• system2 6mo ago
  Whatever they did, with the max plan, my daily usage quota was consumed in less than 10 minutes. Weird, let's hope they fix the usage now.
• huksley 6mo ago
  Just add this, it works better than Opus 4.7 vim ~/.claude/settings.json { "model": "claude-opus-4-6", "fastMode": false, "effortLevel": "high", "alwaysThinkingEnabled": true, "autoCompactWindow": 700000 }
  • meowface 6mo ago
    Wouldn't xhigh or max work better
    • evasion_capped 6mo ago
      Because of this https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2Fde3bcf9733b61f57234d8c45e663b1bd48677ea1-3840x2160.png&w=3840&q=75 https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-...
• zagwdt 6mo ago
  ngl lost alot of trust in cc after reading this, specially point 1 how do you just do that to millions of users building prod code with your shit
• jimmypk 6mo ago
  [flagged]
• gnegggh 6mo ago
  not the first time. Still not showing thinking are we?
• techpulselab 6mo ago
  [dead]
• RamblingCTO 6mo ago
  Doesn't change anything about opus 4.7 being an absolute buffon. Even going back to opus 4.6 doesn't feel like the magical period maybe 3-4 weeks ago. Gonna go back to openAI
• rohansood15 6mo ago
  If Anthropic couldn't catch these issues before people started screaming at them, do we really believe 50% of software engineering jobs are going away?
• deleted 6mo ago
  [deleted]
• claud_ia 6mo ago
  [dead]
• claud_ia 6mo ago
  [dead]
• varispeed 6mo ago
  It appears that Opus 4.7 has been nerfed already. Can't get any sensible results since yesterday. It just keeps running in circles. Even mention that it is committing fraud by doing superficial work it has been told specifically not to do doesn't help.
  • rimliu 6mo ago
    oh yes. I tried to get some review of a code base after some refactoring. CC produced a complete garbage review. After pointing that out it admitted that that was garbage - and promptly produced another pile of garbage. After the third failed attempt I had to call it a day.
• PeakScripter 6mo ago
  They should really test everything thoroughly and then make it available to general public to avoid these issues!!
• rcarmo 6mo ago
  Actually, I think their deeper problems are twofold: - Claude Code is _vastly_ more wasteful of tokens than anything else I've used. The harness is just plain bad. I use pi.dev and created https://github.com/rcarmo/piclaw https://github.com/rcarmo/piclaw, and the gaps are huge -- even the models through Copilot are incredibly context-greedy when compared to GPT/Codex - 4.7 can be stupidly bad. I went back to 4.6 (which has always been risky to use for anything reliable, but does decent specs and creative code exploration) and Codex/GPT for almost everything. So there is really no reason these days to pay either their subscription or their insanely high per/token price _and_ get bloat across the board.
• hansmayer 6mo ago
  A suggestion to Anthropic, just start charging the real price for your software. Of course you have to dumb it down, when the $200 tier in reality produces 5-10 thousand dollars in monthly costs when used by people who know how to max it out. So then you come up with creative nonsense like "adaptive thinking" when your tool is sometimes working and sometimes outright not - the irony of "intelligent tools" not "thinking" aside. Of course this would kind of ruin your current value proposition as charging the actual price would make your core idea of making large swaths of skilled population un-employed, unfeasible but I am sure if you feed it into the Claude, it will find some points for and against, just like how Karpathy uses his LLM of choice to excrement his blog posts.
  • noname120 6mo ago
    https://danluu.com/cocktail-ideas/ https://danluu.com/cocktail-ideas/
    • hansmayer 6mo ago
      I am afraid you may be trying to prove a wrong point here :)
      • noname120 6mo ago
        True I didn’t read carefully enough the last part of your comment
  • whalesalad 6mo ago
    > when the $200 tier in reality produces 5-10 thousand dollars in monthly costs are you asserting that the actual dollar cost to anthropic for a heavy user was 5-10k? or are you basing this on the (fabricated) value of those tokens, ie potentially lost revenue from a pay-per-token user.
    • hansmayer 6mo ago
      I am basing it on self-reports of advanced users on popular platforms such as reddit, reporting of Ed Zitron AND the official affidavit of the Anthropic CFO in relation to their court-filed complaint against DoD (for being excluded), stating that the total revenue of Anthropic since its founding and TO DATE had been at meager 5 Billion dollars. So wasting hundreds of billions of dollars per year to make a cumulative revenue of 5B to date does not quite sound like a financially sound business to me.
• noname120 6mo ago
  So now the solution is to input a “ping” message every hour so that it keeps the cache warm?
• sscaryterry 6mo ago
  Glad there is finally some ownership. It is a pity that this was mostly because AMD embarrassed them on GitHub. Users have been reporting these issues for weeks, but were mostly ignored.
• HarHarVeryFunny 6mo ago
  And the reason why Claude Code is so buggy ... https://techtrenches.dev/p/the-snake-that-ate-itself-what-claude https://techtrenches.dev/p/the-snake-that-ate-itself-what-cl...
• anonyfox 6mo ago
  I refuse to believe that caching tiers for longer than 1 hour would be impossible to transparently build and use to avoid all this complexity to begin with, nor that it would be that expensive to maintain in 2026 when the bulk costs are on inference anyways which would even be reduced by occasional longer time cache hits.
• whh 6mo ago
  Thanks Anthropic, and a big thanks to your Claude Code team for the customer obsession here. I've just noticed the Command + Backspace fix and even the nice little Ctrl + y addition as a fix for accidents. I really appreciate these little touches.
• YetAnotherNick 6mo ago
  Why don't they monitor average prompt and response token length(both cached and uncached) per interaction. Seems this could have solved all their previous unnoticed degradation. Also bit surprised they don't have any automated quality check. They can run something like swe bench before each release. Both of these seem like a basic thing even for startup, let alone some product generating billions in revenue.
• dgreensp 6mo ago
  This reveals a staggering level of incompetence, if that’s really all it is, and lack of transparency. They don’t have ANY product-level quality tests that picked this up? Many users did their own tests and published them. It’s not hard. And these users’ complaints were initially dismissed. I don’t think the high vs medium change is really on par with the others. That’s a setting you change in the UI, and depending on what you are doing, both effort levels are pretty capable, they just operate a bit differently. Unless I’m missing something and they are saying they were doing some kind of routing behind the scenes. If they are constantly pushing major changes to the prompts and workings of the tool, without communicating about it, and without testing, it’s likely there are other bugs and quality-degrading changes beyond the ones in this article, which would make a lot of sense.
  • kolinko 6mo ago
    There were few systems like claude in the past, to testing rulebook is not really written yet. And far from obvious.
    • sockgrant 6mo ago
      LLM evals are well established, are these not applicable here?
  • rekrsiv 6mo ago
    Time is finite and regression testing always gets punted to the back of the line when humans are excited. This simply reveals a staggering level of humanity.
    • FartyMcFarter 6mo ago
      Software engineering is not a new field. Best practices on testing are mature now, and Anthropic has poached enough engineers from companies with a solid understanding of those practices. Yet, their flagship product got three really bad changes shipped into it and only resolved after more than a month. This raises another question: with all the industry-wide boasting about AI-driven productivity, why does the leading company in agentic coding take over a month to fix severe customer-reported issues?
• powera 6mo ago
  I'm not sure they've found/understand it yet. My two main theories: 1. A bunch of people with new Claude Code codebases in December now are working with a larger codebase, causing more context. Claude reads a lot of code files, and doesn't effectively prune from the context as far as I can tell. I find myself having to hint Claude regularly about what files to read (and not read) to avoid having 75k of unrelated files in the context window. 2. Claude Code tries to do more now, for the benefit of people who don't know exactly what they want. The trade-off is that it's worse at doing exactly what people want, when they do know. The "small fix" becomes a large endeavor for Claude.
• btbuildem 6mo ago
  A lot of this could have been avoided with more openness and transparency. Communicate the changes you are making. Leverage the community using your product(s). Reveal more about the building blocks (system prompts, harness, etc) so people can better understand how to use your tools. I understand they're in an existential battle with the other SOTA houses -- but secrecy, straight out lies and opaque "communication" is not the way to win it. Not when the OSS stack is hot on their heels anyway.
• MyUltiDev 6mo ago
  [dead]
• conorliu 6mo ago
  [dead]
• thesnide 6mo ago
  In my last session had the claude agent generated code, then tests, then the fixtures from passing the tests to the code. and gloated that all passed immediatly. once i saw that, i'm not surprised anymore.
• calmbonsai 6mo ago
  Move really fast and break all the things. Yeah, these guys are reminding me of the 20-teens where AWS started pushing out rando edge-case services with little regard to quality or integration that ended up hurting the quality of the core services too.
• yurukusa 6mo ago
  [dead]
• ginkgotree 6mo ago
  When will they fix it? This is what is important.
• adnasalk 6mo ago
  [flagged]
• kk_mors 6mo ago
  [flagged]