6 ms·

This can't be understated. I started using it heavily earlier this summer and it felt like magic. Someone signing up now based on how I described my personal ex

by mh- 1y ago

This can't be understated. I started using it heavily earlier this summer and it felt like magic. Someone signing up now based on how I described my personal experiences with it then would think I was out of my mind. For technical tasks it has been a net negative for me for the last several weeks. (Speaking of both Claude Code and the desktop app, both Sonnet and Opus >=4, on the Max plan.)

pqdbr 1y ago
Same here. Even with Opus in Claude Code I'm getting terrible results, sometimes feeling we went back to the GPT 3.5 eon. And it seems they are implementing heavily token-saving measures: the model does not read context anymore unless you force it to, making up method calls as it goes.
- mh- 1y ago
  The simplest thing I frequently ask of regular Claude (not Code) in the desktop app: "Use your web search tool to find me the go-to component for doing xyz in $language $framework. Always link the GitHub repo in your response." Previously Sonnet 4 would return a good answer to this at least 80% of the time. Now even Opus 4.1 with extended thinking frequently ignores my ask for it to use the search tool, which allows it to hallucinate a component in a library. Or maybe an entire repo. It's gone backwards severely. (If someone from Anthropic sees this, feel free to reach out for chat IDs/share links. I have dozens.)
  - spicybright 1y ago
    Glad I'm not crazy. I actually noticed both 4 models are just garbage. I started running my prompts through those, and Sonnet 3.7 comparing the results. Sonnet 3.7 is way better at everything.
    - idonotknowwhy 1y ago
      You're not crazy, and this isn't new for Anthropic. Something is off with Opus4.1, I actually saw it make 2 "typos" last week (I've never seen a model like this make a dumb "typo" before). And it's missing details that it understood last month (can easily test this if you have some chats in OpenWebUI or LibreChat, just go in and hit regenerate). Sonnet 3.5 did this last year a few times, it'd have days where it wasn't working properly, and sure enough, I'd jump online and see "Claude's been lobotomized again". They also experiment with injecting hidden system prompts from time to time. Eg. if you ask for a story about some IP, it'll interrupt your prompt and remind the model not to infringe copyright. (We could see this via API with prompt engineering, adding a "!repeat" "debug prompt" that revealed it, though they seem to have patched that now. > I started running my prompts through those, and Sonnet 3.7 comparing the results. Sonnet 3.7 is way better at everything. Same here. And on API, the old Opus 3 is also unaffected (though that model is too old for coding).
  - dingnuts 1y ago
    How is this better/faster than typing "xyz language framework site://github.com" into Kagi IDK about you but I find it faster to type a few keywords and click the first result than to wait for "extended thinking" to warm up a cup of hot water only to ignore "your ask" (it's a "request," not an "ask," unless you're talking to a Product Manager with corporate brain damage) to search and then outputs bullshit. I can only assume after you waste $0.10 asking Claude and reading the bullshit, you use normal search. Truly revolutionary rechnology
- j45 1y ago
  I’m running into this as well. Might be Claude optimizing for general use cases compared to code and that affecting the code side? Feels strange, because Claude api isn’t the same as the web tool so I didn’t expect Claude code to be the same. It might be a case of having to learn to read Claude best practice docs and keep up with them. Normally I’d have Claude read them itself and update an approach to use. Not sure that works as well anymore.
pc86 1y ago
I hesitate to use phrases like "bait and switch" but it seems like every model gets released and is borderline awe-inspiring, then as adoption increases, and load increases, it's like it gets hit in the head with a hammer and is basically useless for anything beyond a multi-step google search.
- rootnod3 1y ago
  AI is not useful in the long term is is unsustainable. News at 11.
- otabdeveloper4 1y ago
  No, that's just the normal slope of the hype curve as you start figuring out how the man behind the curtain operates.
- j45 1y ago
  It’s important to jump on new models super early while the rails get out in. Anyone remember GPT4 the day it launched? :)
- dingnuts 1y ago
  I think it's a psychological bias of some sort. When the feeling of newness wears off and you realize the model is still kind of shit, you have an imperfect memory of the first few uses when you were excited and have repressed the failures from that period. As the hype wears off you become more critical and correctly evaluate the model
  - lacy_tinpot 1y ago
    It's not because it's actually tracked and even acknowledged by the companies themselves.
  - Uehreka 1y ago
    I get that it’s fun and stylish to tell people they aren’t aware of their own cognitive biases, but it’s also a difficult take to falsify, which is why I generally have a high bar for people to clear when they want to assert that something is all in people’s heads. People seem to turn to this with a lot when the suspicion many people have is difficult to verify. And while I don’t trust a suspicion just because it’s held by a lot of people, I also won’t allow myself to embrace the comforting certainty of “it’s surely false and it’s psychological bias”. Sometimes we just need to not be sure what’s going on.
probably_wrong 1y ago
Have you considered perhaps that you are, indeed, out of your mind? Or more precisely, that you could be rationalizing what is essentially a random process? Based on the discussions here it seems that every model is either about to be great or was great in the past but now is not. Sucks for those of us who are stuck in the now, though.
- mh- 1y ago
  > Have you considered perhaps that you are, indeed, out of your mind? Yes, but I'll revisit.
- hkt 1y ago
  It seems plausible enough that they're trying to squeeze as much out of their hardware as possible and getting the balance wrong. As prices for hardware capable of running local LLMs drop and local models improve, this will become less prevalent and the option of running your own will become more widespread, probably killing this kind of service outside of enterprise. Even if it doesn't kill that service, it'll be _considerably_ better to be operating your own as you have control over what is actually running. On that note, I strongly recommend qwen3:4b. It is _bonkers_ how good it is, especially considering how relatively tiny it is.
  - j45 1y ago
    Thanks. Mind sharing which kinds of Claude tasks you are able to run on qwen3:4b?
- groby_b 1y ago
  "that every model is either about to be great or was great in the past but now is not" FWIW, Codex-CLI w/ ChatGPT5 medium is great right now. Objectively accelerating me. Not a coding god like some posters would have it, but overall freeing up time for me. Observably. Assuming I haven't had since-cured delusions, the same was true for Claude Code, but isn't any more. Concrete supporting evidence: From time to time, I have coding CLIs port older projects of varying (but small-ish) sizes from JS to TS. Claude Code used to do well on that. Repeatedly. I did another test last Sunday, and it dug a momentous hole for itself that even liberal sprinkling of 'as unknown' everywhere couldn't solve. Codex managed both the ab-initio port and was able to undig from CC's massive hole abandoned mid-port. So I'd say the evidence points somewhat against random process, given repeated testing shows clear signal both of past capability and of recent loss of capability. The idea that it's a "random" process is misguided.
gjvc 1y ago
"can't be overstated", you mean
- mh- 1y ago
  You're absolutely right! I should have used the correct word when writing the Hacker News comment. (lol, yes, thank you.)
- glenstein 1y ago
  This one is interesting because I have seen a fair amount of "can't be understated" on reddit also. Interesting case of linguistic drift.
  - mh- 1y ago
    In my case it was just me straight up using the wrong word by accident. Parent commenter caught it inside the edit window but I left it alone so their comment wasn't out of context. :)
  - gjvc 1y ago
    linguistic drift my ass
data-ottawa 1y ago
I don’t think you’re crazy, something is off in their models. As an example I’ve been using an MCP tool to provide table schemas to Claude for months. There was a point where it stopped recognizing the tool unless mentioned in early August. Maybe that’s related to their degraded quality issue. This morning after pulling the correct schema info Sonnet started hallucinating columns (from Shopify’s API docs) and added them to my query. That’s a use case I’ve been doing daily for months and in the last few weeks has gone from consistent low supervision to flaky and low quality. I don’t know what’s going on, Sonnet has definitely felt worse, and the timeline matches their status page incident, but it’s definitely not resolved. Opus 4.1 also feels flaky, it feels like it’s less consistent about recalling earlier prompt details than 4.0. I personally am frustrated that there’s no refund or anything after a month of degraded performance, and they’ve had a lot of downtime.
- dingnuts 1y ago
  I have read so many anecdotes about so many models that "were great" and aren't now. I actually think this is psychological bias. It got a few things right early on, and that's what you remember. As time passes, the errors add up, until the memory doesn't match reality. The "new shiny" feeling goes away, and you perceive it for what it really is: a kind of shitty slot machine > personally am frustrated that there’s no refund or anything after a month of degraded performance lol, LMAO. A company operates a shitty slot machine at a loss and you're surprised they have "issues" that reduce your usage? I'm not paying for any of this shit until these companies figure out how to align incentives. If they make more by applying limits, or charge me when the machine makes errors, that's good for them and bad for me! Why should I continue to pay to pull on the slot machine lever? It's a waste of time and money. I'll be richer and more productive if I just write the code myself, and the result will be better too.
  - reactordev 1y ago
    This is equivalent to people reminiscing about WoW or EverQuest saying gaming peaked back then… I think you’re right. I think it’s complete bias with a little bit of “it does more tasks now” so it might behave a bit differently to the same prompt. I also think you’re right that there’s an incentive to dumb it down so you pull the lever more. Just 2 more $1 spins and maybe you’ll hit jackpot. Really it’s the enshitification of the SOTA for profits and glory.
alvis 1y ago
And lest we forget opus was accidentally dumber last week! https://status.anthropic.com/incidents/72f99lh1cj2c https://status.anthropic.com/incidents/72f99lh1cj2c
OtomotO 1y ago
This, so much this... I signed up for Claude over a week ago and I totally regret it! Previously I was using it and some ChatGPT here and there (also had a subscription in the past) and I felt like Claude added some more value. But it's getting so unstable. It generates code, I see it doing that, and then it throws the code away and gives me the previous version of something 1:1 as a new version. And then I have to waste CO2 to tell it to please don't do that and then sometimes it generates what I want, sometimes it just generates it again, just to throw it away immediately... This is soooooooo annoying and the reason I canceled my subscription!
- brandon272 1y ago
  > But it's getting so unstable. It generates code, I see it doing that, and then it throws the code away and gives me the previous version of something 1:1 as a new version. I've had the same experience. Totally unreliable.
  - actsasbuffoon 1y ago
    I regularly have this happen: 1. Ask Claude to fix something 2. It fails to fix the issue 3. I tell it that the fix didn’t work 4. It reverts its failed fix and tells me everything is working now. This is like finding a decapitated body, trying to bring it back to life by smooshing the severed head against the neck, realizing that didn’t bring them back to life, dropping the head back on the ground, and saying, “There; I’ve saved them now.”
    - johnisgood 1y ago
      Gosh, can't we get back to Sonnet 3.5 or whichever was the version around a year ago? It worked so well for me.
  - jononor 1y ago
    This happens to me a lot. Almost once per session now, and not even when things are complicated. The model also thinks it has done the changes. So it seems a UI/state bug, not on the model side.
darepublic 1y ago
[flagged]
fuomag9 1y ago
I felt like the model degraded lately as well, I've been using Claude everyday for months now
- j45 1y ago
  I’m considering trying the api directly for a bit with Claude code to compare but need a test quite first to compare all 3.
otabdeveloper4 1y ago
Congrats, you grew up. It's not Claude's fault.
allisdust 1y ago
Yup. Opus 4.1 has been feeling like absolute dog shit and it made me give up in frustration several times. They really did downgrade their models. Max plan is a joke now. I'm barely using Pro level tokens since its a net negative on my productivity. Enshittification is now truly in place.
trunnell 1y ago
https://status.anthropic.com/incidents/72f99lh1cj2c https://status.anthropic.com/incidents/72f99lh1cj2c They recently resolved two bugs affecting model quality, one of which was in production Aug 5-Sep 4. They also wrote: Importantly, we never intentionally degrade model quality as a result of demand or other factors, and the issues mentioned above stem from unrelated bugs. Sibling comments are claiming the opposite, attributing malice where the company itself says it was a screw up. Perhaps we should take Anthropic at its word, and also recognize that model performance will follow a probability distribution even for similar tasks, even without bugs making thing worse.
- mh- 1y ago
  The problem is twofold: - They're reporting that only impacted Haiku 3.5 and Sonnet 4. I used neither model during the time period I'm concerned with. - It took them a month to publicly acknowledge that issue, so now we lack confidence there isn't another underlying issue going undetected (or undisclosed, less charitably) that affects Opus.
  - trunnell 1y ago
    now we lack confidence there isn't another underlying issue You can be confident there is a non-zero rate of errors and defects in any complex service that's moving as fast as the frontier model providers!
    - mh- 1y ago
      Of course. Totally agree, and that's why (I think) I'm being as charitable as possible in this thread.
  - criemen 1y ago
    They posted > We are continuing to monitor for any ongoing quality issues, including reports of degradation for Claude Opus 4.1. I take that as acknowledgment that there might be an issue with Opus 4.1 (granted, undetected still), but not undisclosed, and they're actively looking for it? I'd not jump to "they must be hiding things" yet. They're building, deploying and scaling their service at incredible pace, they, as we all, are bound to get some things wrong.
bongodongobob 1y ago
Thanks for the confirmation. Lately it's been telling me it has made edits or written code yet it's nowhere to be seen. It's been messing up extremely simple tasks like "move this knob from the bottom of the screen to the right". Over and over it will insist it made the changes but it hasn't. Getting confused about completely different sections of code and files. I picked up Claude at the beginning of the summer and have had the same experience.
teknologist 1y ago
Here's a useful tracker for how "stupid" the models are now and over some preset time periods: https://aistupidlevel.info https://aistupidlevel.info
yumraj 1y ago
I had even posted a Ask HN: if people had experienced issues with Claude Code since for me it's slowed down substantially, it'll frequently just pause and take much longer. I have a Claude Max 5X plan. I've been running ccusage to monitor and my usage in $ terms has dropped to a 1/3 of what it was few weeks ago. While some of it could be due to how I'm using it, but a drop of 60%-70% cannot be attributed to that alone and I think is partly due to the performance. To add: frequently, as in almost every time: 1) it'll start doing something and will go silent for a long time. 2) pressing esc to interrupt will take a long time to take action since it's probably stuck doing something. Earlier, interrupting via esc used to be almost instantaneous. So, I still like it, but at my 1/3 drop in measured usage I'm almost tempted to go back to Pro and see if that'll meet my needs.

by mh- 1y ago

pqdbr 1y ago

Same here. Even with Opus in Claude Code I'm getting terrible results, sometimes feeling we went back to the GPT 3.5 eon. And it seems they are implementing heavily token-saving measures: the model does not read context anymore unless you force it to, making up method calls as it goes.

mh- 1y ago
The simplest thing I frequently ask of regular Claude (not Code) in the desktop app: "Use your web search tool to find me the go-to component for doing xyz in $language $framework. Always link the GitHub repo in your response." Previously Sonnet 4 would return a good answer to this at least 80% of the time. Now even Opus 4.1 with extended thinking frequently ignores my ask for it to use the search tool, which allows it to hallucinate a component in a library. Or maybe an entire repo. It's gone backwards severely. (If someone from Anthropic sees this, feel free to reach out for chat IDs/share links. I have dozens.)
- spicybright 1y ago
  Glad I'm not crazy. I actually noticed both 4 models are just garbage. I started running my prompts through those, and Sonnet 3.7 comparing the results. Sonnet 3.7 is way better at everything.
  - idonotknowwhy 1y ago
    You're not crazy, and this isn't new for Anthropic. Something is off with Opus4.1, I actually saw it make 2 "typos" last week (I've never seen a model like this make a dumb "typo" before). And it's missing details that it understood last month (can easily test this if you have some chats in OpenWebUI or LibreChat, just go in and hit regenerate). Sonnet 3.5 did this last year a few times, it'd have days where it wasn't working properly, and sure enough, I'd jump online and see "Claude's been lobotomized again". They also experiment with injecting hidden system prompts from time to time. Eg. if you ask for a story about some IP, it'll interrupt your prompt and remind the model not to infringe copyright. (We could see this via API with prompt engineering, adding a "!repeat" "debug prompt" that revealed it, though they seem to have patched that now. > I started running my prompts through those, and Sonnet 3.7 comparing the results. Sonnet 3.7 is way better at everything. Same here. And on API, the old Opus 3 is also unaffected (though that model is too old for coding).
- dingnuts 1y ago
  How is this better/faster than typing "xyz language framework site://github.com" into Kagi IDK about you but I find it faster to type a few keywords and click the first result than to wait for "extended thinking" to warm up a cup of hot water only to ignore "your ask" (it's a "request," not an "ask," unless you're talking to a Product Manager with corporate brain damage) to search and then outputs bullshit. I can only assume after you waste $0.10 asking Claude and reading the bullshit, you use normal search. Truly revolutionary rechnology
j45 1y ago
I’m running into this as well. Might be Claude optimizing for general use cases compared to code and that affecting the code side? Feels strange, because Claude api isn’t the same as the web tool so I didn’t expect Claude code to be the same. It might be a case of having to learn to read Claude best practice docs and keep up with them. Normally I’d have Claude read them itself and update an approach to use. Not sure that works as well anymore.

pc86 1y ago

I hesitate to use phrases like "bait and switch" but it seems like every model gets released and is borderline awe-inspiring, then as adoption increases, and load increases, it's like it gets hit in the head with a hammer and is basically useless for anything beyond a multi-step google search.

rootnod3 1y ago
AI is not useful in the long term is is unsustainable. News at 11.
otabdeveloper4 1y ago
No, that's just the normal slope of the hype curve as you start figuring out how the man behind the curtain operates.
j45 1y ago
It’s important to jump on new models super early while the rails get out in. Anyone remember GPT4 the day it launched? :)
dingnuts 1y ago
I think it's a psychological bias of some sort. When the feeling of newness wears off and you realize the model is still kind of shit, you have an imperfect memory of the first few uses when you were excited and have repressed the failures from that period. As the hype wears off you become more critical and correctly evaluate the model
- lacy_tinpot 1y ago
  It's not because it's actually tracked and even acknowledged by the companies themselves.
- Uehreka 1y ago
  I get that it’s fun and stylish to tell people they aren’t aware of their own cognitive biases, but it’s also a difficult take to falsify, which is why I generally have a high bar for people to clear when they want to assert that something is all in people’s heads. People seem to turn to this with a lot when the suspicion many people have is difficult to verify. And while I don’t trust a suspicion just because it’s held by a lot of people, I also won’t allow myself to embrace the comforting certainty of “it’s surely false and it’s psychological bias”. Sometimes we just need to not be sure what’s going on.

probably_wrong 1y ago

Have you considered perhaps that you are, indeed, out of your mind? Or more precisely, that you could be rationalizing what is essentially a random process? Based on the discussions here it seems that every model is either about to be great or was great in the past but now is not. Sucks for those of us who are stuck in the now, though.

mh- 1y ago
> Have you considered perhaps that you are, indeed, out of your mind? Yes, but I'll revisit.
hkt 1y ago
It seems plausible enough that they're trying to squeeze as much out of their hardware as possible and getting the balance wrong. As prices for hardware capable of running local LLMs drop and local models improve, this will become less prevalent and the option of running your own will become more widespread, probably killing this kind of service outside of enterprise. Even if it doesn't kill that service, it'll be _considerably_ better to be operating your own as you have control over what is actually running. On that note, I strongly recommend qwen3:4b. It is _bonkers_ how good it is, especially considering how relatively tiny it is.
- j45 1y ago
  Thanks. Mind sharing which kinds of Claude tasks you are able to run on qwen3:4b?
groby_b 1y ago
"that every model is either about to be great or was great in the past but now is not" FWIW, Codex-CLI w/ ChatGPT5 medium is great right now. Objectively accelerating me. Not a coding god like some posters would have it, but overall freeing up time for me. Observably. Assuming I haven't had since-cured delusions, the same was true for Claude Code, but isn't any more. Concrete supporting evidence: From time to time, I have coding CLIs port older projects of varying (but small-ish) sizes from JS to TS. Claude Code used to do well on that. Repeatedly. I did another test last Sunday, and it dug a momentous hole for itself that even liberal sprinkling of 'as unknown' everywhere couldn't solve. Codex managed both the ab-initio port and was able to undig from CC's massive hole abandoned mid-port. So I'd say the evidence points somewhat against random process, given repeated testing shows clear signal both of past capability and of recent loss of capability. The idea that it's a "random" process is misguided.

gjvc 1y ago

"can't be overstated", you mean

mh- 1y ago
You're absolutely right! I should have used the correct word when writing the Hacker News comment. (lol, yes, thank you.)
glenstein 1y ago
This one is interesting because I have seen a fair amount of "can't be understated" on reddit also. Interesting case of linguistic drift.
- mh- 1y ago
  In my case it was just me straight up using the wrong word by accident. Parent commenter caught it inside the edit window but I left it alone so their comment wasn't out of context. :)
- gjvc 1y ago
  linguistic drift my ass

data-ottawa 1y ago

I don’t think you’re crazy, something is off in their models. As an example I’ve been using an MCP tool to provide table schemas to Claude for months. There was a point where it stopped recognizing the tool unless mentioned in early August. Maybe that’s related to their degraded quality issue. This morning after pulling the correct schema info Sonnet started hallucinating columns (from Shopify’s API docs) and added them to my query. That’s a use case I’ve been doing daily for months and in the last few weeks has gone from consistent low supervision to flaky and low quality. I don’t know what’s going on, Sonnet has definitely felt worse, and the timeline matches their status page incident, but it’s definitely not resolved. Opus 4.1 also feels flaky, it feels like it’s less consistent about recalling earlier prompt details than 4.0. I personally am frustrated that there’s no refund or anything after a month of degraded performance, and they’ve had a lot of downtime.

dingnuts 1y ago
I have read so many anecdotes about so many models that "were great" and aren't now. I actually think this is psychological bias. It got a few things right early on, and that's what you remember. As time passes, the errors add up, until the memory doesn't match reality. The "new shiny" feeling goes away, and you perceive it for what it really is: a kind of shitty slot machine > personally am frustrated that there’s no refund or anything after a month of degraded performance lol, LMAO. A company operates a shitty slot machine at a loss and you're surprised they have "issues" that reduce your usage? I'm not paying for any of this shit until these companies figure out how to align incentives. If they make more by applying limits, or charge me when the machine makes errors, that's good for them and bad for me! Why should I continue to pay to pull on the slot machine lever? It's a waste of time and money. I'll be richer and more productive if I just write the code myself, and the result will be better too.
- reactordev 1y ago
  This is equivalent to people reminiscing about WoW or EverQuest saying gaming peaked back then… I think you’re right. I think it’s complete bias with a little bit of “it does more tasks now” so it might behave a bit differently to the same prompt. I also think you’re right that there’s an incentive to dumb it down so you pull the lever more. Just 2 more $1 spins and maybe you’ll hit jackpot. Really it’s the enshitification of the SOTA for profits and glory.

alvis 1y ago

And lest we forget opus was accidentally dumber last week! https://status.anthropic.com/incidents/72f99lh1cj2c https://status.anthropic.com/incidents/72f99lh1cj2c

OtomotO 1y ago

This, so much this... I signed up for Claude over a week ago and I totally regret it! Previously I was using it and some ChatGPT here and there (also had a subscription in the past) and I felt like Claude added some more value. But it's getting so unstable. It generates code, I see it doing that, and then it throws the code away and gives me the previous version of something 1:1 as a new version. And then I have to waste CO2 to tell it to please don't do that and then sometimes it generates what I want, sometimes it just generates it again, just to throw it away immediately... This is soooooooo annoying and the reason I canceled my subscription!

brandon272 1y ago
> But it's getting so unstable. It generates code, I see it doing that, and then it throws the code away and gives me the previous version of something 1:1 as a new version. I've had the same experience. Totally unreliable.
- actsasbuffoon 1y ago
  I regularly have this happen: 1. Ask Claude to fix something 2. It fails to fix the issue 3. I tell it that the fix didn’t work 4. It reverts its failed fix and tells me everything is working now. This is like finding a decapitated body, trying to bring it back to life by smooshing the severed head against the neck, realizing that didn’t bring them back to life, dropping the head back on the ground, and saying, “There; I’ve saved them now.”
  - johnisgood 1y ago
    Gosh, can't we get back to Sonnet 3.5 or whichever was the version around a year ago? It worked so well for me.
- jononor 1y ago
  This happens to me a lot. Almost once per session now, and not even when things are complicated. The model also thinks it has done the changes. So it seems a UI/state bug, not on the model side.

darepublic 1y ago

[flagged]

fuomag9 1y ago

I felt like the model degraded lately as well, I've been using Claude everyday for months now

j45 1y ago
I’m considering trying the api directly for a bit with Claude code to compare but need a test quite first to compare all 3.

otabdeveloper4 1y ago

Congrats, you grew up. It's not Claude's fault.

allisdust 1y ago

Yup. Opus 4.1 has been feeling like absolute dog shit and it made me give up in frustration several times. They really did downgrade their models. Max plan is a joke now. I'm barely using Pro level tokens since its a net negative on my productivity. Enshittification is now truly in place.

trunnell 1y ago

https://status.anthropic.com/incidents/72f99lh1cj2c https://status.anthropic.com/incidents/72f99lh1cj2c They recently resolved two bugs affecting model quality, one of which was in production Aug 5-Sep 4. They also wrote: Importantly, we never intentionally degrade model quality as a result of demand or other factors, and the issues mentioned above stem from unrelated bugs. Sibling comments are claiming the opposite, attributing malice where the company itself says it was a screw up. Perhaps we should take Anthropic at its word, and also recognize that model performance will follow a probability distribution even for similar tasks, even without bugs making thing worse.

mh- 1y ago
The problem is twofold: - They're reporting that only impacted Haiku 3.5 and Sonnet 4. I used neither model during the time period I'm concerned with. - It took them a month to publicly acknowledge that issue, so now we lack confidence there isn't another underlying issue going undetected (or undisclosed, less charitably) that affects Opus.
- trunnell 1y ago
  now we lack confidence there isn't another underlying issue You can be confident there is a non-zero rate of errors and defects in any complex service that's moving as fast as the frontier model providers!
  - mh- 1y ago
    Of course. Totally agree, and that's why (I think) I'm being as charitable as possible in this thread.
- criemen 1y ago
  They posted > We are continuing to monitor for any ongoing quality issues, including reports of degradation for Claude Opus 4.1. I take that as acknowledgment that there might be an issue with Opus 4.1 (granted, undetected still), but not undisclosed, and they're actively looking for it? I'd not jump to "they must be hiding things" yet. They're building, deploying and scaling their service at incredible pace, they, as we all, are bound to get some things wrong.

bongodongobob 1y ago

Thanks for the confirmation. Lately it's been telling me it has made edits or written code yet it's nowhere to be seen. It's been messing up extremely simple tasks like "move this knob from the bottom of the screen to the right". Over and over it will insist it made the changes but it hasn't. Getting confused about completely different sections of code and files. I picked up Claude at the beginning of the summer and have had the same experience.

teknologist 1y ago

Here's a useful tracker for how "stupid" the models are now and over some preset time periods: https://aistupidlevel.info https://aistupidlevel.info

yumraj 1y ago

I had even posted a Ask HN: if people had experienced issues with Claude Code since for me it's slowed down substantially, it'll frequently just pause and take much longer. I have a Claude Max 5X plan. I've been running ccusage to monitor and my usage in $ terms has dropped to a 1/3 of what it was few weeks ago. While some of it could be due to how I'm using it, but a drop of 60%-70% cannot be attributed to that alone and I think is partly due to the performance. To add: frequently, as in almost every time: 1) it'll start doing something and will go silent for a long time. 2) pressing esc to interrupt will take a long time to take action since it's probably stuck doing something. Earlier, interrupting via esc used to be almost instantaneous. So, I still like it, but at my 1/3 drop in measured usage I'm almost tempted to go back to Pro and see if that'll meet my needs.