6 ms·

This is gonna be game-changing for the next 2-4 weeks before they nerf the model. Then for the next 2-3 months people complaining about the degradation will be

This is gonna be game-changing for the next 2-4 weeks before they nerf the model. Then for the next 2-3 months people complaining about the degradation will be labeled “skill issue”. Then a sacrificial Anthropic engineer will “discover” a couple obscure bugs that “in some cases” might have lead to less than optimal performance. Still largely a user skill issue though. Then a couple months later they’ll release Opus 4.7 and go through the cycle again. My allegiance to these companies is now measured in nerf cycles. I’m a nerf cycle customer.

film42 11mo ago
This is why I migrated my apps that need an LLM to Gemini. No model degradation so far all through the v2.5 model generation. What is Anthropic doing? Swapping for a quantized version of the model?
blurbleblurble 11mo ago
I fully agree that this is what's happening. I'm quite convinced after about a year of using all these tools via the "pro" plans that all these companies are throttling their models in sophisticated ways that have a poorly understood but significant impact on quality and consistency. Gpt-5.1-* are fully nerfed for me at the moment. Maybe they're giving others the real juice but they're not giving it to me. Gpt-5-* gave me quite good results 2 weeks ago, now I'm just getting incoherent crap at 20 minute intervals. Maybe I should just start paying via tokens for a hopefully more consistent experience.
- throwuxiytayq 11mo ago
  y’all hallucinating harder than GPT2 on DMT
  - deleted 11mo ago
    [deleted]
  - koolala 11mo ago
    Do you not believe that Intelligence Throttling exists what-so-ever? It's a lot like overworking a person in real life with too many tasks at once except its a supercomputer.
    - F7F7F7 11mo ago
      Claude uses Haiku to summarize and name all chats. Regardless of what model the user is using. If people don’t think that Anthropic is doing a lot more behind the scenes they are borderline delusional.
    - dcre 11mo ago
      I do not believe it exists.
Capricorn2481 11mo ago
Thank god people are noticing this. I'm pretty sick of companies putting a higher number next to models and programmers taking that at face value. This reminds me of audio production debates about niche hardware emulations, like which company emulated the 1176 compressor the best. The differences between them all are so minute and insignificant, eventually people just insist they can "feel" the difference. Basically, whoever is placeboing the hardest. Such is the case with LLMs. A tool that is already hard to measure because it gives different output with the same repeated input, and now people try to do A/B tests with models that are basically the same. The field has definitely made strides in how small models can be, but I've noticed very little improvement since gpt-4.
lukev 11mo ago
There are two possible explanations for this behavior: the model nerf is real, or there's a perceptual/psychological shift. However, benchmarks exist. And I haven't seen any empirical evidence that the performance of a given model version grows worse over time on benchmarks (in general.) Therefore, some combination of two things are true: 1. The nerf is psychologial, not actual. 2. The nerf is real but in a way that is perceptual to humans, but not benchmarks. #1 seems more plausible to me a priori, but if you aren't inclined to believe that, you should be positively intrigued by #2, since it points towards a powerful paradigm shift of how we think about the capabilities of LLMs in general... it would mean there is an "x-factor" that we're entirely unable to capture in any benchmark to date.
- blurbleblurble 11mo ago
  I'm pretty sure this isn't happening with the API versions as much as with the "pro plan" (loss leader priced) routers. I imagine that there are others like me working on hard problems for long periods with the model setting pegged to high. Why wouldn't the companies throttle us? It could even just be that they just apply simple rate limits and that this degrades the effectiveness of the feedback loop between the person and the model. If I have to wait 20 minutes for GPT-5.1-codex-max medium to look at `git diff` and give a paltry and inaccurate summary (yes this is where things are at for me right now, all this week) it's not going to be productive.
  - fakwandi_priv 11mo ago
    I run the same config but it tends to fly through those commands on the weekends, very noticeable difference. I wouldn’t be surprised that the subscription users have a (much) lower priority. That said I don’t go beyond 70% of my weekly limit so there’s that.
- zsoltkacsandi 11mo ago
  > The nerf is psychologial, not actual Once I tested this, I gave the same task for a model after the release and a couple weeks later. In the first attempt it produced a well-written code that worked beautifully, I started to worry about the jobs of the software engineers. Second attempt was a nightmare, like a butcher acting as a junior developer performing a surgery on a horse. Is this empirical evidence? And this is not only my experience. Calling this phychological is gaslighting.
all2 11mo ago
Interestingly, I canceled my Claude subscription. I've paid through the first week of December, so it dries up on the 7th of December. As soon as I had canceled, Claude Code started performing substantially better. I gave it a design spec (a very loose design spec) and it one-shotted it. I'll grant that it was a collection of docker containers and a web API, but still. I've not seen that level of performance from Claude before, and I'm thinking I'll have to move to 'pay as you go' (pay --> cancel immediately) just to take advantage of this increased performance.
- blinding-streak 11mo ago
  That's really interesting. After cancelling, it goes into retention mode, akin to when one cancels other online services? For example, I cancelled Peacock the other day and it offered a deal of $1.99/mo for 6 months if I stayed. Very intriguing, curious if others have seen this.
  - typpilol 11mo ago
    I got this on the dominos pizza app recently. I clicked the bread sticks by mistake and clocked out, and a pop up came up and offered me the bread sticks for $1.99 as well. So now whenever I get Dominos I click and back out of everything to get any free coupons
    - F7F7F7 11mo ago
      It’s retargeting and it happens much more often than you think. Try the same thing at pretty much any e-commerce store. Works best if you checkout as a guest (using only your email) and get all the way up to payment. A day later you’ll typically get a discount coupon and an invitation to finish checking out.
TIPSIO 11mo ago
Hilarious sarcastic comment but actually true sentiment. For all we know this is just the Opus 4.0 re-released
yesco 11mo ago
With Claude specifically I've grown confident they have been sneakily experimenting with context compression to save money and doing a very bad job at it. However for this same reason one shot batch usage or one off questions & answers that don't depend on larger context windows don't seem to see this degradation.
- unshavedyak 11mo ago
  They added a "How is claude doing?" rating a while back which backs this statement up imo. Tons of A/B tests going on i bet.
  - F7F7F7 11mo ago
    Every time I type in cap case or use a 4 letter word with Claude I’ll get hit with the 1 questions survey. More times than not the answer is 1 (bad, IIRC). Then it’s 2 for fine. I can only ever remember hitting 3 once.
idonotknowwhy 11mo ago
100%. They've been nerfing the model periodically since at least Sonnet 3.5, but this time it's so bad I ended up swapping out to GLM4.6 just to finish off a simple feature.
fHr 11mo ago
haha couldn't have put this better, exactly this
vagab0nd 11mo ago
I did not know this but it's consistent with the behaviors of the CEO.
WNWceAJ9R9Ezc4 11mo ago
Accurate.
nullbio 11mo ago
You're forgetting the step where they write a nefarious paper for their marketing team about the "world-ending dangers" of the capabilities they've discovered within their new model, and push it out to their web of media companies who make bank from the ad-revenue from clicks on their doomsday articles while furthering the regulatory capture goals of the hypocritically Palantir-partnered Anthropic.
- arresin 11mo ago
  And then Dario gives an interview on why open source models should be banned due to _____.
  - rvz 11mo ago
    The predictable business of selling fear; with the goal of control and regulatory capture.
F7F7F7 11mo ago
Couldn’t have said it better myself. I’ve cancelled my x20 two times now and they keep pulling me back.
stingraycharles 11mo ago
I’m disappointed that this type of discourse has now entered HN. I expected a more evidence-based less “nerf cycle” discussion over here.
- Etheryte 11mo ago
  This is nothing new and it's been discussed numerous times. Would you also say we need more evidence that Meta is tracking people?
  - stingraycharles 11mo ago
    We have evidence for that. There is no hard evidence for purposefully model degradation for cost optimization after the initial release, other than a lot of emotional discussions in vibe coding subs.
hacb 11mo ago
it will be just enough time to finish my quarter roadmap and chill until january

film42 11mo ago

This is why I migrated my apps that need an LLM to Gemini. No model degradation so far all through the v2.5 model generation. What is Anthropic doing? Swapping for a quantized version of the model?

blurbleblurble 11mo ago

I fully agree that this is what's happening. I'm quite convinced after about a year of using all these tools via the "pro" plans that all these companies are throttling their models in sophisticated ways that have a poorly understood but significant impact on quality and consistency. Gpt-5.1-* are fully nerfed for me at the moment. Maybe they're giving others the real juice but they're not giving it to me. Gpt-5-* gave me quite good results 2 weeks ago, now I'm just getting incoherent crap at 20 minute intervals. Maybe I should just start paying via tokens for a hopefully more consistent experience.

throwuxiytayq 11mo ago
y’all hallucinating harder than GPT2 on DMT
- deleted 11mo ago
  [deleted]
- koolala 11mo ago
  Do you not believe that Intelligence Throttling exists what-so-ever? It's a lot like overworking a person in real life with too many tasks at once except its a supercomputer.
  - F7F7F7 11mo ago
    Claude uses Haiku to summarize and name all chats. Regardless of what model the user is using. If people don’t think that Anthropic is doing a lot more behind the scenes they are borderline delusional.
  - dcre 11mo ago
    I do not believe it exists.

Capricorn2481 11mo ago

Thank god people are noticing this. I'm pretty sick of companies putting a higher number next to models and programmers taking that at face value. This reminds me of audio production debates about niche hardware emulations, like which company emulated the 1176 compressor the best. The differences between them all are so minute and insignificant, eventually people just insist they can "feel" the difference. Basically, whoever is placeboing the hardest. Such is the case with LLMs. A tool that is already hard to measure because it gives different output with the same repeated input, and now people try to do A/B tests with models that are basically the same. The field has definitely made strides in how small models can be, but I've noticed very little improvement since gpt-4.

lukev 11mo ago

There are two possible explanations for this behavior: the model nerf is real, or there's a perceptual/psychological shift. However, benchmarks exist. And I haven't seen any empirical evidence that the performance of a given model version grows worse over time on benchmarks (in general.) Therefore, some combination of two things are true: 1. The nerf is psychologial, not actual. 2. The nerf is real but in a way that is perceptual to humans, but not benchmarks. #1 seems more plausible to me a priori, but if you aren't inclined to believe that, you should be positively intrigued by #2, since it points towards a powerful paradigm shift of how we think about the capabilities of LLMs in general... it would mean there is an "x-factor" that we're entirely unable to capture in any benchmark to date.

blurbleblurble 11mo ago
I'm pretty sure this isn't happening with the API versions as much as with the "pro plan" (loss leader priced) routers. I imagine that there are others like me working on hard problems for long periods with the model setting pegged to high. Why wouldn't the companies throttle us? It could even just be that they just apply simple rate limits and that this degrades the effectiveness of the feedback loop between the person and the model. If I have to wait 20 minutes for GPT-5.1-codex-max medium to look at `git diff` and give a paltry and inaccurate summary (yes this is where things are at for me right now, all this week) it's not going to be productive.
- fakwandi_priv 11mo ago
  I run the same config but it tends to fly through those commands on the weekends, very noticeable difference. I wouldn’t be surprised that the subscription users have a (much) lower priority. That said I don’t go beyond 70% of my weekly limit so there’s that.
zsoltkacsandi 11mo ago
> The nerf is psychologial, not actual Once I tested this, I gave the same task for a model after the release and a couple weeks later. In the first attempt it produced a well-written code that worked beautifully, I started to worry about the jobs of the software engineers. Second attempt was a nightmare, like a butcher acting as a junior developer performing a surgery on a horse. Is this empirical evidence? And this is not only my experience. Calling this phychological is gaslighting.

all2 11mo ago

Interestingly, I canceled my Claude subscription. I've paid through the first week of December, so it dries up on the 7th of December. As soon as I had canceled, Claude Code started performing substantially better. I gave it a design spec (a very loose design spec) and it one-shotted it. I'll grant that it was a collection of docker containers and a web API, but still. I've not seen that level of performance from Claude before, and I'm thinking I'll have to move to 'pay as you go' (pay --> cancel immediately) just to take advantage of this increased performance.

blinding-streak 11mo ago
That's really interesting. After cancelling, it goes into retention mode, akin to when one cancels other online services? For example, I cancelled Peacock the other day and it offered a deal of $1.99/mo for 6 months if I stayed. Very intriguing, curious if others have seen this.
- typpilol 11mo ago
  I got this on the dominos pizza app recently. I clicked the bread sticks by mistake and clocked out, and a pop up came up and offered me the bread sticks for $1.99 as well. So now whenever I get Dominos I click and back out of everything to get any free coupons
  - F7F7F7 11mo ago
    It’s retargeting and it happens much more often than you think. Try the same thing at pretty much any e-commerce store. Works best if you checkout as a guest (using only your email) and get all the way up to payment. A day later you’ll typically get a discount coupon and an invitation to finish checking out.

TIPSIO 11mo ago

Hilarious sarcastic comment but actually true sentiment. For all we know this is just the Opus 4.0 re-released

yesco 11mo ago

With Claude specifically I've grown confident they have been sneakily experimenting with context compression to save money and doing a very bad job at it. However for this same reason one shot batch usage or one off questions & answers that don't depend on larger context windows don't seem to see this degradation.

unshavedyak 11mo ago
They added a "How is claude doing?" rating a while back which backs this statement up imo. Tons of A/B tests going on i bet.
- F7F7F7 11mo ago
  Every time I type in cap case or use a 4 letter word with Claude I’ll get hit with the 1 questions survey. More times than not the answer is 1 (bad, IIRC). Then it’s 2 for fine. I can only ever remember hitting 3 once.

idonotknowwhy 11mo ago

100%. They've been nerfing the model periodically since at least Sonnet 3.5, but this time it's so bad I ended up swapping out to GLM4.6 just to finish off a simple feature.

fHr 11mo ago

haha couldn't have put this better, exactly this

vagab0nd 11mo ago

I did not know this but it's consistent with the behaviors of the CEO.

WNWceAJ9R9Ezc4 11mo ago

Accurate.

nullbio 11mo ago

You're forgetting the step where they write a nefarious paper for their marketing team about the "world-ending dangers" of the capabilities they've discovered within their new model, and push it out to their web of media companies who make bank from the ad-revenue from clicks on their doomsday articles while furthering the regulatory capture goals of the hypocritically Palantir-partnered Anthropic.

arresin 11mo ago
And then Dario gives an interview on why open source models should be banned due to _____.
- rvz 11mo ago
  The predictable business of selling fear; with the goal of control and regulatory capture.

F7F7F7 11mo ago

Couldn’t have said it better myself. I’ve cancelled my x20 two times now and they keep pulling me back.

stingraycharles 11mo ago

I’m disappointed that this type of discourse has now entered HN. I expected a more evidence-based less “nerf cycle” discussion over here.

Etheryte 11mo ago
This is nothing new and it's been discussed numerous times. Would you also say we need more evidence that Meta is tracking people?
- stingraycharles 11mo ago
  We have evidence for that. There is no hard evidence for purposefully model degradation for cost optimization after the initial release, other than a lot of emotional discussions in vibe coding subs.

hacb 11mo ago

it will be just enough time to finish my quarter roadmap and chill until january