4 ms·

I'm not sure how much I trust Anthropic recently. This coming right after a noticeable downgrade just makes me think Opus 4.7 is going to be the same Opus i wa

I'm not sure how much I trust Anthropic recently. This coming right after a noticeable downgrade just makes me think Opus 4.7 is going to be the same Opus i was experiencing a few months ago rather than actual performance boost. Anthropic need to build back some trust and communicate throtelling/reasoning caps more clearly.

aurareturn 6mo ago
They don't have enough compute for all their customers. OpenAI bet on more compute early on which prompted people to say they're going to go bankrupt and collapse. But now it seems like it's a major strategic advantage. They're 2x'ing usage limits on Codex plans to steal CC customers and it seems to be working. It seems like 90% of Claude's recent problems are strictly lack of compute related.
- endymion-light 6mo ago
  Honestly, I personally would rather a time-out than the quality of my response noticably downgrading. I think what I found especially distrustful is the responses from employees claiming that no degredation has occured. An honest response of "Our compute is busy, use X model?" would be far better than silent downgrading.
  - Barbing 6mo ago
    Are they convinced that claiming they have technical issues while continuing to adjust their internal levers to choose which customers to serve is holistically the best path?
- Wojtkie 6mo ago
  Is that why Anthropic recently gave out free credits for use in off-hours? Possibly an attempt to more evenly distribute their compute load throughout the day?
  - DaedalusII 6mo ago
    i suspect they get cheap off peak electricity and compute is cheaper at those times
    - jedberg 6mo ago
      That's not really how datacenter power works. It's usually a bulk buy with a 95th percentile usage.
    - cheeze 6mo ago
      I think it's a lot simpler than that. At peak, gpus are all running hot. During low volume, they aren't.
  - ac29 6mo ago
    That was the carrot, but it was followed immediately by the stick (5 hour session limits were halved during peak hours)
  - troupo 6mo ago
    > Is that why Anthropic recently gave out free credits for use in off-hours? That was the carrot for the stick. The limits and the issues were never officially recognized or communicated. Neither have been the "off-hours credits". You would only know about them if you logged in to your dashboard. When is the last time you logged in there?
- mattas 6mo ago
  Hard for me to reconcile the idea that they don't have enough compute with the idea that they are also losing money to subsidies.
  - Glemllksdf 6mo ago
    They are loosing money because the model training costs billions.
    - ACCount37 6mo ago
      Model inference compute over model lifetime is ~10x of model training compute now for major providers. Expected to climb as demand for AI inference rises.
      - howdareme9 6mo ago
        They are constantly training and getting rid of older models, they are losing money
        ACCount37 6mo ago
        Which part of "over model lifetime" did you not understand?
        adgjlsfhk1 6mo ago
        That's not a sufficient condition for profitability if both inference and scaling costs continue to increase over time.
      - Glemllksdf 6mo ago
        For sure and growth also costs money for buying DCs etc.
  - anthonypasq 6mo ago
    they clearly arent losing money, i dont understand why people think this is true
- _boffin_ 6mo ago
  You state your hypnosis quite confidently. Can you tell me how taking down authentication many times is related to GPU capacity?
- Glemllksdf 6mo ago
  Its a hard game to play anyway. Anthropics revenue is increasing very fast. OpenAI though made crazy claims after all its responsible for the memory prices. In parallel anthropic announced partnership with google and broadcom for gigawatts of TPU chips while also announcing their own 50 Billion invest in compute. OpenAI always believed in compute though and i'm pretty sure plenty of people want to see what models 10x or 100x or 1000x can do.
- MikeNotThePope 6mo ago
  Prepare for the prices to go up!
- sagarpatil 6mo ago
  It worked. Although I have a Claude Code subscription, I got the ChatGPT Pro plan, and 5.4 xHigh at 1.5x speed was better than 4.6 with adaptive thinking disabled. I was working all day, about 8 hours, and did not run into any limits. 5.4 surprised me many times by doing things I usually would not do myself, because I am lazy, so yeah, I am sticking with 5.4 for now until all the Claude drama is over.
- arispen 6mo ago
  I bet that's the real reason why they're not releasing Mythos ;)
- HarHarVeryFunny 6mo ago
  Betting on continued exponential growth is basically a game of chicken. Growth has to slow down and level off at some point as adoption and usage saturates. It's a bit like playing roulette by always betting on black and doubling your bet every time you lose. When you eventually, inevitably, do lose, your loss is going to be huge because you've been doubling your bet at each stage. With LLM model generations and investment, it goes something like this. Let's say profits have been doubling year over year for each new model/investment cycle, and you want to bet on this doubling continuing forever. Year 1 you get $10B in profit, and spend $20B on extra capacity for next year Year 2 you get $20B in profit, and spend $40B on extra capacity Year 3 you get $30B in profit, and spend $??? on extra capacity You're already in trouble. Profit growth from Year 2 to 3 was "only" 50% vs the doubling you were gambling on, so you've now lost $10B ($40B spent only earnt you $30B of profit), and what are you going to do? Double down like the roulette player? The longer the pattern of profit doubling goes, before it slows down, the worse it will end for you, since your bets are doubling each year. Saying "woo hoo, look at me! risk pays!" is a bit like saying the same while playing russian (not casino) roulette for money. I worked for Acorn Computers UK in the early 80's and saw something similar firsthand. The brand new personal computer market was exploding, a once in a lifetime phenomenon, that no-one knew how to forecast. To make matters worse the market was highly seasonal with most sales at xmas, so the company had to guess what continued year-on-year exponential growth might look like (brand new market - no-one had a clue), and plan/spend ahead and stock warehouses full of computers ready for xmas. Sadly Acorn took the Sam Altman highly optimistic/irresponsible approach, got the forecast wrong, and was left with a huge warehouse full of rapidly depreciating computers. The company never fully recovered, although ARM rose out of the ashes.
GaryBluto 6mo ago
> This coming right after a noticeable downgrade just makes me think Opus 4.7 is going to be the same Opus i was experiencing a few months ago rather than actual performance boost. If they are indeed doing this, I wonder how long they can keep it up?
ffsm8 6mo ago
Usually they're hemorrhaging performance while training. From that it's pretty likely they were training mythos for the last few weeks, and then distilling it to opus 4.7 Pure speculation of course, but would also explain the sudden performance gains for mythos - and why they're not releasing it to the general public (because it's the undistilled version which is too expensive to run)
- utopcell 6mo ago
  Mythos is speculated to have 10 trillion parameters. Almost certainly they were training it for months.
  - ffsm8 6mo ago
    Naturally, it is however noticeable that in the lead up to a model release we always get massively degraded performance for the preceeding few weeks It's been like that for each model release within the last year
    - merlindru 6mo ago
      but how true is this? this is almost impossible to measure and those that do[1] find no significant difference i personally haven't noticed any downgrade at all. it's entirely possible there's a mass delusion going on where everyone gets wowed by 4.6 initially, then accepts the new baseline and gets used to it, then thinks that baseline is no longer impressive and thus degraded it doesn't help that anthropic changed defaults for its claude code harness for all users suddenly the best and only evidence i've seen for actual degradation is that the web version of opus 4.6 failed the car wash test, and since you cannot simply choose to "disable adaptive thinking" and other parameters with the web version, you truly may have gotten a worse product [1] https://marginlab.ai/trackers/claude-code-historical-performance/ https://marginlab.ai/trackers/claude-code-historical-perform...
batshit_beaver 6mo ago
What I want to know is why my bedrock-backed Claude gets dumber along with commercial users. Surely they're not touching the bedrock model itself. Only thing I can think of is that updates to the harness are the main cause of performance degradation.
- deleted 6mo ago
  [deleted]
- b--l 6mo ago
  If we learned anything from the code leak is that they essentially do not know what is in the blackbox of the code for that 500k line mass. So that's plausible.
- arcatech 6mo ago
  I believe AWS forwards requests (for Clause models) to Anthropic’s servers. They don’t host those models.
3s 6mo ago
Not to mention their recent integration of Persona ID verification - that was the last straw for me.
dear_prudence 6mo ago
same experience, it has not been a reliable tool for the last few months

aurareturn 6mo ago

They don't have enough compute for all their customers. OpenAI bet on more compute early on which prompted people to say they're going to go bankrupt and collapse. But now it seems like it's a major strategic advantage. They're 2x'ing usage limits on Codex plans to steal CC customers and it seems to be working. It seems like 90% of Claude's recent problems are strictly lack of compute related.

endymion-light 6mo ago
Honestly, I personally would rather a time-out than the quality of my response noticably downgrading. I think what I found especially distrustful is the responses from employees claiming that no degredation has occured. An honest response of "Our compute is busy, use X model?" would be far better than silent downgrading.
- Barbing 6mo ago
  Are they convinced that claiming they have technical issues while continuing to adjust their internal levers to choose which customers to serve is holistically the best path?
Wojtkie 6mo ago
Is that why Anthropic recently gave out free credits for use in off-hours? Possibly an attempt to more evenly distribute their compute load throughout the day?
- DaedalusII 6mo ago
  i suspect they get cheap off peak electricity and compute is cheaper at those times
  - jedberg 6mo ago
    That's not really how datacenter power works. It's usually a bulk buy with a 95th percentile usage.
  - cheeze 6mo ago
    I think it's a lot simpler than that. At peak, gpus are all running hot. During low volume, they aren't.
- ac29 6mo ago
  That was the carrot, but it was followed immediately by the stick (5 hour session limits were halved during peak hours)
- troupo 6mo ago
  > Is that why Anthropic recently gave out free credits for use in off-hours? That was the carrot for the stick. The limits and the issues were never officially recognized or communicated. Neither have been the "off-hours credits". You would only know about them if you logged in to your dashboard. When is the last time you logged in there?
mattas 6mo ago
Hard for me to reconcile the idea that they don't have enough compute with the idea that they are also losing money to subsidies.
- Glemllksdf 6mo ago
  They are loosing money because the model training costs billions.
  - ACCount37 6mo ago
    Model inference compute over model lifetime is ~10x of model training compute now for major providers. Expected to climb as demand for AI inference rises.
    - howdareme9 6mo ago
      They are constantly training and getting rid of older models, they are losing money
      - ACCount37 6mo ago
        Which part of "over model lifetime" did you not understand?
        adgjlsfhk1 6mo ago
        That's not a sufficient condition for profitability if both inference and scaling costs continue to increase over time.
    - Glemllksdf 6mo ago
      For sure and growth also costs money for buying DCs etc.
- anthonypasq 6mo ago
  they clearly arent losing money, i dont understand why people think this is true
_boffin_ 6mo ago
You state your hypnosis quite confidently. Can you tell me how taking down authentication many times is related to GPU capacity?
Glemllksdf 6mo ago
Its a hard game to play anyway. Anthropics revenue is increasing very fast. OpenAI though made crazy claims after all its responsible for the memory prices. In parallel anthropic announced partnership with google and broadcom for gigawatts of TPU chips while also announcing their own 50 Billion invest in compute. OpenAI always believed in compute though and i'm pretty sure plenty of people want to see what models 10x or 100x or 1000x can do.
MikeNotThePope 6mo ago
Prepare for the prices to go up!
sagarpatil 6mo ago
It worked. Although I have a Claude Code subscription, I got the ChatGPT Pro plan, and 5.4 xHigh at 1.5x speed was better than 4.6 with adaptive thinking disabled. I was working all day, about 8 hours, and did not run into any limits. 5.4 surprised me many times by doing things I usually would not do myself, because I am lazy, so yeah, I am sticking with 5.4 for now until all the Claude drama is over.
arispen 6mo ago
I bet that's the real reason why they're not releasing Mythos ;)
HarHarVeryFunny 6mo ago
Betting on continued exponential growth is basically a game of chicken. Growth has to slow down and level off at some point as adoption and usage saturates. It's a bit like playing roulette by always betting on black and doubling your bet every time you lose. When you eventually, inevitably, do lose, your loss is going to be huge because you've been doubling your bet at each stage. With LLM model generations and investment, it goes something like this. Let's say profits have been doubling year over year for each new model/investment cycle, and you want to bet on this doubling continuing forever. Year 1 you get $10B in profit, and spend $20B on extra capacity for next year Year 2 you get $20B in profit, and spend $40B on extra capacity Year 3 you get $30B in profit, and spend $??? on extra capacity You're already in trouble. Profit growth from Year 2 to 3 was "only" 50% vs the doubling you were gambling on, so you've now lost $10B ($40B spent only earnt you $30B of profit), and what are you going to do? Double down like the roulette player? The longer the pattern of profit doubling goes, before it slows down, the worse it will end for you, since your bets are doubling each year. Saying "woo hoo, look at me! risk pays!" is a bit like saying the same while playing russian (not casino) roulette for money. I worked for Acorn Computers UK in the early 80's and saw something similar firsthand. The brand new personal computer market was exploding, a once in a lifetime phenomenon, that no-one knew how to forecast. To make matters worse the market was highly seasonal with most sales at xmas, so the company had to guess what continued year-on-year exponential growth might look like (brand new market - no-one had a clue), and plan/spend ahead and stock warehouses full of computers ready for xmas. Sadly Acorn took the Sam Altman highly optimistic/irresponsible approach, got the forecast wrong, and was left with a huge warehouse full of rapidly depreciating computers. The company never fully recovered, although ARM rose out of the ashes.

GaryBluto 6mo ago

> This coming right after a noticeable downgrade just makes me think Opus 4.7 is going to be the same Opus i was experiencing a few months ago rather than actual performance boost. If they are indeed doing this, I wonder how long they can keep it up?

ffsm8 6mo ago

Usually they're hemorrhaging performance while training. From that it's pretty likely they were training mythos for the last few weeks, and then distilling it to opus 4.7 Pure speculation of course, but would also explain the sudden performance gains for mythos - and why they're not releasing it to the general public (because it's the undistilled version which is too expensive to run)

utopcell 6mo ago
Mythos is speculated to have 10 trillion parameters. Almost certainly they were training it for months.
- ffsm8 6mo ago
  Naturally, it is however noticeable that in the lead up to a model release we always get massively degraded performance for the preceeding few weeks It's been like that for each model release within the last year
  - merlindru 6mo ago
    but how true is this? this is almost impossible to measure and those that do[1] find no significant difference i personally haven't noticed any downgrade at all. it's entirely possible there's a mass delusion going on where everyone gets wowed by 4.6 initially, then accepts the new baseline and gets used to it, then thinks that baseline is no longer impressive and thus degraded it doesn't help that anthropic changed defaults for its claude code harness for all users suddenly the best and only evidence i've seen for actual degradation is that the web version of opus 4.6 failed the car wash test, and since you cannot simply choose to "disable adaptive thinking" and other parameters with the web version, you truly may have gotten a worse product [1] https://marginlab.ai/trackers/claude-code-historical-performance/ https://marginlab.ai/trackers/claude-code-historical-perform...

batshit_beaver 6mo ago

What I want to know is why my bedrock-backed Claude gets dumber along with commercial users. Surely they're not touching the bedrock model itself. Only thing I can think of is that updates to the harness are the main cause of performance degradation.

deleted 6mo ago
[deleted]
b--l 6mo ago
If we learned anything from the code leak is that they essentially do not know what is in the blackbox of the code for that 500k line mass. So that's plausible.
arcatech 6mo ago
I believe AWS forwards requests (for Clause models) to Anthropic’s servers. They don’t host those models.

3s 6mo ago

Not to mention their recent integration of Persona ID verification - that was the last straw for me.

dear_prudence 6mo ago

same experience, it has not been a reliable tool for the last few months