3 ms·

> - No trust that they won't nerf the tool/model behind the feature I actually trust that they will.

by palata 7mo ago

gardenhedge 7mo ago
Yeah, I build my workflows with two things in mind: 1) that AI will be more advanced in the future 2) that the AI I am using will be worse in the future
- freedomben 7mo ago
  Same! I actually have some comments in my codebase now like this one: # Note: This is inefficient, but deterministic and predictable. Previous attempts at improvements led to hard-to-predict bugs and were scrapped. TODO improve this function when AI gets better I don't love it or even like it, but it is realistic.
dvfjsdhgfv 7mo ago
I believe the current game everybody plays is: * make sure the model maxes out all benchmarks * release it * after some time, nerf it * repeat the same with the next model However, the net sum is positive: in general, models from 2026 are better than those from 2024.
- snek_case 7mo ago
  I guess there's a pretty clear incentive to nerf the current model right before the next model is about to come out.
  - chinathrow 7mo ago
    Wouldn't that amount to fraud?
    - deleted 7mo ago
      [deleted]
    - tomwojcik 7mo ago
      Serious question, do we actually know what we're paying for? All I know is it's access to models via cli, aka Claude Code. We don't know what models they use, how system prompt changes or what are the actual rate limits (Yet Anthropic will become 1 trillion dollars company in a moment).
      - xienze 7mo ago
        > We don't know what models they use, how system prompt changes or what are the actual rate limits (Yet Anthropic will become 1 trillion dollars company in a moment). Not just that, but there’s really no way to come to an objective consensus of how well the model is performing in the first place. See: literally every thread discussing a Claude outage or change of some kind. “Opus is absolutely incredible, it’s one shotting work that would take me months” immediately followed by “no it’s totally nerfed now, it can’t even implement bubble sort for me.”
        ElFitz 7mo ago
        > See: literally every thread discussing a Claude outage or change of some kind. “Opus is absolutely incredible, it’s one shotting work that would take me months” immediately followed by “no it’s totally nerfed now, it can’t even implement bubble sort for me.” Funny: I’m literally, at this very moment, working on a way to monitor that across users. Wasn’t the initial goal, but it should do that nicely as well ^^
        a1o 7mo ago
        I feel like if I start something from scratch with it it gets what feels like 80% right, but then it takes a lot more time to do the last 20%, and if you decide to change scope after or just be more specific it is like it gets dumber the longer you work with it. If you can think truly modular and spend a ton of time breaking your problem in small units, and then work in your units separately then maybe what it does could be maintainable. But even there I am unsure. I spent an entire day trying to get it to do a node graph right - like the visual of it - and it is still so so. But like a single small script that does a specific small thing, yeah, that it can do. You still better make sure you can test it easily though.
- _blk 7mo ago
  yup, after the token-increase from CC from two weeks ago, I'm now consistently filling the 1M context window that never went above 30-40% a few days ago. Did they turn it off? I used to see the Co-Authored by Opus 4.6 (1M Context Window) in git commits, now the advert line is gone. I never turned it on or off, maybe the defaults changed but /model doesn't show two different context sizes for Opus 4.6 I never asked for a 1M context window, then I got it and it was nice, now it's as if it was gone again .. no biggie but if they had advertised it as a free-trial (which it feels like) I wouldn't have opted in. Anyways, seems I'm just ranting, I still like Claude, yes but nonetheless it still feels like the game you described above.
  - robwwilliams 7mo ago
    Yep; second time in five months we have gone from 1 million back to 200 thousand.
    - _blk 7mo ago
      hmm, I just reverted to 2.1.98 and now with /model default has the (1M context) and opus is without (200k) .. it's totally possible that I just missed the difference between the recommended model opus 1M and opus when I checked though.
  - dr_kiszonka 7mo ago
    The default prompt cache TTL changed from 1 hour to 5 minutes. Maybe this is what you are experiencing.
  - troupo 7mo ago
    They are now literally blaming users for using their product as advertised: https://x.com/lydiahallie/status/2039800718371307603 https://x.com/lydiahallie/status/2039800718371307603 --- start quote --- Digging into reports, most of the fastest burn came down to a few token-heavy patterns. Some tips: • Sonnet 4.6 is the better default on Pro. Opus burns roughly twice as fast. Switch at session start. • Lower the effort level or turn off extended thinking when you don't need deep reasoning. Switch at session start. • Start fresh instead of resuming large sessions that have been idle ~1h • Cap your context window, long sessions cost more CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000 --- end quote --- https://x.com/bcherny/status/2043163965648515234 https://x.com/bcherny/status/2043163965648515234 --- start quote --- We defaulted to medium [reasoning] as a result of user feedback about Claude using too many tokens. When we made the change, we (1) included it in the changelog and (2) showed a dialog when you opened Claude Code so you could choose to opt out. Literally nothing sneaky about it — this was us addressing user feedback in an obvious and explicit way. --- end quote ---

> - No trust that they won't nerf the tool/model behind the feature I actually trust that they will.

by palata 7mo ago

> - No trust that they won't nerf the tool/model behind the feature I actually trust that they will.

Yeah, I build my workflows with two things in mind: 1) that AI will be more advanced in the future 2) that the AI I am using will be worse in the future

freedomben 7mo ago
Same! I actually have some comments in my codebase now like this one: # Note: This is inefficient, but deterministic and predictable. Previous attempts at improvements led to hard-to-predict bugs and were scrapped. TODO improve this function when AI gets better I don't love it or even like it, but it is realistic.

dvfjsdhgfv 7mo ago

I believe the current game everybody plays is: * make sure the model maxes out all benchmarks * release it * after some time, nerf it * repeat the same with the next model However, the net sum is positive: in general, models from 2026 are better than those from 2024.

snek_case 7mo ago
I guess there's a pretty clear incentive to nerf the current model right before the next model is about to come out.
- chinathrow 7mo ago
  Wouldn't that amount to fraud?
  - deleted 7mo ago
    [deleted]
  - tomwojcik 7mo ago
    Serious question, do we actually know what we're paying for? All I know is it's access to models via cli, aka Claude Code. We don't know what models they use, how system prompt changes or what are the actual rate limits (Yet Anthropic will become 1 trillion dollars company in a moment).
    - xienze 7mo ago
      > We don't know what models they use, how system prompt changes or what are the actual rate limits (Yet Anthropic will become 1 trillion dollars company in a moment). Not just that, but there’s really no way to come to an objective consensus of how well the model is performing in the first place. See: literally every thread discussing a Claude outage or change of some kind. “Opus is absolutely incredible, it’s one shotting work that would take me months” immediately followed by “no it’s totally nerfed now, it can’t even implement bubble sort for me.”
      - ElFitz 7mo ago
        > See: literally every thread discussing a Claude outage or change of some kind. “Opus is absolutely incredible, it’s one shotting work that would take me months” immediately followed by “no it’s totally nerfed now, it can’t even implement bubble sort for me.” Funny: I’m literally, at this very moment, working on a way to monitor that across users. Wasn’t the initial goal, but it should do that nicely as well ^^
      - a1o 7mo ago
        I feel like if I start something from scratch with it it gets what feels like 80% right, but then it takes a lot more time to do the last 20%, and if you decide to change scope after or just be more specific it is like it gets dumber the longer you work with it. If you can think truly modular and spend a ton of time breaking your problem in small units, and then work in your units separately then maybe what it does could be maintainable. But even there I am unsure. I spent an entire day trying to get it to do a node graph right - like the visual of it - and it is still so so. But like a single small script that does a specific small thing, yeah, that it can do. You still better make sure you can test it easily though.
_blk 7mo ago
yup, after the token-increase from CC from two weeks ago, I'm now consistently filling the 1M context window that never went above 30-40% a few days ago. Did they turn it off? I used to see the Co-Authored by Opus 4.6 (1M Context Window) in git commits, now the advert line is gone. I never turned it on or off, maybe the defaults changed but /model doesn't show two different context sizes for Opus 4.6 I never asked for a 1M context window, then I got it and it was nice, now it's as if it was gone again .. no biggie but if they had advertised it as a free-trial (which it feels like) I wouldn't have opted in. Anyways, seems I'm just ranting, I still like Claude, yes but nonetheless it still feels like the game you described above.
- robwwilliams 7mo ago
  Yep; second time in five months we have gone from 1 million back to 200 thousand.
  - _blk 7mo ago
    hmm, I just reverted to 2.1.98 and now with /model default has the (1M context) and opus is without (200k) .. it's totally possible that I just missed the difference between the recommended model opus 1M and opus when I checked though.
- dr_kiszonka 7mo ago
  The default prompt cache TTL changed from 1 hour to 5 minutes. Maybe this is what you are experiencing.
- troupo 7mo ago
  They are now literally blaming users for using their product as advertised: https://x.com/lydiahallie/status/2039800718371307603 https://x.com/lydiahallie/status/2039800718371307603 --- start quote --- Digging into reports, most of the fastest burn came down to a few token-heavy patterns. Some tips: • Sonnet 4.6 is the better default on Pro. Opus burns roughly twice as fast. Switch at session start. • Lower the effort level or turn off extended thinking when you don't need deep reasoning. Switch at session start. • Start fresh instead of resuming large sessions that have been idle ~1h • Cap your context window, long sessions cost more CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000 --- end quote --- https://x.com/bcherny/status/2043163965648515234 https://x.com/bcherny/status/2043163965648515234 --- start quote --- We defaulted to medium [reasoning] as a result of user feedback about Claude using too many tokens. When we made the change, we (1) included it in the changelog and (2) showed a dialog when you opened Claude Code so you could choose to opt out. Literally nothing sneaky about it — this was us addressing user feedback in an obvious and explicit way. --- end quote ---