5 ms·

My experience is similar. At first Claude was super smart and get even very complicated things right. Now even super simple tasks are almost impossible to finis

My experience is similar. At first Claude was super smart and get even very complicated things right. Now even super simple tasks are almost impossible to finish right, even if I really chop things into small steps. Also it's much slower even on Pro account than a few weeks ago.

strictnein 1y ago
I'm on the $200 / month account and its also slower than a few weeks ago. And struggling more and more. I used to think of it as a decent sr dev working alongside me. Not it feels like an untrained intern that takes 4-5 shots to get things right. Hallucinated tables, columns, and HTML templates are its new favorite thing. And calling things "done" that aren't even half done and don't work in the slightest.
- brookst 1y ago
  Same plan, same experience. Trying to get it to develop and execute tests and it frequently modifies the test to succeed even if the libraries it calls fail, and then explains that it’s doing so because the test itself works but the underlying app has errors. Yes, I know. That’s what the test was for.
  - zarzavat 1y ago
    Anthropic, if you're listening, please allow zoned access enforcement within files. I want to be able to say "this section of the file is for testing", delineated by comments, and forbid Claude from editing it without permission. My fear when using Claude is that it will change a test and I won't notice. Splitting tests into different files works but it's often not feasible, e.g. if I want to write unit tests for a symbol that is not exported.
    - geeunits 1y ago
      Does already, read the docs
      - boie0025 1y ago
        I think a link would have been far more helpful than "RTFM". Especially for those of us reading this exchange outside of the line of fire.
        geeunits 1y ago
        Don't put the onus (Opus!) on me! Just a dad approach to helping. If there's enough time to writ prose about the problem you could at least rtfm first!
        simonw 1y ago
        If you know something is covered by the documentation it's useful to provide a link, especially if that documentation is difficult to find. (I couldn't find that documentation when I went looking just now.)
        geeunits 1y ago
        Step 1: https://docs.anthropic.com https://docs.anthropic.com Step 2: Type 'Allowed Tools' Step 3: Click: https://docs.anthropic.com/en/docs/claude-code/sdk/sdk-headless#configuration-options https://docs.anthropic.com/en/docs/claude-code/sdk/sdk-headl... Step 4: Read Step 5: Example --allowedTools "Read,Grep,WebSearch" Step 6: Profit?
        simonw 1y ago
        The original question was about this: > allow zoned access enforcement within files. I want to be able to say "this section of the file is for testing", delineated by comments, and forbid Claude from editing it without permission.
        holbrad 1y ago
        So you've completely misunderstood what the discussion is about... Maybe rtft ? Read the fucking thread.
    - blyat 1y ago
      I've had some middling success with this by utilizing CLAUDE.md and language features. Two approaches in C#: 1) use partial classes and create a 'rule' in CLAUDE.md to never touch named files, e.g. User.cs (edits allowed) User.Protected.cs (not allowed by convention) and 2) a no-AI-allowed attribute, e.g. [DontModifyThisClassOrAttributeOrMethodOrWhatever] and instructions to never modify said target. Can be much more granular and Claude Code seems to respect it.
- keyle 1y ago
  There must be a term coined for AI degradation... At least with local LLM, it's crap, but it's consistent crap!
  - beefnugs 1y ago
    Dynamic spurious profit probing. See how many users N times their usage without giving up forever. They have to do something because you can't really fist advertisements into an api
    - dmix 1y ago
      OP is paying $200/m and anthropic is very much in the hyper funded growth stage. I very much doubt they are going accountant mode on it yet Likely the common young startup issues: a mix of scaling issues and poorly implemented changes. Improve one thing, make other stuff worse etc
      - jazzyjackson 1y ago
        Probably not accountant mode but haven't they always had daily quotas that get used up? Like they don't want everyone hitting the service nonstop because they don't have enough GPUs to run inference at peak times of day? So it could be a matter of serving more highly quantized model because giving bad results has higher user retention than "try again later"
- cyanydeez 1y ago
  Gotta assume theyre reducing overall compute with smaller models cause 200$ aint squat for their investment.
ranguna 1y ago
It's still pretty good on my side. I'm just paying for the pro version.
insane_dreamer 1y ago
On Max and also find it slower recently. Also yesterday tried to use it to debug some AWS issue and it tried to send me down so many wrong paths, and suggested changes that were either plain wrong or had unintended consequences, that if I didn't actually know my stuff and had followed blindly, the results would have been pretty bad or at least a huge time waster. When I called it out it would quickly reverse course ("You're right of course!") and it did provide some helpful snippets but I was unimpressed. What I find it excellent at is for throw-away scripts to do small jobs or automate little things--stuff I could do but would take me a lot longer (especially in bash).

strictnein 1y ago

I'm on the $200 / month account and its also slower than a few weeks ago. And struggling more and more. I used to think of it as a decent sr dev working alongside me. Not it feels like an untrained intern that takes 4-5 shots to get things right. Hallucinated tables, columns, and HTML templates are its new favorite thing. And calling things "done" that aren't even half done and don't work in the slightest.

brookst 1y ago
Same plan, same experience. Trying to get it to develop and execute tests and it frequently modifies the test to succeed even if the libraries it calls fail, and then explains that it’s doing so because the test itself works but the underlying app has errors. Yes, I know. That’s what the test was for.
- zarzavat 1y ago
  Anthropic, if you're listening, please allow zoned access enforcement within files. I want to be able to say "this section of the file is for testing", delineated by comments, and forbid Claude from editing it without permission. My fear when using Claude is that it will change a test and I won't notice. Splitting tests into different files works but it's often not feasible, e.g. if I want to write unit tests for a symbol that is not exported.
  - geeunits 1y ago
    Does already, read the docs
    - boie0025 1y ago
      I think a link would have been far more helpful than "RTFM". Especially for those of us reading this exchange outside of the line of fire.
      - geeunits 1y ago
        Don't put the onus (Opus!) on me! Just a dad approach to helping. If there's enough time to writ prose about the problem you could at least rtfm first!
        simonw 1y ago
        If you know something is covered by the documentation it's useful to provide a link, especially if that documentation is difficult to find. (I couldn't find that documentation when I went looking just now.)
        geeunits 1y ago
        Step 1: https://docs.anthropic.com https://docs.anthropic.com Step 2: Type 'Allowed Tools' Step 3: Click: https://docs.anthropic.com/en/docs/claude-code/sdk/sdk-headless#configuration-options https://docs.anthropic.com/en/docs/claude-code/sdk/sdk-headl... Step 4: Read Step 5: Example --allowedTools "Read,Grep,WebSearch" Step 6: Profit?
        simonw 1y ago
        The original question was about this: > allow zoned access enforcement within files. I want to be able to say "this section of the file is for testing", delineated by comments, and forbid Claude from editing it without permission.
        holbrad 1y ago
        So you've completely misunderstood what the discussion is about... Maybe rtft ? Read the fucking thread.
  - blyat 1y ago
    I've had some middling success with this by utilizing CLAUDE.md and language features. Two approaches in C#: 1) use partial classes and create a 'rule' in CLAUDE.md to never touch named files, e.g. User.cs (edits allowed) User.Protected.cs (not allowed by convention) and 2) a no-AI-allowed attribute, e.g. [DontModifyThisClassOrAttributeOrMethodOrWhatever] and instructions to never modify said target. Can be much more granular and Claude Code seems to respect it.
keyle 1y ago
There must be a term coined for AI degradation... At least with local LLM, it's crap, but it's consistent crap!
- beefnugs 1y ago
  Dynamic spurious profit probing. See how many users N times their usage without giving up forever. They have to do something because you can't really fist advertisements into an api
  - dmix 1y ago
    OP is paying $200/m and anthropic is very much in the hyper funded growth stage. I very much doubt they are going accountant mode on it yet Likely the common young startup issues: a mix of scaling issues and poorly implemented changes. Improve one thing, make other stuff worse etc
    - jazzyjackson 1y ago
      Probably not accountant mode but haven't they always had daily quotas that get used up? Like they don't want everyone hitting the service nonstop because they don't have enough GPUs to run inference at peak times of day? So it could be a matter of serving more highly quantized model because giving bad results has higher user retention than "try again later"
cyanydeez 1y ago
Gotta assume theyre reducing overall compute with smaller models cause 200$ aint squat for their investment.

ranguna 1y ago

It's still pretty good on my side. I'm just paying for the pro version.

insane_dreamer 1y ago

On Max and also find it slower recently. Also yesterday tried to use it to debug some AWS issue and it tried to send me down so many wrong paths, and suggested changes that were either plain wrong or had unintended consequences, that if I didn't actually know my stuff and had followed blindly, the results would have been pretty bad or at least a huge time waster. When I called it out it would quickly reverse course ("You're right of course!") and it did provide some helpful snippets but I was unimpressed. What I find it excellent at is for throw-away scripts to do small jobs or automate little things--stuff I could do but would take me a lot longer (especially in bash).