9 ms·

my issue hasn't been for a long time now that the code they write works or doesn't work. My issues all stem from that it works, but does the wrong thing

jmathai 9mo ago
I think it's worth understanding why. Because that's not everyone's experience and there's a chance you could make a change such that you find it extremely useful. There's a lesser chance that you're working on a code base that Claude Code just isn't capable of helping with.
simonw 9mo ago
If it does the wrong thing you tell it what the right thing is and have it try again. With the latest models if you're clear enough with your requirements you'll usually find it does the right thing on the first try.
- cadamsdotcom 9mo ago
  Even better, have it write code to describe the right thing then run its code against that, taking yourself out of that loop.
- giancarlostoro 9mo ago
  And if you've told it too many times to fix it, tell it someone has a gun to your head, for some reason it almost always gets it right this very next time.
  - dare944 9mo ago
    If you're a developer at the dawn of the AI revolution, there is absolutely a gun to your head.
    - giancarlostoro 9mo ago
      Yeah, if anyone can truly afford the AI empire. Remember all these "leading" companies are running it at a loss, so most companies paying for it are severely underpaying the cost of it all. We would need an insane technological breakthrough of unlimited memory and power before I start to worry, and at that point, I'll just look for a new career.
- aurumque 9mo ago
  In a circuitous way, you can rather successfully have one agent write a specification and another one execute the code changes. Claude code has a planning mode that lets you work with the model to create a robust specification that can then be executed, asking the sort of leading questions for which it already seems to know it could make an incorrect assumption. I say 'agent' but I'm really just talking about separate model contexts, nothing fancy.
  - mikestorrent 9mo ago
    Cursor's planning functionality is very similar and I have found that I can even use "cheap" models like their Composer-1 and get great results in the planning phase, and then turn on Sonnet or Opus to actually produce the plan. 90% of the stuff I need to argue about is during the planning phase, so I save a ton of tokens and rework just making a really good spec. It turns out that Waterfall was always the correct method, it's just really slow ;)
    - aurumque 9mo ago
      Did you know that software specifications used to be almost entirely flow charts? There is something to be said for that and waterfall.
- GoatInGrey 9mo ago
  There are several rubs with that operating protocol extending beyond the "you're holding it wrong" claim. 1) There exists a threshold, only identifiable in retrospect, past which it would have been faster to locate or write the code yourself than to navigate the LLM's correction loop or otherwise ensure one-shot success. 2) The intuition and motivations of LLMs derive from a latent space that the LLM cannot actually access. I cannot get a reliable answer on why the LLM chose the approaches it did; it can only retroactively confabulate. Unlike human developers who can recall off-hand, or at least review associated tickets and meeting notes to jog their memory. The LLM prompter always documenting sufficiently to bridge this LLM provenance gap hits rub #1. 3) Gradually building prompt dependency where one's ability to take over from the LLM declines and one can no longer answer questions or develop at the same velocity themselves. 4) My development costs increasingly being determined by the AI labs and hardware vendors they partner with. Particularly when the former will need to increase prices dramatically over the coming years to break even with even 2025 economics.
  - simonw 9mo ago
    The value I'm getting from this stuff is so large that I'll take those risks, personally.
    - th0ma5 9mo ago
      [flagged]
      - scubbo 9mo ago
        Many people - simonw is the most visible of them, but there are countless others - have given up trying to convinced folks who are determined to not be convinced, and are simply enjoying their increased productivity. This is not a competition or an argument.
        llmslave2 9mo ago
        Maybe they are struggling to convince others because they are unable to produce evidence that is able to convince people? My experience scrolling X and HN is a bunch of people going "omg opus omg Claude Code I'm 10x more productive" and that's it. Just hand wavy anecdotes based on their own perceived productivity. I'm open to being convinced but just saying stuff is not convincing. It's the opposite, it feels like people have been put under a spell. I'm following The Primeagen, he's doing a series where he is trying these tools on stream and following peoples advice on how to use them the best. He's actually quite a good programmer so I'm eager to see how it goes. So far he isn't impressed and thus neither am I. If he cracks it and unlocks significant productivity then I will be convinced.
- Capricorn2481 9mo ago
  > With the latest models if you're clear enough with your requirements you'll usually find it does the right thing on the first try That's great that this is your experience, but it's not a lot of people's. There are projects where it's just not going to know what to do. I'm working in a web framework that is a Frankenstein-ing of Laravel and October CMS. It's so easy for the agent to get confused because, even when I tell it this is a different framework, it sees things that look like Laravel or October CMS and suggests solutions that are only for those frameworks. So there's constant made up methods and getting stuck in loops. The documentation is terrible, you just have to read the code. Which, despite what people say, Cursor is terrible at, because embeddings are not a real way to read a codebase.
  - simonw 9mo ago
    I'm working mostly in a web framework that's used by me and almost nobody else (the weird little ASGI wrapper buried in Datasette) and I find the coding agents pick it up pretty fast. One trick I use that might work for you as well: Clone GitHub.com/simonw/datasette to /tmp then look at /tmp/docs/datasette for documentation and search the code if you need to Try that with your own custom framework and it might unblock things. If your framework is missing documentation tell Claude Code to write itself some documentation based on what it learns from reading the code!
    - Capricorn2481 9mo ago
      > I'm working mostly in a web framework that's used by me and almost nobody else (the weird little ASGI wrapper buried in Datasette) and I find the coding agents pick it up pretty fast Potentially because there is no baggage with similar frameworks. I'm sure it would have an easier time with this if it was not spun off from other frameworks. > If your framework is missing documentation tell Claude Code to write itself some documentation based on what it learns from reading the code! If Claude cannot read the code well enough to begin with, and needs supplemental documentation, I certainly don't want it generating the docs from the code. That's just compounding hallucinations on top of each other.
solumunus 9mo ago
Correct it then, and next time craft a more explicit plan.
- wubrr 9mo ago
  The more explicit/detailed your plan, the more context it uses up, the less accurate and generally functional it is. Don't get me wrong, it's amazing, but on a complex problem with large enough context it will consistently shit the bed.
  - solumunus 9mo ago
    It usually works well for me. With very big tasks I break the plan into multiple MD files with the relevant context included and work through in individual sessions, updating remaining plans appropriately at the end of each one (usually there will be decision changes or additions during iteration).
  - rectang 9mo ago
    The human still has to manage complexity. A properly modularized and maintainable code base is much easier for the LLM to operate on — but the LLM has difficulty keeping the code base in that state without strong guidance. Putting “Make minimal changes” in my standard prompt helped a lot with the tendency of basically all agents to make too many changes at once. With that addition it became possible to direct the LLM to make something similar to the logical progression of commits I would have made anyway, but now don’t have to work as hard at crafting. Most of the hype merchants avoid the topic of maintainability because they’re playing to non-technical management skeptical of the importance of engineering fundamentals. But everything I’ve experienced so far working with LLMs screams that the fundamentals are more important than ever.
  - pigpop 9mo ago
    It takes a lot of plan to use up the context and most of the time the agent doesn't need the whole plan, they just need what's relevant to the current task.
zmmmmm 9mo ago
> My issues all stem from that it works, but does the wrong thing It's an opportunity, not a problem. Because it means there's a gap in your specifications and then your tests. I use Aider not Claude but I run it with Anthropic models. And what I found is that comprehensively writing up the documentation for a feature spec style before starting eliminates a huge amount of what you're referring to. It serves a triple purpose (a) you get the documentation, (b) you guide the AI and (c) it's surprising how often this helps to refine the feature itself. Sometimes I invoke the AI to help me write the spec as well, asking it to prompt for areas where clarification is needed etc.
- giancarlostoro 9mo ago
  This is how Beads works, especially with Claude Code. What I do is I tell Claude to always create a Bead when I tell it to add something, or about something that needs to be added, then I start brainstorming, and even ask it to do market research what are top apps doing for x, y or z. Then ask it to update the bead (I call them tasks) and then finally when its got enough detail, I tell it, do all of these in parallel.
  - beoberha 9mo ago
    Beads is amazing. It’s such a simple concept but elevates agentic coding to another levels

jmathai 9mo ago

I think it's worth understanding why. Because that's not everyone's experience and there's a chance you could make a change such that you find it extremely useful. There's a lesser chance that you're working on a code base that Claude Code just isn't capable of helping with.

simonw 9mo ago

If it does the wrong thing you tell it what the right thing is and have it try again. With the latest models if you're clear enough with your requirements you'll usually find it does the right thing on the first try.

cadamsdotcom 9mo ago
Even better, have it write code to describe the right thing then run its code against that, taking yourself out of that loop.
giancarlostoro 9mo ago
And if you've told it too many times to fix it, tell it someone has a gun to your head, for some reason it almost always gets it right this very next time.
- dare944 9mo ago
  If you're a developer at the dawn of the AI revolution, there is absolutely a gun to your head.
  - giancarlostoro 9mo ago
    Yeah, if anyone can truly afford the AI empire. Remember all these "leading" companies are running it at a loss, so most companies paying for it are severely underpaying the cost of it all. We would need an insane technological breakthrough of unlimited memory and power before I start to worry, and at that point, I'll just look for a new career.
aurumque 9mo ago
In a circuitous way, you can rather successfully have one agent write a specification and another one execute the code changes. Claude code has a planning mode that lets you work with the model to create a robust specification that can then be executed, asking the sort of leading questions for which it already seems to know it could make an incorrect assumption. I say 'agent' but I'm really just talking about separate model contexts, nothing fancy.
- mikestorrent 9mo ago
  Cursor's planning functionality is very similar and I have found that I can even use "cheap" models like their Composer-1 and get great results in the planning phase, and then turn on Sonnet or Opus to actually produce the plan. 90% of the stuff I need to argue about is during the planning phase, so I save a ton of tokens and rework just making a really good spec. It turns out that Waterfall was always the correct method, it's just really slow ;)
  - aurumque 9mo ago
    Did you know that software specifications used to be almost entirely flow charts? There is something to be said for that and waterfall.
GoatInGrey 9mo ago
There are several rubs with that operating protocol extending beyond the "you're holding it wrong" claim. 1) There exists a threshold, only identifiable in retrospect, past which it would have been faster to locate or write the code yourself than to navigate the LLM's correction loop or otherwise ensure one-shot success. 2) The intuition and motivations of LLMs derive from a latent space that the LLM cannot actually access. I cannot get a reliable answer on why the LLM chose the approaches it did; it can only retroactively confabulate. Unlike human developers who can recall off-hand, or at least review associated tickets and meeting notes to jog their memory. The LLM prompter always documenting sufficiently to bridge this LLM provenance gap hits rub #1. 3) Gradually building prompt dependency where one's ability to take over from the LLM declines and one can no longer answer questions or develop at the same velocity themselves. 4) My development costs increasingly being determined by the AI labs and hardware vendors they partner with. Particularly when the former will need to increase prices dramatically over the coming years to break even with even 2025 economics.
- simonw 9mo ago
  The value I'm getting from this stuff is so large that I'll take those risks, personally.
  - th0ma5 9mo ago
    [flagged]
    - scubbo 9mo ago
      Many people - simonw is the most visible of them, but there are countless others - have given up trying to convinced folks who are determined to not be convinced, and are simply enjoying their increased productivity. This is not a competition or an argument.
      - llmslave2 9mo ago
        Maybe they are struggling to convince others because they are unable to produce evidence that is able to convince people? My experience scrolling X and HN is a bunch of people going "omg opus omg Claude Code I'm 10x more productive" and that's it. Just hand wavy anecdotes based on their own perceived productivity. I'm open to being convinced but just saying stuff is not convincing. It's the opposite, it feels like people have been put under a spell. I'm following The Primeagen, he's doing a series where he is trying these tools on stream and following peoples advice on how to use them the best. He's actually quite a good programmer so I'm eager to see how it goes. So far he isn't impressed and thus neither am I. If he cracks it and unlocks significant productivity then I will be convinced.
Capricorn2481 9mo ago
> With the latest models if you're clear enough with your requirements you'll usually find it does the right thing on the first try That's great that this is your experience, but it's not a lot of people's. There are projects where it's just not going to know what to do. I'm working in a web framework that is a Frankenstein-ing of Laravel and October CMS. It's so easy for the agent to get confused because, even when I tell it this is a different framework, it sees things that look like Laravel or October CMS and suggests solutions that are only for those frameworks. So there's constant made up methods and getting stuck in loops. The documentation is terrible, you just have to read the code. Which, despite what people say, Cursor is terrible at, because embeddings are not a real way to read a codebase.
- simonw 9mo ago
  I'm working mostly in a web framework that's used by me and almost nobody else (the weird little ASGI wrapper buried in Datasette) and I find the coding agents pick it up pretty fast. One trick I use that might work for you as well: Clone GitHub.com/simonw/datasette to /tmp then look at /tmp/docs/datasette for documentation and search the code if you need to Try that with your own custom framework and it might unblock things. If your framework is missing documentation tell Claude Code to write itself some documentation based on what it learns from reading the code!
  - Capricorn2481 9mo ago
    > I'm working mostly in a web framework that's used by me and almost nobody else (the weird little ASGI wrapper buried in Datasette) and I find the coding agents pick it up pretty fast Potentially because there is no baggage with similar frameworks. I'm sure it would have an easier time with this if it was not spun off from other frameworks. > If your framework is missing documentation tell Claude Code to write itself some documentation based on what it learns from reading the code! If Claude cannot read the code well enough to begin with, and needs supplemental documentation, I certainly don't want it generating the docs from the code. That's just compounding hallucinations on top of each other.

solumunus 9mo ago

Correct it then, and next time craft a more explicit plan.

wubrr 9mo ago
The more explicit/detailed your plan, the more context it uses up, the less accurate and generally functional it is. Don't get me wrong, it's amazing, but on a complex problem with large enough context it will consistently shit the bed.
- solumunus 9mo ago
  It usually works well for me. With very big tasks I break the plan into multiple MD files with the relevant context included and work through in individual sessions, updating remaining plans appropriately at the end of each one (usually there will be decision changes or additions during iteration).
- rectang 9mo ago
  The human still has to manage complexity. A properly modularized and maintainable code base is much easier for the LLM to operate on — but the LLM has difficulty keeping the code base in that state without strong guidance. Putting “Make minimal changes” in my standard prompt helped a lot with the tendency of basically all agents to make too many changes at once. With that addition it became possible to direct the LLM to make something similar to the logical progression of commits I would have made anyway, but now don’t have to work as hard at crafting. Most of the hype merchants avoid the topic of maintainability because they’re playing to non-technical management skeptical of the importance of engineering fundamentals. But everything I’ve experienced so far working with LLMs screams that the fundamentals are more important than ever.
- pigpop 9mo ago
  It takes a lot of plan to use up the context and most of the time the agent doesn't need the whole plan, they just need what's relevant to the current task.

zmmmmm 9mo ago

> My issues all stem from that it works, but does the wrong thing It's an opportunity, not a problem. Because it means there's a gap in your specifications and then your tests. I use Aider not Claude but I run it with Anthropic models. And what I found is that comprehensively writing up the documentation for a feature spec style before starting eliminates a huge amount of what you're referring to. It serves a triple purpose (a) you get the documentation, (b) you guide the AI and (c) it's surprising how often this helps to refine the feature itself. Sometimes I invoke the AI to help me write the spec as well, asking it to prompt for areas where clarification is needed etc.

giancarlostoro 9mo ago
This is how Beads works, especially with Claude Code. What I do is I tell Claude to always create a Bead when I tell it to add something, or about something that needs to be added, then I start brainstorming, and even ask it to do market research what are top apps doing for x, y or z. Then ask it to update the bead (I call them tasks) and then finally when its got enough detail, I tell it, do all of these in parallel.
- beoberha 9mo ago
  Beads is amazing. It’s such a simple concept but elevates agentic coding to another levels