4 ms·

When you let Claude run free over changes big enough to have this thing be meaningful, are you really getting good enough code? When I just set Claude loose fo

When you let Claude run free over changes big enough to have this thing be meaningful, are you really getting good enough code? When I just set Claude loose for long periods, I get incomprehensible, brittle code. I don't do maintenance so maybe that's the difference but I have not had good results from big, unsupervised changes.

ishsup 1y ago
yeah that’s a fair experience, we’ve seen similar when leaving Claude unsupervised for too long. The way we use Omnara, it’s more about staying in the loop for those moments when Claude needs clarification or a quick decision, so you can keep it on track without babysitting the terminal the whole time
- deleted 1y ago
  [deleted]
adastra22 1y ago
Subagents has changed this for me. I routinely run half hour to hour long tasks with no human intervention, and actually get good results at the end (most of the time, not all of the time). The reason isn’t that AI models have gotten better, although they clearly have, but that using subagents (1) keeps context clear of false starts and errors that otherwise poison the AI’s view of the project, and (2) by throwing in directives to run subagents that keep the main agent aligned (e.g. code review agents), it gets nudged back on course a surprisingly high percentage of the time.
- MrGreenTea 1y ago
  Would you elaborate a bit on how you use subagents? I tend to use them sporadically, for example for it to research something or to analyse the code base a bit. But I'm not yet letting it run for long.
  - adastra22 1y ago
    Sure. First of all, although I do spend a lot of time interacting with Claude Code in chat format, that is not what I am talking about here. I have setup Claude Code with very specific instructions for use of agents, which I'll get to in a second. First of all, there's a lot of collections of subagent definitions out there. I rolled my own, then later found others that worked better. I'm currently using this curated collection: https://github.com/VoltAgent/awesome-claude-code-subagents https://github.com/VoltAgent/awesome-claude-code-subagents CLAUDE.md has instructions to list `.agents/agents/**/*.md` to find the available agents, and knows to check the frontmatter yaml for a one-line description of what each does. These agents are really just (1) role definitions that prompts the LLM to bias its thinking in a particular way ("You are a senior Rust engineer with deep expertise in ..." -- this actually works really well), and (2) a bunch of rules and guidelines for that role, e.g. in the Rust case to use thiserror and strum crates to avoid boilerplate in Error enums, rules for how to satisfy the linter, etc. Basic project guidelines as they relate to Rust dev. Secondly, my CLAUDE.md for the project has very specific instructions about how the top-level agent should operate, with callouts to specific procedure files to follow. These live in `.agent/action/**/*.md`. For example, I have a git-commit.md protocol definition file, and instructions in CLAUDE.md that "when the user prompts with 'commit' or 'git commit', load git-commit action and follow the directions contained within precisely." Within git-commit.md, there is a clear workflow specification in text or pseudocode. The [text] is my in-line comments to you and not in the original file: """ You are tasked with committing the currently staged changes to the currently active branch of this git repository. You are not authorized to make any changes beyond what has already been staged for commit. You are to follow these procedures exactly. 1. Check that the output of `git diff --staged` is not empty. If it is empty, report to the user that there are no currently staged changes and await further instructions from the user. 2. Stash any unstaged changes, so that the worktree only contains the changes that are to be committed. 3. Run `./check.sh` [a bash script that runs the full CI test suite locally] and verify that no warnings or errors are generated with just the currently staged changes applied. - If the check script doesn't pass, summarize the errors and ask the user if they wish to launch the rust-engineer agent to fix these issues. Then follow the directions given by the user. 4. Run `git diff --staged | cat` and summarize the changes in a git commit message written in the style of the Linux kernel mailing list [I find this to be much better than Claude's default commit message summaries]. 5. Display the output of `git diff --staged --stat` and your suggested git commit message to the user and await feedback. For each response by the user, address any concerns brought up and then generate a new commit message, as needed or instructed, and explicitly ask again for further feedback or confirmation to continue. 6. Only when the user has explicitly given permission to proceed with the commit, without any accompanying actionable feedback, should you proceed to making the commit. Execute 'git commit` with the exact text for the commit message that the user approved. 7. Unstash the non-staged changes that were previously stashed in step 2. 8. Report completion to the user. You are not authorized to deviate from these instructions in any way. """ This one doesn't employ subagents very much, and it is implicitly interactive, but it is smaller and easier to explain. It is, essentially, a call center script for the main agent to follow. In my experience, it does a very good job of following these instructions. This particular one addresses a pet peeve of mine: I hate the auto-commit anti-feature of basically all coding assistants. I'm old-school and want a nice, cleanly curated git history with comprehensible commits that take some refining to get right. It's not just OCD -- my workflow involves being able to git bisect effectively to find bugs, which requires a good git history. ...continued in part 2

ishsup 1y ago

yeah that’s a fair experience, we’ve seen similar when leaving Claude unsupervised for too long. The way we use Omnara, it’s more about staying in the loop for those moments when Claude needs clarification or a quick decision, so you can keep it on track without babysitting the terminal the whole time

deleted 1y ago
[deleted]

adastra22 1y ago

Subagents has changed this for me. I routinely run half hour to hour long tasks with no human intervention, and actually get good results at the end (most of the time, not all of the time). The reason isn’t that AI models have gotten better, although they clearly have, but that using subagents (1) keeps context clear of false starts and errors that otherwise poison the AI’s view of the project, and (2) by throwing in directives to run subagents that keep the main agent aligned (e.g. code review agents), it gets nudged back on course a surprisingly high percentage of the time.

MrGreenTea 1y ago
Would you elaborate a bit on how you use subagents? I tend to use them sporadically, for example for it to research something or to analyse the code base a bit. But I'm not yet letting it run for long.
- adastra22 1y ago
  Sure. First of all, although I do spend a lot of time interacting with Claude Code in chat format, that is not what I am talking about here. I have setup Claude Code with very specific instructions for use of agents, which I'll get to in a second. First of all, there's a lot of collections of subagent definitions out there. I rolled my own, then later found others that worked better. I'm currently using this curated collection: https://github.com/VoltAgent/awesome-claude-code-subagents https://github.com/VoltAgent/awesome-claude-code-subagents CLAUDE.md has instructions to list `.agents/agents/**/*.md` to find the available agents, and knows to check the frontmatter yaml for a one-line description of what each does. These agents are really just (1) role definitions that prompts the LLM to bias its thinking in a particular way ("You are a senior Rust engineer with deep expertise in ..." -- this actually works really well), and (2) a bunch of rules and guidelines for that role, e.g. in the Rust case to use thiserror and strum crates to avoid boilerplate in Error enums, rules for how to satisfy the linter, etc. Basic project guidelines as they relate to Rust dev. Secondly, my CLAUDE.md for the project has very specific instructions about how the top-level agent should operate, with callouts to specific procedure files to follow. These live in `.agent/action/**/*.md`. For example, I have a git-commit.md protocol definition file, and instructions in CLAUDE.md that "when the user prompts with 'commit' or 'git commit', load git-commit action and follow the directions contained within precisely." Within git-commit.md, there is a clear workflow specification in text or pseudocode. The [text] is my in-line comments to you and not in the original file: """ You are tasked with committing the currently staged changes to the currently active branch of this git repository. You are not authorized to make any changes beyond what has already been staged for commit. You are to follow these procedures exactly. 1. Check that the output of `git diff --staged` is not empty. If it is empty, report to the user that there are no currently staged changes and await further instructions from the user. 2. Stash any unstaged changes, so that the worktree only contains the changes that are to be committed. 3. Run `./check.sh` [a bash script that runs the full CI test suite locally] and verify that no warnings or errors are generated with just the currently staged changes applied. - If the check script doesn't pass, summarize the errors and ask the user if they wish to launch the rust-engineer agent to fix these issues. Then follow the directions given by the user. 4. Run `git diff --staged | cat` and summarize the changes in a git commit message written in the style of the Linux kernel mailing list [I find this to be much better than Claude's default commit message summaries]. 5. Display the output of `git diff --staged --stat` and your suggested git commit message to the user and await feedback. For each response by the user, address any concerns brought up and then generate a new commit message, as needed or instructed, and explicitly ask again for further feedback or confirmation to continue. 6. Only when the user has explicitly given permission to proceed with the commit, without any accompanying actionable feedback, should you proceed to making the commit. Execute 'git commit` with the exact text for the commit message that the user approved. 7. Unstash the non-staged changes that were previously stashed in step 2. 8. Report completion to the user. You are not authorized to deviate from these instructions in any way. """ This one doesn't employ subagents very much, and it is implicitly interactive, but it is smaller and easier to explain. It is, essentially, a call center script for the main agent to follow. In my experience, it does a very good job of following these instructions. This particular one addresses a pet peeve of mine: I hate the auto-commit anti-feature of basically all coding assistants. I'm old-school and want a nice, cleanly curated git history with comprehensible commits that take some refining to get right. It's not just OCD -- my workflow involves being able to git bisect effectively to find bugs, which requires a good git history. ...continued in part 2