4 ms·

You can always tell claude to use red-green-refactor and that really is a step-up from "yeah don't forget to write tests and make sure they pass" at the end of

You can always tell claude to use red-green-refactor and that really is a step-up from "yeah don't forget to write tests and make sure they pass" at the end of the prompt, sure. But even better, tell it to create subagents to form red team, green team and refactor team while the main instance coordinates them, respecting the clean-room rules. It really works. The trick is just not mixing/sharing the context. Different instances of the same model do not recognize each other to be more compliant.

aray07 7mo ago
thats a great idea - i have been using codex to do my code reviews since i have it to give better critique on code written by claude but havent tried it with testing yet!
- darkbatman 7mo ago
  codex/gpt is a stubborn model, doubt it would accept claude reviews or counter it. have seen cases where claude is more willing to comply if shared feedback though its just sycophancy too.
codybontecou 7mo ago
This sounds interesting. Can you go a bit deeper or provide references on how to implement the green/red/refactor subagent pattern?
- dmd 7mo ago
  That's the cool bit - you don't have to. CC is perfectly well aware and competent to implement it; just tell it to.
  - irishcoffee 7mo ago
    "So this is how liberty dies... with thunderous applause.” - Padmé Amidala s/liberty/knowledge
  - deleted 7mo ago
    [deleted]
- pastescreenshot 7mo ago
  What has worked better for me is splitting authority, not just prompts. One agent can touch app code, one can only write failing tests plus a short bug hypothesis, and one only reviews the diff and test output. Also make test files read only for the coding agent. That cuts out a surprising amount of self-grading behavior.
  - huslage 7mo ago
    How do you limit access like that?
- elemeno 7mo ago
  It’s not an agentic pattern, it’s an approach to test driven development. You write a failing test for the new functionality that you’re going to add (which doesn’t exist yet, so the test is red). You then write the code until the test passes (that is, goes green).
- dworks 7mo ago
  I built rlm-workflow which has stage gating, TDD and sub-agent support: https://skills.sh/doubleuuser/rlm-workflow/rlm-workflow https://skills.sh/doubleuuser/rlm-workflow/rlm-workflow
magicalist 7mo ago
> But even better, tell it to create subagents to form red team, green team and refactor team while the main instance coordinates them, respecting the clean-room rules. It really works. It helps, but it definitely doesn't always work, particularly as refactors go on and tests have to change. Useless tests start grow in count and important new things aren't tested or aren't tested well. I've had both Opus 4.6 and Codex 5.3 recently tell me the other (or another instance) did a great job with test coverage and depth, only to find tests within that just asserted the test harness had been set up correctly and the functionality that had been in those tests get tested that it exists but its behavior now virtually untested. Reward hacking is very real and hard to guard against.
- lagrange77 7mo ago
  > Reward hacking is very real and hard to guard against. Is it really about rewards? Im genuinely curious. Because its not a RL model.
  - nurettin 7mo ago
    They probably meant goal hacking. (I just made that up)
    - taneq 7mo ago
      I refer to it as ‘wanking’. It’s doing something that’s unproductive but that’s incentivised by its architecture.
      - lagrange77 7mo ago
        I'll use that term from now on. :D
  - magicalist 7mo ago
    > Is it really about rewards? Im genuinely curious. Because its not a RL model. Ha, good point. I was using it informally (you could handwave and call it an intrinsic reward if a model is well aligned to completing tasks as requested), but I hadn't really thought about it. Searching around, it seems like I'm not alone, but it looks like "specification gaming" is also sometimes used, like: https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/ https://deepmind.google/blog/specification-gaming-the-flip-s...
afro88 7mo ago
Good idea, and an improvement, but you still have that fundamental issue: you don't really know what code has been written. You don't know the refactors are right, in alignment with existing patterns etc.
Skidaddle 7mo ago
How exactly do you set up your CC sessions to do this?
SequoiaHope 7mo ago
I’m telling it to use red/green tdd [1] and it will write test that don’t fail and then says “ah the issue is already fixed” and then move on. You really have to watch it very closely. I’m having a huge problem with bad tests in my system despite a “governance model” that I always refer it to which requires red/green tdd. [1] https://simonwillison.net/guides/agentic-engineering-patterns/red-green-tdd/ https://simonwillison.net/guides/agentic-engineering-pattern...
osigurdson 7mo ago
So more stuff happens with this approach but how do you know what it generates is correct?
joegaebel 7mo ago
I've been able to encode Outside-in Test Driven Development into a repeatable workflow. Claude Code follows it to a T, and I've gotten great results. I've written about it more here, and created a repo people can use out of the box to try it out: https://www.joegaebel.com/articles/principled-agentic-software-development/ https://www.joegaebel.com/articles/principled-agentic-softwa... https://github.com/JoeGaebel/outside-in-tdd-starter https://github.com/JoeGaebel/outside-in-tdd-starter
- codethief 7mo ago
  Thanks for sharing! > When asking Claude Code to write tests, I find they are inevitably coupled to implementation details, mockist, brittle, and missing coverage. Interestingly, I haven't noticed any of that so far, using Claude Code on a new-ish project (couple 10k loc). However, I also went out of my way in my CLAUDE.md to instruct it to write functional code, avoid side effects / push side effects to the shell (functional core, imperative shell), avoid mocks in tests, etc. etc.
  - joegaebel 7mo ago
    That's the kicker - you taught it conventions that it follows. Even better (in my view) is to ensure you're getting higher coverage and tests focused on behaviour by getting it to write the tests first. Even moreso by ensuring it writes "feature complete" tests for each feature first. Even moreso by running mutation testing to backfill tests for logic it didn't cover.
kaizenb 7mo ago
Works for PR reviews. Separating context for code review with the same model has significant impact. ● Separation of concerns. No single agent plans, implements, and verifies. The agent that writes the code is never the agent that checks it.

aray07 7mo ago

thats a great idea - i have been using codex to do my code reviews since i have it to give better critique on code written by claude but havent tried it with testing yet!

darkbatman 7mo ago
codex/gpt is a stubborn model, doubt it would accept claude reviews or counter it. have seen cases where claude is more willing to comply if shared feedback though its just sycophancy too.

codybontecou 7mo ago

This sounds interesting. Can you go a bit deeper or provide references on how to implement the green/red/refactor subagent pattern?

dmd 7mo ago
That's the cool bit - you don't have to. CC is perfectly well aware and competent to implement it; just tell it to.
- irishcoffee 7mo ago
  "So this is how liberty dies... with thunderous applause.” - Padmé Amidala s/liberty/knowledge
- deleted 7mo ago
  [deleted]
pastescreenshot 7mo ago
What has worked better for me is splitting authority, not just prompts. One agent can touch app code, one can only write failing tests plus a short bug hypothesis, and one only reviews the diff and test output. Also make test files read only for the coding agent. That cuts out a surprising amount of self-grading behavior.
- huslage 7mo ago
  How do you limit access like that?
elemeno 7mo ago
It’s not an agentic pattern, it’s an approach to test driven development. You write a failing test for the new functionality that you’re going to add (which doesn’t exist yet, so the test is red). You then write the code until the test passes (that is, goes green).
dworks 7mo ago
I built rlm-workflow which has stage gating, TDD and sub-agent support: https://skills.sh/doubleuuser/rlm-workflow/rlm-workflow https://skills.sh/doubleuuser/rlm-workflow/rlm-workflow

magicalist 7mo ago

> But even better, tell it to create subagents to form red team, green team and refactor team while the main instance coordinates them, respecting the clean-room rules. It really works. It helps, but it definitely doesn't always work, particularly as refactors go on and tests have to change. Useless tests start grow in count and important new things aren't tested or aren't tested well. I've had both Opus 4.6 and Codex 5.3 recently tell me the other (or another instance) did a great job with test coverage and depth, only to find tests within that just asserted the test harness had been set up correctly and the functionality that had been in those tests get tested that it exists but its behavior now virtually untested. Reward hacking is very real and hard to guard against.

lagrange77 7mo ago
> Reward hacking is very real and hard to guard against. Is it really about rewards? Im genuinely curious. Because its not a RL model.
- nurettin 7mo ago
  They probably meant goal hacking. (I just made that up)
  - taneq 7mo ago
    I refer to it as ‘wanking’. It’s doing something that’s unproductive but that’s incentivised by its architecture.
    - lagrange77 7mo ago
      I'll use that term from now on. :D
- magicalist 7mo ago
  > Is it really about rewards? Im genuinely curious. Because its not a RL model. Ha, good point. I was using it informally (you could handwave and call it an intrinsic reward if a model is well aligned to completing tasks as requested), but I hadn't really thought about it. Searching around, it seems like I'm not alone, but it looks like "specification gaming" is also sometimes used, like: https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/ https://deepmind.google/blog/specification-gaming-the-flip-s...

afro88 7mo ago

Good idea, and an improvement, but you still have that fundamental issue: you don't really know what code has been written. You don't know the refactors are right, in alignment with existing patterns etc.

Skidaddle 7mo ago

How exactly do you set up your CC sessions to do this?

SequoiaHope 7mo ago

I’m telling it to use red/green tdd [1] and it will write test that don’t fail and then says “ah the issue is already fixed” and then move on. You really have to watch it very closely. I’m having a huge problem with bad tests in my system despite a “governance model” that I always refer it to which requires red/green tdd. [1] https://simonwillison.net/guides/agentic-engineering-patterns/red-green-tdd/ https://simonwillison.net/guides/agentic-engineering-pattern...

osigurdson 7mo ago

So more stuff happens with this approach but how do you know what it generates is correct?

joegaebel 7mo ago

I've been able to encode Outside-in Test Driven Development into a repeatable workflow. Claude Code follows it to a T, and I've gotten great results. I've written about it more here, and created a repo people can use out of the box to try it out: https://www.joegaebel.com/articles/principled-agentic-software-development/ https://www.joegaebel.com/articles/principled-agentic-softwa... https://github.com/JoeGaebel/outside-in-tdd-starter https://github.com/JoeGaebel/outside-in-tdd-starter

codethief 7mo ago
Thanks for sharing! > When asking Claude Code to write tests, I find they are inevitably coupled to implementation details, mockist, brittle, and missing coverage. Interestingly, I haven't noticed any of that so far, using Claude Code on a new-ish project (couple 10k loc). However, I also went out of my way in my CLAUDE.md to instruct it to write functional code, avoid side effects / push side effects to the shell (functional core, imperative shell), avoid mocks in tests, etc. etc.
- joegaebel 7mo ago
  That's the kicker - you taught it conventions that it follows. Even better (in my view) is to ensure you're getting higher coverage and tests focused on behaviour by getting it to write the tests first. Even moreso by ensuring it writes "feature complete" tests for each feature first. Even moreso by running mutation testing to backfill tests for logic it didn't cover.

kaizenb 7mo ago

Works for PR reviews. Separating context for code review with the same model has significant impact. ● Separation of concerns. No single agent plans, implements, and verifies. The agent that writes the code is never the agent that checks it.