3 ms·

If coding agents can't test the code as they're editing it they're no different from pasting your entire codebase into ChatGPT and crossing your fingers. Out o

If coding agents can't test the code as they're editing it they're no different from pasting your entire codebase into ChatGPT and crossing your fingers. Out of curiosity, how do you get Claude Code or Codex to actually do this? I asked this question here before: https://news.ycombinator.com/item?id=46792066 https://news.ycombinator.com/item?id=46792066

SJMG 9mo ago
Instruct it to test as it goes along. Add whatever testing base command to your list of trusted tools.
simonw 9mo ago
I don't use CLAUDE.md, I instead use simple token-efficient conventions. Most importantly all of my Python projects use a pyproject.toml file with this pattern: [dependency-groups] dev = ["pytest"] Which means I can tell the agent: Run "uv run pytest" And it will run the tests - without first needing to setup a virtual environment or install dependencies or anything like that. I wrote more about that pattern here: https://til.simonwillison.net/uv/dependency-groups https://til.simonwillison.net/uv/dependency-groups For more complex test suites I'll give it more detailed instructions. For testing web apps I used to tell it "use playwright" or "use playwright Python". I'm currently experimenting with my own simple CLI browser automation tool. This means I can tell it: Run "uvx rodney --help" and then use rodney to test this change The --help output tells it everything it needs to use the tool - here's that document in the repo: https://github.com/simonw/rodney/blob/10b2a6c81f9f3fb36ce4d1ffd113b46feb18e332/main.go#L98 https://github.com/simonw/rodney/blob/10b2a6c81f9f3fb36ce4d1... I've recently started having the bots "manually" test changes with a new tool I built called Showboat. It's less than a week old but it's so far been working really well: https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ https://simonwillison.net/2026/Feb/10/showboat-and-rodney/
- maleldil 9mo ago
  If you don't use CLAUDE.md, do you tell the agent to run pytest every single session?

SJMG 9mo ago

Instruct it to test as it goes along. Add whatever testing base command to your list of trusted tools.

simonw 9mo ago

I don't use CLAUDE.md, I instead use simple token-efficient conventions. Most importantly all of my Python projects use a pyproject.toml file with this pattern: [dependency-groups] dev = ["pytest"] Which means I can tell the agent: Run "uv run pytest" And it will run the tests - without first needing to setup a virtual environment or install dependencies or anything like that. I wrote more about that pattern here: https://til.simonwillison.net/uv/dependency-groups https://til.simonwillison.net/uv/dependency-groups For more complex test suites I'll give it more detailed instructions. For testing web apps I used to tell it "use playwright" or "use playwright Python". I'm currently experimenting with my own simple CLI browser automation tool. This means I can tell it: Run "uvx rodney --help" and then use rodney to test this change The --help output tells it everything it needs to use the tool - here's that document in the repo: https://github.com/simonw/rodney/blob/10b2a6c81f9f3fb36ce4d1ffd113b46feb18e332/main.go#L98 https://github.com/simonw/rodney/blob/10b2a6c81f9f3fb36ce4d1... I've recently started having the bots "manually" test changes with a new tool I built called Showboat. It's less than a week old but it's so far been working really well: https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ https://simonwillison.net/2026/Feb/10/showboat-and-rodney/

maleldil 9mo ago
If you don't use CLAUDE.md, do you tell the agent to run pytest every single session?