3 ms·

I have a Just task that runs linters (ruff and pyright, in my case), formatter, tests and pre-commit hooks, and have Claude run it every time it thinks it's don

I have a Just task that runs linters (ruff and pyright, in my case), formatter, tests and pre-commit hooks, and have Claude run it every time it thinks it's done with a change. It's good enough that when the checks pass, it's usually complete.

simlevesque 1y ago
A tip for everyone doing this: pipe the linters' stdout to /dev/null to save on tokens.
- maleldil 1y ago
  Why? The agent needs the error messages from the linters to know what to do.
  - jamie_ca 1y ago
    If you're running linters for formatting etc, just get the agent to run them on autocorrect and it doesn't need to know the status as urgently.
    - maleldil 1y ago
      That's just one part of it. I want the LLM to see type checking errors, failing test outputs, etc.
      - ozim 1y ago
        Errors shouldn’t be on stdout ;)
        dymk 1y ago
        “Errors” printed by your linter aren’t errors, they’re reports
        ozim 1y ago
        So if I want my CICD break on linter errors I have to implement parsing of stdout? I'd rather have it breakout of the box by going to stderr.
joshstrange 1y ago
This is the best way to approach it but if I had a dollar for each time Claude ran “—no-verify” on the git commits it was doing I’d have 10’s of dollars. Doesn’t matter if you tell it multiple times in CLAUDE.md to not skip checks, it will eventually just skip them so it can commit. It’s infuriating. I hope that as CC evolves there is a better way to tell/force the model to do things like that (linters, formatters, unit/e2e tests, etc).
- 35mm 1y ago
  I’ve found the same issue and also with Rust sometimes skips tests if it thinks they’re taking too long to compile, and says it’s unnecessary because it knows they’ll pass.
  - pixl97 1y ago
    Even AI understands it's Friday. Just push to to production and go home for the weekend.
- mediaman 1y ago
  We should have a finish hook that, when the AI decides it's run, runs the hook, and gives it to the LLM, and it can decide whether the problem is still there. Students don't get to choose whether to take the test, so why do we give AI the choice?
- evertedsphere 1y ago
  a wrapper script?
theshrike79 1y ago
(I code mostly in Go) I have a `task build` command that runs linters, tests and builds the project. All the commands have verbosity tuned down to minimum to not waste context on useless crap. Claude remembers to do it pretty well. I have it in my global CLAUDE.md sot I guess it has more weight? Dunno.

simlevesque 1y ago

A tip for everyone doing this: pipe the linters' stdout to /dev/null to save on tokens.

maleldil 1y ago
Why? The agent needs the error messages from the linters to know what to do.
- jamie_ca 1y ago
  If you're running linters for formatting etc, just get the agent to run them on autocorrect and it doesn't need to know the status as urgently.
  - maleldil 1y ago
    That's just one part of it. I want the LLM to see type checking errors, failing test outputs, etc.
    - ozim 1y ago
      Errors shouldn’t be on stdout ;)
      - dymk 1y ago
        “Errors” printed by your linter aren’t errors, they’re reports
        ozim 1y ago
        So if I want my CICD break on linter errors I have to implement parsing of stdout? I'd rather have it breakout of the box by going to stderr.

joshstrange 1y ago

This is the best way to approach it but if I had a dollar for each time Claude ran “—no-verify” on the git commits it was doing I’d have 10’s of dollars. Doesn’t matter if you tell it multiple times in CLAUDE.md to not skip checks, it will eventually just skip them so it can commit. It’s infuriating. I hope that as CC evolves there is a better way to tell/force the model to do things like that (linters, formatters, unit/e2e tests, etc).

35mm 1y ago
I’ve found the same issue and also with Rust sometimes skips tests if it thinks they’re taking too long to compile, and says it’s unnecessary because it knows they’ll pass.
- pixl97 1y ago
  Even AI understands it's Friday. Just push to to production and go home for the weekend.
mediaman 1y ago
We should have a finish hook that, when the AI decides it's run, runs the hook, and gives it to the LLM, and it can decide whether the problem is still there. Students don't get to choose whether to take the test, so why do we give AI the choice?
evertedsphere 1y ago
a wrapper script?

theshrike79 1y ago

(I code mostly in Go) I have a `task build` command that runs linters, tests and builds the project. All the commands have verbosity tuned down to minimum to not waste context on useless crap. Claude remembers to do it pretty well. I have it in my global CLAUDE.md sot I guess it has more weight? Dunno.