4 ms·

Another really nice use case building very sophisticated test tooling. Normally a company might not allocate enough resources to a task like that but with Claud

by qaq 1y ago

Another really nice use case building very sophisticated test tooling. Normally a company might not allocate enough resources to a task like that but with Claude Code it's a no brainer. Also can create very sophisticated mocks like say db mock that can parse all queries in the codebase and apply them to in memory fake tables. Would be total pain to build and maintain by hand but with claude code takes literally minutes.

airstrike 1y ago
In my experience LLMs are notoriously bad at tests, so this is, to me, one of the worst use cases possible.
- qaq 1y ago
  In my experience they are great for test tooling. For actual tests after I have covered a number of cases it's very workable to tell it to identify gaps and edge cases and propose tests than I'd say I accept about 70% of it suggestions.
- lopatin 1y ago
  While people's experience with LLMs is pretty varied and subjective, saying they're bad at writing tests just isn't true. Claude Code is incredible at writing tests and testing infrastructure.
  - danielbln 1y ago
    It worth mentioning that one should tell CC to not overmock, and to produce only truly relevant tests. I use an agent that I invoke to spot this stuff, because I've run into some truly awful overmocked non-tests before.
- cpursley 1y ago
  In JS, yes. It stacks with good testing stories, it’s quite good.
skydhash 1y ago
Hill I’m willing to die on (metaphorically): If your test structure is a pain to interact with, that usually means some bad decisions somewhere in the architecture of your project.
- qaq 1y ago
  Sure but thats pretty orthogonal to the above. Say had to create parser+mapper for a domain specific query language that than gets mapped to a number of SQL backends. Used cc to create custom test harness that can use SQLC style files to drive tests. They are way more readable and much easier to maintain vs plain Rust tests for this task. Took half a day to create with cc would prob take like a week without it.

by qaq 1y ago

airstrike 1y ago

In my experience LLMs are notoriously bad at tests, so this is, to me, one of the worst use cases possible.

qaq 1y ago
In my experience they are great for test tooling. For actual tests after I have covered a number of cases it's very workable to tell it to identify gaps and edge cases and propose tests than I'd say I accept about 70% of it suggestions.
lopatin 1y ago
While people's experience with LLMs is pretty varied and subjective, saying they're bad at writing tests just isn't true. Claude Code is incredible at writing tests and testing infrastructure.
- danielbln 1y ago
  It worth mentioning that one should tell CC to not overmock, and to produce only truly relevant tests. I use an agent that I invoke to spot this stuff, because I've run into some truly awful overmocked non-tests before.
cpursley 1y ago
In JS, yes. It stacks with good testing stories, it’s quite good.

skydhash 1y ago

Hill I’m willing to die on (metaphorically): If your test structure is a pain to interact with, that usually means some bad decisions somewhere in the architecture of your project.

qaq 1y ago
Sure but thats pretty orthogonal to the above. Say had to create parser+mapper for a domain specific query language that than gets mapped to a number of SQL backends. Used cc to create custom test harness that can use SQLC style files to drive tests. They are way more readable and much easier to maintain vs plain Rust tests for this task. Took half a day to create with cc would prob take like a week without it.