4 ms·
Use evals Coming soon, unit, behavioural and regression tests for your prompts and skills :P
by Leynos 7mo ago
Use evals
Coming soon, unit, behavioural and regression tests for your prompts and skills :P
- stingraycharles 7mo agoHow do you use evals when you’re using Claude Code, given that Claude Code also changes their prompts all the time? You’ll have: * Claude model version * Claude Code prompts and tools * Your own prompts and skills and whatnot * Your repository’s source code (= the input) All of those change constantly, it’s not like it’s some kind of SWE benchmark.
- Leynos 6mo agoYou just said it. If consistency is that important, keep consistent versions of model, harness, prompts, skills, etc., and regression test changes. That way lies madness :)