15 ms·
> But even better, tell it to create subagents to form red team, green team and refactor team while the main instance coordinates them, respecting the clean-roo
by magicalist 7mo ago
> But even better, tell it to create subagents to form red team, green team and refactor team while the main instance coordinates them, respecting the clean-room rules. It really works.
It helps, but it definitely doesn't always work, particularly as refactors go on and tests have to change. Useless tests start grow in count and important new things aren't tested or aren't tested well.
I've had both Opus 4.6 and Codex 5.3 recently tell me the other (or another instance) did a great job with test coverage and depth, only to find tests within that just asserted the test harness had been set up correctly and the functionality that had been in those tests get tested that it exists but its behavior now virtually untested.
Reward hacking is very real and hard to guard against.
- lagrange77 7mo ago> Reward hacking is very real and hard to guard against. Is it really about rewards? Im genuinely curious. Because its not a RL model.
- nurettin 7mo agoThey probably meant goal hacking. (I just made that up)
- taneq 7mo agoI refer to it as ‘wanking’. It’s doing something that’s unproductive but that’s incentivised by its architecture.
- lagrange77 7mo agoI'll use that term from now on. :D
- magicalist 7mo ago> Is it really about rewards? Im genuinely curious. Because its not a RL model. Ha, good point. I was using it informally (you could handwave and call it an intrinsic reward if a model is well aligned to completing tasks as requested), but I hadn't really thought about it. Searching around, it seems like I'm not alone, but it looks like "specification gaming" is also sometimes used, like: https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/ https://deepmind.google/blog/specification-gaming-the-flip-s...
- gbnwl 7mo agoI'm noticing terms related to DL/RL/NLP are being used more and more informally as AI takes over more of the cultural zeitgeist and people want to use the fancy new terms of the era, even if inaccurately. A friend told me he "trained and fine tuned a custom agent" for his work when what he meant was he modified a claude.md file.
- collingreen 7mo agoRespectfully, your friend doesn't know what he is talking about and is saying things that just "feel right" (vibe talking??). Which might be exactly how technical terms lose their meaning so perhaps you're exactly right.
- hexaga 7mo agoThere is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. And with that comes reward hacking - which isn't really about looking for more reward but rather that the model has learned patterns of behavior that got reward in the train env. That is, any kind of vulnerability in the train env manifests as something you'd recognize as reward hacking in the real world: making tests pass _no matter what_ (because the train env rewarded that behavior), being wildly sycophantic (because the human evaluators rewarded that behavior), etc.
- lagrange77 7mo ago> There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model. Hm, as i understand it, parts of the training of e.g. ChatGPT could be called RL models. But the subject to be trained/fine tuned is still a seq2seq next token predictor transformer neural net.
- hexaga 7mo agoRL is simply a broad category of training methods. It's not really an architecture per se: modern GPTs are trained first on reconstruction objective on massive text corpora (the 'large language' part), then on various RL objectives +/- more post-training depending on which lab.
- egeozcan 7mo agoThe trick is, with the setup I mentioned, you change the rewards. The concept is: Red Team (Test Writers), write tests without seeing implementation. They define what the code should do based on specs/requirements only. Rewarded by test failures. A new test that passes immediately is suspicious as it means either the implementation already covers it (diminishing returns) or the test is tautological. Red's ideal outcome is a well-named test that fails, because that represents a gap between spec and implementation that didn't previously have a tripwire. Their proxy metric is "number of meaningful new failures introduced" and the barrier prevents them from writing tests pre-adapted to pass. Green Team (Implementers), write implementation to pass tests without seeing the test code directly. They only see test results (pass/fail) and the spec. Rewarded by turning red tests green. Straightforward, but the barrier makes the reward structure honest. Without it, Green could satisfy the reward trivially by reading assertions and hard-coding. With it, Green has to actually close the gap between spec intent and code behavior, using error messages as noisy gradient signal rather than exact targets. Their reward is "tests that were failing now pass," and the only reliable strategy to get there is faithful implementation. Refactor Team, improve code quality without changing behavior. They can see implementation but are constrained by tests passing. Rewarded by nothing changing (pretty unusual in this regard). Reward is that all tests stay green while code quality metrics improve. They're optimizing a secondary objective (readability, simplicity, modularity, etc.) under a hard constraint (behavioral equivalence). The spec barrier ensures they can't redefine "improvement" to include feature work. If you have any code quality tools, it makes sense to give the necessary skills to use them to this team. It's worth being honest about the limits. The spec itself is a shared artifact visible to both Red and Green, so if the spec is vague, both agents might converge on the same wrong interpretation, and the tests will pass for the wrong reason. The Coordinator (your main claude/codex/whatever instance) mitigates this by watching for suspiciously easy green passes (just tell it) and probing the spec for ambiguity, but it's not a complete defense.
- skybrian 7mo agoHow do you define visibility rules? Is that possible for subagents?
- 7mo ago
- SoftTalker 7mo agoA refactor should not affect the tests at all should it? If it does, it's more than a refactor.
- magicalist 7mo agoIt depends on what you mean by "refactor" and how exactly you're testing, I guess, but that's not really at the heart of the point. red-green-refactor could also be used for adding new features, for instance, or an entire codebase, I guess.
- gchamonlive 7mo agoIt can if your refactor needs to deal with interface changes, like moving methods around, changing argument order etc... all these need to propagate to the tests
- bluGill 7mo agoYour tests are an assertion that 'no matter what this will never change'. If your interface can change then you are testing implementation details instead of the behavior users care about. the above is really hard. A lot of tdd 'experts' don't understand is and teach fragile tests that are not worth having.
- switchbak 7mo agoRefactoring is changing the design of the code without affecting the behaviour. You can change an interface and not change the behaviour. I have rarely heard such a rigid interpretation such as this.
- 8note 7mo agohttps://www.hyrumslaw.com/ https://www.hyrumslaw.com/ your implementation is your interface. its a bit naive or hating-your-users to assume your tests are what your users care about. theyre dealing with everything, regardless of what youve tested or not.
- esafak 7mo agoPeriodically reviewing tests is worthwhile but rarely done; writing tests alone is already disliked.
- eru 7mo ago> Useless tests start grow in count and important new things aren't tested or aren't tested well. You can use coverage information, and you should cull your tests every once in a while I guess. Property based testing also helps.
- skeledrew 7mo agoDid you look at the actual coverage yourself though? That's something I always have active, and if I see it low in an area, I'll be asking questions.