4 ms·
I tell the agent to NEVER write comments in the system prompt and it ignore it like 90% of the time. RLHF is a helluva drug.
by throwatdem12311 2mo ago
I tell the agent to NEVER write comments in the system prompt and it ignore it like 90% of the time. RLHF is a helluva drug.
- bombcar 2mo agoWe trained agents on millions of pages of documentation telling them to write good comments and good code and then we tell them never to write any comments. It’s almost the “we built a robot who loves to play Sonatas and gave it no hands” type of thing.
- benregenspan 2mo agoBeing overly trained on comments in documentation could be one of the reasons why Claude models write frustrating comments (which often manifest as written justifications of how the code satisfies the prompt). A comment in a tutorial is going to be geared to explaining how the code relates back to the tutorial task or restating what the code does, rather than documenting the "why" of surprising code.
- embedding-shape 2mo agoWhat models are you using? I've had "Never add any new code comments" in my system prompts for like a year at this point, seems every model above 14B picks this up just fine. Are you using llama 7b or similar for testing this?
- throwatdem12311 2mo agoPretty much Opus 5 exclusively.