3 ms·

How do you stop LLMs from making comments? In my experience, LLMs treat requirements for code output as suggestions

deleted 1mo ago
[deleted]
miki_oomiri 1mo ago
Ask the agent to write a script to run after each changes, against the newly added code. Use that script as a super linter. That’s the only way I found to strictly enforce some rules, like the no comments rule, without enforcing them against my own changes or old code.
- eru 1mo ago
  You could run the script mechanically against the diff (assuming you use version control). No need to rely on the agent.
ks2048 1mo ago
If you don’t trust the code to write a decent comment, why trust it write good code? Of course, ensuring compilation or other checks can verify some code, which it can’t do for comments. But comments still serve the same purpose as human comments.
- Infernal 1mo ago
  If I understand correctly, it’s not that the LLM can’t write a good comment, it’s that you want to be able to interpret and understand the generated code without comments - and in that process end up writing comments yourself.
  - manwe150 1mo ago
    I don’t get that argument. Most of the time by the end of the session the comments from the agent encode tricky details that I told the agent to write down so it stops making “simplifying” assumptions. Thus comments at the end of a couple days of agent-only coding, when I start to actually read and edit the prose, contain the details which aren’t possible to know from reading the local code. It may help that my last couple sessions before I start reading the code myself are variations on telling the agent to self-review and improve the comments in specific ways, so the comments left are only those the agent thought remain meaningful at clarifying unexpected interactions between the local code and other code that needs to be referenced to understand it.
- robby_w_g 1mo ago
  The actual code output has improved a lot over the past year. I’ve found it matches existing patterns better, and the code is succinct so I can easily tweak it if I don’t like the way the agent wrote it. The problem with comments is that LLMs tend to copy their verbose chat output format and insert session/prompt specific details. It makes me think that LLMs aren’t constrained in their comment output the same way they are with their code output
embedding-shape 1mo ago
Add "Don't add any code comments anywhere" to your system prompt. If the model doesn't follow this, you want to start using a better model ASAP, because SOTA models for the last year or so, been able to following this without an issue.
- yehoshuapw 1mo ago
  I've come recently across arxiv 2604.20911 which claims "do" rules persist much better then "don't" rules.
  - embedding-shape 1mo ago
    Alright, what you'd put instead of "Don't add any code comments anywhere"? I agree with the general guidance, but it's a general one and not applicable for everything. Some things cannot be expressed in a "do" way rather than "don't".
    - yehoshuapw 1mo ago
      I don't know. I fully agree, and never did really try out this in depth yet. I suspect rules with negations are not the same as don't rules, but unknown if really true, if so: "when writing code do not add comments, code should not need it" may work
      - embedding-shape 1mo ago
        Your proposal is still a "Prohibition-type constraint" that the paper you linked earlier say "isn't good". Some of these constraints we want simply aren't possible without adding "do not" somewhere in the line, even if you prefix/suffix it with other stuff, as you noticed yourself :)
      - wongarsu 1mo ago
        I don't think changing "don't" to "do not" is what the paper authors had in mind I use things like "Your code should be self-documenting, so as to require as few comments as possible. Add comments to explain "why" or give important context not apparent from the code itself, but keep them to necessary comments only" But that's a much laxer rule. I don't think you can truly express "no comments, ever" without a "don't" rule.
nicky0 1mo ago
In my experience, Claude adds loads of comments, but Codex (GPT-5.5) never adds any.

deleted 1mo ago

[deleted]

Ask the agent to write a script to run after each changes, against the newly added code. Use that script as a super linter. That’s the only way I found to strictly enforce some rules, like the no comments rule, without enforcing them against my own changes or old code.

eru 1mo ago
You could run the script mechanically against the diff (assuming you use version control). No need to rely on the agent.

ks2048 1mo ago

If you don’t trust the code to write a decent comment, why trust it write good code? Of course, ensuring compilation or other checks can verify some code, which it can’t do for comments. But comments still serve the same purpose as human comments.

Infernal 1mo ago
If I understand correctly, it’s not that the LLM can’t write a good comment, it’s that you want to be able to interpret and understand the generated code without comments - and in that process end up writing comments yourself.
- manwe150 1mo ago
  I don’t get that argument. Most of the time by the end of the session the comments from the agent encode tricky details that I told the agent to write down so it stops making “simplifying” assumptions. Thus comments at the end of a couple days of agent-only coding, when I start to actually read and edit the prose, contain the details which aren’t possible to know from reading the local code. It may help that my last couple sessions before I start reading the code myself are variations on telling the agent to self-review and improve the comments in specific ways, so the comments left are only those the agent thought remain meaningful at clarifying unexpected interactions between the local code and other code that needs to be referenced to understand it.
robby_w_g 1mo ago
The actual code output has improved a lot over the past year. I’ve found it matches existing patterns better, and the code is succinct so I can easily tweak it if I don’t like the way the agent wrote it. The problem with comments is that LLMs tend to copy their verbose chat output format and insert session/prompt specific details. It makes me think that LLMs aren’t constrained in their comment output the same way they are with their code output

embedding-shape 1mo ago

Add "Don't add any code comments anywhere" to your system prompt. If the model doesn't follow this, you want to start using a better model ASAP, because SOTA models for the last year or so, been able to following this without an issue.

yehoshuapw 1mo ago
I've come recently across arxiv 2604.20911 which claims "do" rules persist much better then "don't" rules.
- embedding-shape 1mo ago
  Alright, what you'd put instead of "Don't add any code comments anywhere"? I agree with the general guidance, but it's a general one and not applicable for everything. Some things cannot be expressed in a "do" way rather than "don't".
  - yehoshuapw 1mo ago
    I don't know. I fully agree, and never did really try out this in depth yet. I suspect rules with negations are not the same as don't rules, but unknown if really true, if so: "when writing code do not add comments, code should not need it" may work
    - embedding-shape 1mo ago
      Your proposal is still a "Prohibition-type constraint" that the paper you linked earlier say "isn't good". Some of these constraints we want simply aren't possible without adding "do not" somewhere in the line, even if you prefix/suffix it with other stuff, as you noticed yourself :)
    - wongarsu 1mo ago
      I don't think changing "don't" to "do not" is what the paper authors had in mind I use things like "Your code should be self-documenting, so as to require as few comments as possible. Add comments to explain "why" or give important context not apparent from the code itself, but keep them to necessary comments only" But that's a much laxer rule. I don't think you can truly express "no comments, ever" without a "don't" rule.

nicky0 1mo ago

In my experience, Claude adds loads of comments, but Codex (GPT-5.5) never adds any.