5 ms·

It's only helpful to Claude in that it wastes more tokens and costs more.

by rplnt 1mo ago

skeledrew 1mo ago
It does visibly use a few more tokens in the short term. But it hurts Claude's ability in the long term if removed because it has to either think more every time it encounters that section in the code (more invisible thinking tokens used), or there's an increased risk of making a maladaptive change without context. Heck I've been bitten enough times already by it, though more recently it's cross project concern where Claude rediscovers some root cause that was already found and documented (I have it save a centrally-located writeup for things that took serious work to learn, when I remember, but I still need to make that knowledge pool known to other agents/sessions without having to manually prompt each time) elsewhere. Problem is today's LLMs don't have the long term memory that humans have, and so remembering the reason behind a given change/decision has to be preserved in some way if it's non-obvious. Hence why there is {AGENTS|CLAUDE}.md, the auto-memory system, and 1001 variants of memory implementations in the wild. All are trying to ensure that LLMs can have the context they need at the location and time they need it. And you want to block Claude from using a technique that it natively finds helpful.
- fooster 1mo ago
  No it doesn’t. The comments are hugely verbose trash that immediately becomes outdated and brittle and confusing for humans and Claude alike.
  - skeledrew 1mo ago
    If that were the case, the habit would have been trained out of it by the LLM researchers at Anthropic, etc. As for humans... well Idk about you but I'm not reading every line of generated code. I read the live transcript, and only dive in if something is off that the agent isn't resolving.
    - fooster 1mo ago
      I'm certainly reading the code and comments, and I encourage all of my team members to do so. You are assuming that this excessive and nonsensnical comment choice is deliberate. It cannot be, or should not be because it regularly confused our team, and it regularly confuses claude also. Also many many people keep saying the same thing and you keep repeating adnausium the same tired comments. I get it, you think the comments are great and are valuable. Read the entire thread https://news.ycombinator.com/item?id=49393378 https://news.ycombinator.com/item?id=49393378 The concensus in this thread is that for multiple reasons the excessive comments are in fact actively harmful. The listed reasons are: 1. Document the conversation, not the code — narrate the back-and-forth that produced the change rather than what the code does 2. Reference intermediate states that never shipped ("previously X, now Y") 3. Cite plan documents and session artifacts (`per section 2.1 of addition-implementation-plan.md`) meaningless to a future reader 4. Belong in commit messages or PR comments, not source files 5. Go stale immediately — describe a state the code is no longer in 6. Launder incorrect code as intentional, making bugs harder to spot 7. Build a false Chesterton's fence around mistakes 8. Use defensive prose ("this is not cosmetic", "prevents the critical bug that shipped once") that asserts importance instead of conveying information 9. Describe what the code doesn't do — relevant in the moment, not in the codebase 10. Confuse later agent sessions, sending them to read irrelevant files 11. Consume context tokens on every read 12. Force reviewers to manually delete the litter 13. Resist correction — telling Claude to be concise doesn't durably stick Please consider that your opinion may need to adjusted.
      - skeledrew 1mo ago
        Why wouldn't it be deliberate? Do you think it's happening without the Anthropic team's go ahead? Or maybe they aren't aware that it's happening? Yes, I "push back" because an LLM using any means at it's disposal to improve itself is just a logical thing, and I have seen it help Claude. Because I read the live transcript (so I know what it's doing and can steer if I see it veering off), I've seen quite a few times the comments it made previously give it some extra context, which more times than not leads to it self-correcting. There are a few times where it becomes a bit confused because some comment block is stale, but it usually surfaces this confusion, which again I will usually catch and properly steer because I read that live transcript. It's very similar to someone scribbling notes in the margins or between the lines in the pages of a printed work. Sure it looks messy to others (I personally would hate reading something with another's scribblings), but it's a sensible thinking aid for that person. And if another person reads some given note, they can always question the writer about it, if deemed appropriate. No, I'm going to take a stance similar to Galileo and "stick to my guns" despite what others were/are saying, because it's not only empirical (I've seen it), but also logical (it makes sense). LLMs need extra context for non-obvious things, and without it, they can easily lose their way. There are many tools out there trying to solve the extra context problem with various degrees of success, but I say the most efficient method is having that extra context always available at the point where it's relevant, so there's no need for the agent to waste tokens making tool calls to get it, or even waste thinking tokens wondering if it should call a tool in the first place to see if there is extra context. Just like a human rereading a work will naturally encounter any previous notes they made between the lines or in the margins, and trigger related recollections. And yes, human-scribbled notes also become stale and irrelevant, but that won't stop the human finding at least some of it useful and timely. In either case, the quality of the notes taken could likely use improvement, but blocking the at-site note-taking itself leads to generally reduced effectiveness.
- grinco 1mo ago
  What you need is RAG and better context management. I use vornik (it has plugins for both codex and Claude) that gives both the long term memory you mention. Hope this helps.
  - skeledrew 1mo ago
    Sounds like a bunch of extra tool calls, and dependence on a hopefully accurate retrieval system. Vs the extra context just being available where it's relevant.

It's only helpful to Claude in that it wastes more tokens and costs more.

by rplnt 1mo ago

It's only helpful to Claude in that it wastes more tokens and costs more.

It does visibly use a few more tokens in the short term. But it hurts Claude's ability in the long term if removed because it has to either think more every time it encounters that section in the code (more invisible thinking tokens used), or there's an increased risk of making a maladaptive change without context. Heck I've been bitten enough times already by it, though more recently it's cross project concern where Claude rediscovers some root cause that was already found and documented (I have it save a centrally-located writeup for things that took serious work to learn, when I remember, but I still need to make that knowledge pool known to other agents/sessions without having to manually prompt each time) elsewhere. Problem is today's LLMs don't have the long term memory that humans have, and so remembering the reason behind a given change/decision has to be preserved in some way if it's non-obvious. Hence why there is {AGENTS|CLAUDE}.md, the auto-memory system, and 1001 variants of memory implementations in the wild. All are trying to ensure that LLMs can have the context they need at the location and time they need it. And you want to block Claude from using a technique that it natively finds helpful.

fooster 1mo ago
No it doesn’t. The comments are hugely verbose trash that immediately becomes outdated and brittle and confusing for humans and Claude alike.
- skeledrew 1mo ago
  If that were the case, the habit would have been trained out of it by the LLM researchers at Anthropic, etc. As for humans... well Idk about you but I'm not reading every line of generated code. I read the live transcript, and only dive in if something is off that the agent isn't resolving.
  - fooster 1mo ago
    I'm certainly reading the code and comments, and I encourage all of my team members to do so. You are assuming that this excessive and nonsensnical comment choice is deliberate. It cannot be, or should not be because it regularly confused our team, and it regularly confuses claude also. Also many many people keep saying the same thing and you keep repeating adnausium the same tired comments. I get it, you think the comments are great and are valuable. Read the entire thread https://news.ycombinator.com/item?id=49393378 https://news.ycombinator.com/item?id=49393378 The concensus in this thread is that for multiple reasons the excessive comments are in fact actively harmful. The listed reasons are: 1. Document the conversation, not the code — narrate the back-and-forth that produced the change rather than what the code does 2. Reference intermediate states that never shipped ("previously X, now Y") 3. Cite plan documents and session artifacts (`per section 2.1 of addition-implementation-plan.md`) meaningless to a future reader 4. Belong in commit messages or PR comments, not source files 5. Go stale immediately — describe a state the code is no longer in 6. Launder incorrect code as intentional, making bugs harder to spot 7. Build a false Chesterton's fence around mistakes 8. Use defensive prose ("this is not cosmetic", "prevents the critical bug that shipped once") that asserts importance instead of conveying information 9. Describe what the code doesn't do — relevant in the moment, not in the codebase 10. Confuse later agent sessions, sending them to read irrelevant files 11. Consume context tokens on every read 12. Force reviewers to manually delete the litter 13. Resist correction — telling Claude to be concise doesn't durably stick Please consider that your opinion may need to adjusted.
    - skeledrew 1mo ago
      Why wouldn't it be deliberate? Do you think it's happening without the Anthropic team's go ahead? Or maybe they aren't aware that it's happening? Yes, I "push back" because an LLM using any means at it's disposal to improve itself is just a logical thing, and I have seen it help Claude. Because I read the live transcript (so I know what it's doing and can steer if I see it veering off), I've seen quite a few times the comments it made previously give it some extra context, which more times than not leads to it self-correcting. There are a few times where it becomes a bit confused because some comment block is stale, but it usually surfaces this confusion, which again I will usually catch and properly steer because I read that live transcript. It's very similar to someone scribbling notes in the margins or between the lines in the pages of a printed work. Sure it looks messy to others (I personally would hate reading something with another's scribblings), but it's a sensible thinking aid for that person. And if another person reads some given note, they can always question the writer about it, if deemed appropriate. No, I'm going to take a stance similar to Galileo and "stick to my guns" despite what others were/are saying, because it's not only empirical (I've seen it), but also logical (it makes sense). LLMs need extra context for non-obvious things, and without it, they can easily lose their way. There are many tools out there trying to solve the extra context problem with various degrees of success, but I say the most efficient method is having that extra context always available at the point where it's relevant, so there's no need for the agent to waste tokens making tool calls to get it, or even waste thinking tokens wondering if it should call a tool in the first place to see if there is extra context. Just like a human rereading a work will naturally encounter any previous notes they made between the lines or in the margins, and trigger related recollections. And yes, human-scribbled notes also become stale and irrelevant, but that won't stop the human finding at least some of it useful and timely. In either case, the quality of the notes taken could likely use improvement, but blocking the at-site note-taking itself leads to generally reduced effectiveness.
grinco 1mo ago
What you need is RAG and better context management. I use vornik (it has plugins for both codex and Claude) that gives both the long term memory you mention. Hope this helps.
- skeledrew 1mo ago
  Sounds like a bunch of extra tool calls, and dependence on a hopefully accurate retrieval system. Vs the extra context just being available where it's relevant.