4 ms·
The research mostly points to LLM-generated context lowering performance. Human-generated context improves performance, but any kind of AGENTS.md file increases
by noemit 7mo ago
The research mostly points to LLM-generated context lowering performance. Human-generated context improves performance, but any kind of AGENTS.md file increases token use, for what they say is "fake thinking." More research is needed.
- d1sxeyes 7mo agoAgree. Also, sometimes I intentionally want the agent to do something differently to how it would naturally solve the problem. For example, there might be a specific design decision that the agent should adhere to. Obviously, this will lead to slower task completion, higher inference costs etc. because I’m asking the agent not to take the path of least resistance. This kind of benchmark completely misses that nuance.
- stingraycharles 7mo agoI’d say that it needs to be maintained and reviewed by a human, but it’s perfectly fine to let an LLM generate it.
- sheept 7mo agoIf you let an LLM generate it (e.g. Claude's /init), it'll be a lot more verbose then it needs to be, which wastes tokens and deemphasizes any project-specific preferences you actually want the agent to heed.
- stingraycharles 7mo agoObviously you’re not going to let it generate it on auto-pilot.