4 ms·
There is an annoying phenomenon with LLMs called "context dilution" or "attention dilution" that was outlined in the Lost in the Middle paper. As the context gr
by austin-cheney 2mo ago
There is an annoying phenomenon with LLMs called "context dilution" or "attention dilution" that was outlined in the Lost in the Middle paper. As the context grows, a model starts paying less attention to instructions in the middle of the context in favor of what is at the beginning and the end.
So then what happens if the agents.md file is colossal and precise and all the required execution instructions are buried in the middle? Do the agents then just fail to execute?
- lhk931122 2mo agoYeah. That's why people these days avoid long descriptions and instead keep things as short as possible. in my experience, it seems like LLM can't recognize (or less attention value) unless it's structured in a deductive or inductive way