3 ms·
I don't buy the arguments made here. You can't call it attention optimized without opening the LLM brain and assessing what happens in the attention layers. Doe
by eric-burel 1y ago
I don't buy the arguments made here. You can't call it attention optimized without opening the LLM brain and assessing what happens in the attention layers. Does the quoted paper did some of that? I know Anthropic is advanced in this area but I haven't seen that many results elsewhere yet. I mean the fact that optimizing for attention makes better prompt is a solid hypothesis, but I don't read a proof here.
- ozgung 1y agoI think the author is confusing the attention mechanism with attention in the general sense. He refers to a phenomenon called “attention drift”, but neither Google nor LLM searches return any relevant references to that term. In fact, ChatGPT only cited this same blog post.
- eric-burel 1y agoI think LLM vendors papers entertain such confusions, the lack of peer reviewing tends to favour grandiose but barely defined wording as such. At least author seems to have checked a few papers that do some empirical validation (which is still different from establishing any kind of theory of prompt engineering, there is no other way than cutting the model up to do that).
- jdefr89 1y agoYea they sound like plausible enough arguments.. I often test my vague input against structured ones and for many tasks it didn't seem like hallucinations happened significantly less. Honestly the more structured you have to be, the less "general" your model probably is, and in theory you want your model to be as general as possible. Seems like here, you're simply helping it over-fit.. At least that's what my intuition tells me but I am yet to really check that out either.