Right, I get the goal of removing the outlier activations, but I just don't understand why outlier activations are a consequence of the model trying to "pass".

Right, I get the goal of removing the outlier activations, but I just don't understand why outlier activations are a consequence of the model trying to "pass". The story from the linked paper earlier in the post (https://arxiv.org/pdf/2306.12929.pdf https://arxiv.org/pdf/2306.12929.pdf) is that the model is doing the following: -Learn a near-zero representation for some otherwise low-importance token, like delimiters or whitespace. -When a head wants to "pass", emit an outlier activation to attend to that token nearly-exclusively. But I'm surprised the model can't just use its existing tools (the post-concat projection layer and the following MLP block) to achieve the same thing. And if the answer is that it could do that, but tends to learn to use the outlier activation trick instead, will giving it a new tool that still allows the use of outlier activations be sufficient?