Not quite. If some values are filler and some are not, and the corresponding keys are linearly separable, it's not difficult to find a query where standard atte

Not quite. If some values are filler and some are not, and the corresponding keys are linearly separable, it's not difficult to find a query where standard attention gives low attention scores to filler and high attention scores to non-filler. Attention sinks deal with the problem when everything is filler, so there's no non-filler to allocate attention to.