OP is right in that his change would make the softmax in the attention output zero if it "has nothing to add" (QuietAttention, as he said). Buuut, it's missing

OP is right in that his change would make the softmax in the attention output zero if it "has nothing to add" (QuietAttention, as he said). Buuut, it's missing the forest for the trees. The goal of the last step of attention (ref., Fig. 2, left in https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762) is not to add/say anything (as the author is saying) but to compute the relationship between the tokens (QK^T) and V -- in layman terms, simplifying, which tokens are related to each other. The softmax is there because it gives a representation that is nicer to work with, it gives probabilities, instead of unscaled matrix multiplication. TLDR; author isn't wrong but he isn't right, practically speaking, either.