If I followed this correctly and didn’t mess up my indices, adding 1 to the softmax denominator is exactly equivalent to appending an extra zero to the softmax

If I followed this correctly and didn’t mess up my indices, adding 1 to the softmax denominator is exactly equivalent to appending an extra zero to the softmax input (effectively casting an exp(0) vote for a new null option) and appending an extra row of zeros to V (so the null option is all zeros). The latter seems like something training could figure out by itself (zero doesn’t seem like a hard place to land with the weights producing V, although a bunch of zero weights would be needed), but the former is a bit awkward, as QK^T is quadratic in the weights. In any case, this seems intuitively quite reasonable. But I do wonder whether the 1 in the denominator (equivalent to an exp(0) vote) is the best choice if the goal is to quantize well. 0 is in the middle of the numerical range, and perhaps the implicit null vote should be weighted lower than the middle of the range.