> Finally, we have a third projection matrix, V, which maps x_j -> v_j. The v_j are d_v-dimensional vectors (doesn't have to equal d_k, and again something you

> Finally, we have a third projection matrix, V, which maps x_j -> v_j. The v_j are d_v-dimensional vectors (doesn't have to equal d_k, and again something you arbitrarily choose). These represent the values that the NN would like to pass on from item j to any other item which decides it "likes" the answer of item j. > So, to compute the new information to add to item i, we take the weighted sum of the values v_j that item i liked I don't get this part. If the v_k are d_v dimensional vectors, and if the input items i are all d_f dimensional vectors, then how are you "adding" these values back to the inputs when d_f != d_v? The linked script doesn't seem to do any adding like this - instead, they take the output values and pass them through a linear layer, presumably throwing away the inputs. But your comment hints at the existence of a more residual sort of approach.