3 ms·
> In short, what attention does is it moves the words in a sentence (or piece of text) closer in the word embedding. In that way, the word “bank” in the sentenc
by throwoutway 4y ago
> In short, what attention does is it moves the words in a sentence (or piece of text) closer in the word embedding. In that way, the word “bank” in the sentence “Money in the bank” will be moved closer to the word “money”.
It's unclear to me. How does this "move" closer? Are the vector positions in the NN changed temporarily and it carries a local copy across the blocks?