19 ms·
To pay attention to everything, set the query vector to 0. Then all attention scores will be equal and the attention output is the average of the value vectors.
by yorwba 1y ago
To pay attention to everything, set the query vector to 0. Then all attention scores will be equal and the attention output is the average of the value vectors.