3 ms·
Your last point is true only with positional encodings though, attention itself is a permutation equivariant function
by igorkraw 5y ago
Your last point is true only with positional encodings though, attention itself is a permutation equivariant function