4 ms·
Attention doesn’t have a window, unless you mean there’s a limit to the number of absolute positional embeddings available at train time?
by jerpint 4y ago
Attention doesn’t have a window, unless you mean there’s a limit to the number of absolute positional embeddings available at train time?