4 ms·
>letting the SSM parameters be functions of the input That’s what the attention mechanism has going for it. Different weights for different sets of tokens.
by intalentive 3y ago
>letting the SSM parameters be functions of the input
That’s what the attention mechanism has going for it. Different weights for different sets of tokens.
- wizzard0 3y agoAuthors claim their mechanism is more computationally efficient (both asymptotically and on current hardware) than the Transformer attention, while performing much the same role, yes