3 ms·
Oh? Do you have a link to that paper? I’m no linalg expert, but it was my understanding that attention worked (very basically) because each token attends to al
by dartos 2y ago
Oh? Do you have a link to that paper?
I’m no linalg expert, but it was my understanding that attention worked (very basically) because each token attends to all other tokens in the sequence, but you can’t do that with an unbounded sequence length
- drdeca 2y agoI think it was https://aclanthology.org/2022.tacl-1.49/ https://aclanthology.org/2022.tacl-1.49/ But am now thinking I may have made a mistake, with this showing they can be simulated in TC^0 when I thought the result was both that and that they can simulate TC^0 .