4 ms·
The fact they are getting perfect recall with millions of tokens rules out any of the existing linear attention methods.
by sebzim4500 3y ago
The fact they are getting perfect recall with millions of tokens rules out any of the existing linear attention methods.
- cs702 3y agoI wouldn't be so sure perfect recall rules out linear RNNs, because I haven't seen any conclusive data on their ability to recall. Have you?