3 ms·
> * There's no mention of model performance on recall tasks. Models with attention do well on recall tasks, but models without it, like this one, tend to do poo
by phh 2y ago
> * There's no mention of model performance on recall tasks. Models with attention do well on recall tasks, but models without it, like this one, tend to do poorly.[a] What is the performance of this model on recall tasks?
For what it's worth, RWKV's (another "transformer-less") website on that matter mention that yes it's bad on recall, but for the vast majority of tasks you can just ask the question *before* the content, and it'll handle the task just fine. (I'm just reporting, I haven't spent time trying myself)