2 ms·
Do they believe that it's likely to scale as high as "standard" LLMs? Or is this another state space machine moment where it turns out to be equivalent to them?
by free_bip 2y ago
Do they believe that it's likely to scale as high as "standard" LLMs? Or is this another state space machine moment where it turns out to be equivalent to them?
- kadushka 2y agoThe concept of a memory module, separate from a prediction engine, makes sense to me, our brains might operate like that (short vs long term memory). Many research groups have been trying to implement this idea, recent paper from Meta comes to mind: https://arxiv.org/html/2412.09764 https://arxiv.org/html/2412.09764 If they believe it’s likely to scale, they will try to scale it, so we will either hear about it in a few months, or we won’t.