3 ms·
Can you expand on the "cannot solve fundamentally" part?
by WithinReason 2y ago
Can you expand on the "cannot solve fundamentally" part?
- lucidrains 2y agohttps://arxiv.org/abs/2404.08819 https://arxiv.org/abs/2404.08819
- deleted 2y ago[deleted]
- Der_Einzige 2y agoSo does anything do proper state tracking? And don’t point to the OP since very often purportedly better new architectures end up being basically vaporware (like mamba or rkwv, which still don’t have good quality pre trained models yet)
- impossiblefork 2y agoHow do you mean vaporware? Surely whether a big model using a certain system exists is only a matter of the choices of those with sufficient resources to train it. That's only a matter of their beliefs, not about actual model performance.
- thomasahle 2y agoTransformers and SSMs can't do long computations that are inherently sequential. Unless you give them chain of thought. In which case they do great.