4 ms·
Theoretical limitations of multi-layer Transformer https://arxiv.org/abs/2412.02975 https://arxiv.org/abs/2412.02975 On Limitations of the Transformer Architec
by namaria 1y ago
Theoretical limitations of multi-layer Transformer https://arxiv.org/abs/2412.02975 https://arxiv.org/abs/2412.02975
On Limitations of the Transformer Architecture https://arxiv.org/abs/2402.08164 https://arxiv.org/abs/2402.08164
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory https://arxiv.org/abs/2405.16674 https://arxiv.org/abs/2405.16674
TL;DR transformers are inherently limited with tasks requiring composition of sequential steps