3 ms·
The article is correct about the current state of using LLMs, but I didn't see an explanation WHY they are like this; just more "how". I'm curious about the fu
by causal 1y ago
The article is correct about the current state of using LLMs, but I didn't see an explanation WHY they are like this; just more "how".
I'm curious about the fundamental reason why LLMs and their agents struggle with executive function over time.
- namaria 1y agoTheoretical limitations of multi-layer Transformer https://arxiv.org/abs/2412.02975 https://arxiv.org/abs/2412.02975 On Limitations of the Transformer Architecture https://arxiv.org/abs/2402.08164 https://arxiv.org/abs/2402.08164 Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory https://arxiv.org/abs/2405.16674 https://arxiv.org/abs/2405.16674 TL;DR transformers are inherently limited with tasks requiring composition of sequential steps