28 ms·
Recent example of a proof regarding theoretical limitations of Transformers: https://aclanthology.org/2023.tacl-1.31.pdf https://aclanthology.org/2023.tacl-1.31
by opprobium 2y ago
Recent example of a proof regarding theoretical limitations of Transformers: https://aclanthology.org/2023.tacl-1.31.pdf https://aclanthology.org/2023.tacl-1.31.pdf (also extended to cover SSMs https://arxiv.org/pdf/2404.08819 https://arxiv.org/pdf/2404.08819)
- mewpmewp2 2y agoI'm not sure if this paper corresponds to limits on what it can answer with a single or few tokens, but also the limits where LLM itself is allowed to produce more tokens (chain of thought) as well as use tools (coding) to solve problems?