tons and tons of papers, most of them had some disadvantages. Can't have the cake and eat it too: https://arxiv.org/html/2404.08801v1 https://arxiv.org/html/24

tons and tons of papers, most of them had some disadvantages. Can't have the cake and eat it too: https://arxiv.org/html/2404.08801v1 https://arxiv.org/html/2404.08801v1 Meta Megalodon https://arxiv.org/html/2404.07143v1 https://arxiv.org/html/2404.07143v1 Google Infini-Attention https://arxiv.org/html/2402.13753v1 https://arxiv.org/html/2402.13753v1 LongRoPE and a ton more