It would be interesting to see how does ring attention technique affect this. This maybe still valid for cost reasons, but unlimited context is like in-memory c

It would be interesting to see how does ring attention technique affect this. This maybe still valid for cost reasons, but unlimited context is like in-memory computing vs. traditional. https://arxiv.org/pdf/2310.01889.pdf https://arxiv.org/pdf/2310.01889.pdf