The main problem isn't actually context length most of the time. 128K is plenty for a lot of practical tasks. It's the generation length, both within turns and

The main problem isn't actually context length most of the time. 128K is plenty for a lot of practical tasks. It's the generation length, both within turns and especially across turns. And nobody knows how to increase that significantly yet.