While the general problem of limited context windows remains, the example given regarding Novellas is highly outdated. Even many open models now offer >128k tok

While the general problem of limited context windows remains, the example given regarding Novellas is highly outdated. Even many open models now offer >128k tokens, which makes it possible to fit an entire novella in the context window (using rule of thumb, 128k tokens ~= 100k words ~= 300 pages of english text). The frontier models even support context in the millions of tokens range, making it possible to keep an entire novella series inside the context window. I feel most memory related problems will simply disappear due to increasing context lengths, because attempts at outsourcing LLM memory or producing theoretically infinite context have beein going on for years and have not yet produced anything that can compete with raw scaling of transformers.