3 ms·

So I can let llama2 summarize books now or are there any non-obvious caveats to this approach?

• Sharlin 3y ago
  No. This does nothing to the context length itself which is still a sliding window.
• Tostino 3y ago
  If you want to do that, I have a model trained specifically on a dataset of building recursive summaries. Some of my training documents are 40-50k tokens. https://huggingface.co/Tostino/Inkbot-13B-8k-0.2 https://huggingface.co/Tostino/Inkbot-13B-8k-0.2 Just chunk your document up, and pass in the prior summary along with this chunk of text, you can mention that it is chunk X of Y if you want (which can help with how it starts the summary often).