7 ms·
Yes. You can break the document up and index each part and then tackle it that way. It works surprisingly well. The 4096 token limit is tied to the attention wi
by hnhg 4y ago
Yes. You can break the document up and index each part and then tackle it that way. It works surprisingly well. The 4096 token limit is tied to the attention window, not an API restriction.
- jerpint 4y agoAttention doesn’t have a window, unless you mean there’s a limit to the number of absolute positional embeddings available at train time?
- freezed88 4y agoLlamaIndex offers ways to chunk up your data and store them in data structures for response synthesis: https://gpt-index.readthedocs.io/en/latest/guides/primer.html https://gpt-index.readthedocs.io/en/latest/guides/primer.htm...