3 ms·

Is this anything like what Langchain supports which is various strategies for chat history compression (summarisation)? https://www.pinecone.io/learn/langchain

Is this anything like what Langchain supports which is various strategies for chat history compression (summarisation)? https://www.pinecone.io/learn/langchain-conversational-memory/ https://www.pinecone.io/learn/langchain-conversational-memor... But aside from this, if I have a large document that I want to "chat" with, it looks like I am either chunking it and then selectively retrieving relevant subsections at question time, or I am naively dumping the whole document (that now fits in 32k) and then doing the chat, at a high cost. So 32k (and increased context size in general) does not look to be a huge gamechanger in patterns of use until cost comes down by an order of magnitude or two.

smodo 3y ago
Yes, that’s one example. Relatively simple to implement yourself. Any frontend for chatgpt already does the basic thing, which is to pass the previous messages along with the prompt. I think we may end up with first and second stage completions, where the first stage prepares the context for the second stage. The first stage can be a (tailored) gpt3.5 and the second stage can do the brainy work. That way you can actively control costs by making the first stage forward a context of a given maximum size.