3 ms·

I have no idea what you said, could you perhaps elaborate on some sample ideas this tool helped you and what is the 8K token limit and why was it limiting you?

• dragonwriter 3y ago
  > Does that 8K limit, limit your context length, i.e. history you have per chat thread with the tool? Yes, the token limit for an LLM limits the combination of the prompt (which normally includes the whole conversation history, as the LLM itself has no memory) and response. There's tricks to have a longer conversation without completely forgetting the past (summarization, offloading parts to a database, usually indexed by embedding vectors, and using search to recall relevant history, etc.) but the base case is everything has to fit into context.