5 ms·
The numbers are high, but whether 8k is low depends on your use case. Do you want to process whole book chapters, or feed lots of related documents at the same
by viraptor 3y ago
The numbers are high, but whether 8k is low depends on your use case. Do you want to process whole book chapters, or feed lots of related documents at the same time? If not, and you're just doing a normal question/answer session with some priming prompt, 8k is already a lot.
- kolinko 3y ago8k is very little if you want to add almost any additional data in context, or have a more complicated prompt. Otherwise your knowledge retrieval needs to be almost spot on for llm to provide a proper reply. Ditto with any multi shot prompts.