3 ms·
davinci-003 has a 4k tokens cap, you cannot just 'shove all your information in the prompt.' Can parallelize but then rate limits become a problem. Really curio
by geepytee 4y ago
davinci-003 has a 4k tokens cap, you cannot just 'shove all your information in the prompt.' Can parallelize but then rate limits become a problem. Really curious to hear how these guys are handling it.
- KidComputer 4y agoYou shove the whole corpus in a vector db using embeddings, query the nearest neighbors to an input and inject those into a prompt to pass to GPT. https://langchain.readthedocs.io/en/latest/modules/chains/combine_docs_examples/chat_vector_db.html https://langchain.readthedocs.io/en/latest/modules/chains/co...