3 ms·
Just shove all your information in the prompt. I assume that's what these guys are doing anyway. ChatGPT doesn't yet have any API, but you can use text-davinci-
by LeoPanthera 4y ago
Just shove all your information in the prompt. I assume that's what these guys are doing anyway. ChatGPT doesn't yet have any API, but you can use text-davinci-003, which does have an API and is functionally identical.
The API costs for a large prompt are quite high, and since you have to repeat it with every single query, your costs can go up very quickly.
- gecersiznick 4y agoGotcha! Thanks for the response. So if I have, say, 10 PDF files which I plan to extract text from and use to fine-tune davinci, I would have to supply the entire textual information extracted from the PDF file each time?
- geepytee 4y agodavinci-003 has a 4k tokens cap, you cannot just 'shove all your information in the prompt.' Can parallelize but then rate limits become a problem. Really curious to hear how these guys are handling it.
- KidComputer 4y agoYou shove the whole corpus in a vector db using embeddings, query the nearest neighbors to an input and inject those into a prompt to pass to GPT. https://langchain.readthedocs.io/en/latest/modules/chains/combine_docs_examples/chat_vector_db.html https://langchain.readthedocs.io/en/latest/modules/chains/co...
- hackernudes 4y agoI thought fine-tuning allows you do "inject the prompt" only once. https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning Ah, the rub is that a fine-tuned Davinci costs 6x a standard Davinci. Interesting. https://openai.com/api/pricing/ https://openai.com/api/pricing/