3 ms·
Do the OpenAI APIs support converting prompts to vectors, or are people running their own models locally to do this? Can you recommend any good resources to rea
by HarHarVeryFunny 4y ago
Do the OpenAI APIs support converting prompts to vectors, or are people running their own models locally to do this? Can you recommend any good resources to read up on vector DB approaches to working around context length limits ?
- make3 4y agoopen ai has an embeddings api that ppl use for that https://platform.openai.com/docs/guides/embeddings https://platform.openai.com/docs/guides/embeddings, though whether it's the best model to do that is congested. Contriever is an example of a strong model to do that yourself. see their paper too to learn about the domain. https://github.com/facebookresearch/contriever https://github.com/facebookresearch/contriever
- HarHarVeryFunny 4y agoThanks!
- ukuina 4y agoYes, you can use a local embeddings model like gtr-t5-xl alongside retriever augmentation. This can point you in the right direction: https://haystack.deepset.ai/tutorials/22_pipeline_with_promptnode https://haystack.deepset.ai/tutorials/22_pipeline_with_promp...
- HarHarVeryFunny 4y agoThanks!
- tuanacelik 4y agoIndeed this tutorial on Haystack is a good one as an example: https://haystack.deepset.ai/tutorials/22_pipeline_with_promptnode https://haystack.deepset.ai/tutorials/22_pipeline_with_promp... It combines a retrieval step followed by a prompt layer which inserts the relevant context into the prompt. You can however change the 'retrieval step' with something that uses a proper embedding model and OpenAI also provides those if you want to. I tend to use lighter (cheaper) OSS models for this step though. PS: There's some functionality in the PromptNode to make sure you don't exceed prompt limit.
- HarHarVeryFunny 4y agoThat's great - thanks!