3 ms·
How do you save a document vector and do similarity search with it?
by DennisP 3y ago
How do you save a document vector and do similarity search with it?
- PaulHoule 3y agoThere is this https://github.com/openai/chatgpt-retrieval-plugin https://github.com/openai/chatgpt-retrieval-plugin I just use SBERT which has models I can run locally https://sbert.net/ https://sbert.net/
- nl 3y agoYou encode your document with some kind of embedding, eg HuggingFace Sentence Transformers: https://www.sbert.net/ https://www.sbert.net/ (probably most commonly used) or OpenAI Embeddings: https://platform.openai.com/docs/guides/embeddings/what-are-embeddings https://platform.openai.com/docs/guides/embeddings/what-are-... and then use a vector database (Elastic, Postgres, FAISS or whatever) to do a similarity search.