3 ms·
Thanks, will have a look. Yeah in this case I'd want to provide the text content. I also just found pgvector which seems like it could help? It has a similarit
by politelemon 3y ago
Thanks, will have a look. Yeah in this case I'd want to provide the text content.
I also just found pgvector which seems like it could help? It has a similarity search.
- m3at 3y agoI would only recommend pgvector if you're already primarily relying on postgres and the scale is limited (<1M documents). It won't handle the part that generates the embeddings though. You could use cloud vendors if you're in one of their ecosystem, do it yourself [1] (but model serving can be tricky without prior experience in ML), or use some other service to generate embeddings [2]. Alternatively vespa cloud [3] offer both but… not the easiest to work with, it's tailored for businesses where search is a primary component. Feel free to shoot me an email (in profile) with your context if you have more questions, in case I can help [1] This model is a solid baseline if you're working with English text: https://huggingface.co/sentence-transformers/all-mpnet-base-v2 https://huggingface.co/sentence-transformers/all-mpnet-base-... [2] OpenAI's embeddings is probably the easiest to get started, and the API is straightforward. It's not the best performing embeddings for retrieval but good enough in some cases: https://platform.openai.com/docs/guides/embeddings/use-cases https://platform.openai.com/docs/guides/embeddings/use-cases [3] https://cloud.vespa.ai/ https://cloud.vespa.ai/