4 ms·
To calculate embeddings for free, use this very popular model: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 https://huggingface.co/sentence-tra
by computerex 3y ago
To calculate embeddings for free, use this very popular model:
https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v...
For storing the vectors and doing the vector search:
https://github.com/pgvector/pgvector https://github.com/pgvector/pgvector
`ankane/pgvector` docker image is a drop in replacement for the postgres image, so you can fire this up with docker very quickly.
It's a normal postgres db with a vector datatype. It can index the vectors and allows efficient retrieval. Both AWS RDS and Google Cloud now support this in their managed Postgres offerings, so postgres+pgvector is a viable managed production vectordb solution.
> Also, how granular should the text chunks be?
That depends on the use case, the size of your corpus, the context of the model you are using, how much money you are willing to spend.
> Has anyone been able to achieve reliable results from these? Preferably w/o using Langchain.
Definitely. We use postgres+pgvector with php.
- ClassyJacket 3y ago>This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space Interesting. Can someone explain to me why 384 specifically? How did they arrive at that number?
- computerex 3y agoIt has to do with the architecture of the network used to create the embeddings. The embeddings is actually the output of the final layer of the model. The dimensionality is a function of the number of parameters in that layer. Different models/architectures will produce different dimension embeddings.
- bigbillheck 3y agoIt's not a power of two but is still a nice round number (0x180).
- kiwicopple 3y ago(Disclosure: I work at supabase but this applies to any Postgres host that has the pgvector extension) Here is a guide/notebook using the model above, pgvector, and a python client: https://supabase.com/docs/guides/ai/quickstarts/text-deduplication https://supabase.com/docs/guides/ai/quickstarts/text-dedupli...
- tomrod 3y agoI see the all-MiniLM-L6-v2 is very popular. How does it compare to hkunlp/instructor-large?
- tallytarik 3y agoI’ve had some success with this. I think I’d need to fine tune the model to see better results with some domain-specific terms, but I couldn’t find much information about how to actually do that - what sort of input data you need, how much of it, etc. Would be interested to hear if anyone had more to share about fine tuning these models for semantic search.
- famouswaffles 3y agoInstead of fine-tuning, you might try out the Instructor series from here. https://instructor-embedding.github.io/ https://instructor-embedding.github.io/ https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard Open source and the way it works is that you give an instruction on the type of task or even domain you want the embedding to be tailored to and embeddings change depending on the instructions.
- david_shi 3y agoInstructor performs very well but is still limited to its training corpus. For niche domains, fine-tuning can be helpful in improving performance.