15 ms·
So I just started doing this using LLM embeddings for semantic search. It actually works quite well. You index every piece of data with metadata and it's conten
by asim 1y ago
So I just started doing this using LLM embeddings for semantic search. It actually works quite well. You index every piece of data with metadata and it's content. Then you choose specific metadata fields you might want to correlate on e.g knowing two pieces of data are of type "product" or "design" and then the query will return the related items. OpenAI gets used for turning your query into what can then be used against your index which is basically a vector dB. If you are using Go then chromem-go does this quite easily and has examples.