4 ms·
It isnt very hard. The easy (and probably best alt unless you are big enough to justify training your own LLM everytime your documents change) is to use vector
by victorbjorklund 3y ago
It isnt very hard. The easy (and probably best alt unless you are big enough to justify training your own LLM everytime your documents change) is to use vector search to find the most relevant parts of your documents (I use openai embeeder and pgvector for postgres) and then you feed that text to an LLM (could be GTP4 or Llama) and asks it to answer the question using the text you provide.
- azmodeus 3y agoHow much of an improvement was using OpenAI vectors over roberta or other embeddings for you?
- victorbjorklund 3y agoI actually didn't try any open source embeddings because openai is just so cheap that it didn't make sense to spend time setting it up (probably makes sense if you have a lot of searches). Wanna try some just for fun though, got any recommendations?
- catlover76 3y agoThat isn't what the person really wants to do though. I am building an application to use AI to query sets of documents in the legal space right now, for example, and this method has severe limitations. The LLMs themselves need fine-tuning for the purpose and bigger context windows, not to mention the bottleneck of the initial vector search itself.