3 ms·
By now there seem to exist many different llms that perform almost as good as OpenAI’s gpt 3.5. But what is the sota of adopting llms for the use with custom o
by zapnuk 3y ago
By now there seem to exist many different llms that perform almost as good as OpenAI’s gpt 3.5.
But what is the sota of adopting llms for the use with custom or „live“ data?
I know OpenAI has function calling, and a vector db like pinecone can somehow be used as a knowledge base to introduce more context to the query, or response.
Are there other methods to make the open source llms more useful if you have a huge amount of data?
- muttled 3y agoProblem I'm seeing with RAG-like solutions is you're presenting the LLM with some similar word blocks and hoping it covers all your bases, but won't pick up on 2nd order relationships that would be important for a more complete picture. Which isn't a whole lot better than just showing the search results to the user and might actually introduce faulty information. I think we either need a different model type that can serve as the "memory" or consider options like pre-training to ingest the data that come at the cost of needing much larger servers to perform the operation.
- cjbprime 3y agoI don't know whether you're correct, but I'm not sure I followed your intuition for why the LLM could access the second order relationship during inference, and yet it wouldn't appear in the distance threshold on a vector search during retrieval. Any pointers to writing explaining that?
- ParetoOptimal 3y agoI have an example of this I've been using, only gpt4 and nous capybara 34B have gotten right: > what is the song from the deadpool movies that begins with arf arf. I get some wild examples and many llms get stuck insisting it is "Shoop".
- _boffin_ 3y agoYou may want to read this: https://arxiv.org/pdf/2304.03442.pdf https://arxiv.org/pdf/2304.03442.pdf