3 ms·
Problem I'm seeing with RAG-like solutions is you're presenting the LLM with some similar word blocks and hoping it covers all your bases, but won't pick up on
by muttled 3y ago
Problem I'm seeing with RAG-like solutions is you're presenting the LLM with some similar word blocks and hoping it covers all your bases, but won't pick up on 2nd order relationships that would be important for a more complete picture. Which isn't a whole lot better than just showing the search results to the user and might actually introduce faulty information. I think we either need a different model type that can serve as the "memory" or consider options like pre-training to ingest the data that come at the cost of needing much larger servers to perform the operation.
- cjbprime 3y agoI don't know whether you're correct, but I'm not sure I followed your intuition for why the LLM could access the second order relationship during inference, and yet it wouldn't appear in the distance threshold on a vector search during retrieval. Any pointers to writing explaining that?
- ParetoOptimal 3y agoI have an example of this I've been using, only gpt4 and nous capybara 34B have gotten right: > what is the song from the deadpool movies that begins with arf arf. I get some wild examples and many llms get stuck insisting it is "Shoop".
- _boffin_ 3y agoYou may want to read this: https://arxiv.org/pdf/2304.03442.pdf https://arxiv.org/pdf/2304.03442.pdf