4 ms·

There is a nice opensource extension that does exactly that: https://github.com/memex-life/memex https://github.com/memex-life/memex I tried to use it, and it

There is a nice opensource extension that does exactly that: https://github.com/memex-life/memex https://github.com/memex-life/memex I tried to use it, and it just does not work very well, I don't it's because implementation is bad, it's just RAG (retrieval based generation) does not work well outside of some simple use cases.

danielbln 3y ago
What are those simple use cases, and where do you see Retrieval Augmented Generation fall over?
- coolvision 3y ago
  I think it works better if query is a larger chunk of text. Like, if you have an email from a customer and want to compose a response based on some relevant documentation, it should work well. But for a use case where you want to retrieve something from browsing history you would mainly use a short search query, just few words. in this case embeddings are too ambiguous and relevance of retrieved content is not great.
  - janalsncm 3y ago
    That’s not a problem with RAG itself that’s an issue with your retriever. In the original RAG paper they used two vanilla BERT models and cosine similarity but there’s no requirement you do that. Use any retriever that gets you high precision. Use BM25 if you want, it’s simple and cheap. You’re right in saying there’s not enough semantic meaning in the text of the query. The domain of queries and the domain of documents are very different. That’s why a real retrieval system will train the query encoder and doc encoder to be closer in their embedding space using click data. This is what Google is doing.
    - ayaanmomin 3y ago
      "train the query encoder and doc encoder to be closer in their embedding space using click data" <- Any papers/resources you know where I can learn more about this process?
      - janalsncm 3y ago
        Triplet loss. https://pytorch.org/docs/stable/generated/torch.nn.TripletMarginLoss.html https://pytorch.org/docs/stable/generated/torch.nn.TripletMa... Triplet loss takes an anchor, positive, and negative. In this case the anchor is your query, the positive is a similar doc, and the negative is a dissimilar doc. When you train, backpropagate the loss to both the doc and the query encoder.

danielbln 3y ago

What are those simple use cases, and where do you see Retrieval Augmented Generation fall over?

coolvision 3y ago
I think it works better if query is a larger chunk of text. Like, if you have an email from a customer and want to compose a response based on some relevant documentation, it should work well. But for a use case where you want to retrieve something from browsing history you would mainly use a short search query, just few words. in this case embeddings are too ambiguous and relevance of retrieved content is not great.
- janalsncm 3y ago
  That’s not a problem with RAG itself that’s an issue with your retriever. In the original RAG paper they used two vanilla BERT models and cosine similarity but there’s no requirement you do that. Use any retriever that gets you high precision. Use BM25 if you want, it’s simple and cheap. You’re right in saying there’s not enough semantic meaning in the text of the query. The domain of queries and the domain of documents are very different. That’s why a real retrieval system will train the query encoder and doc encoder to be closer in their embedding space using click data. This is what Google is doing.
  - ayaanmomin 3y ago
    "train the query encoder and doc encoder to be closer in their embedding space using click data" <- Any papers/resources you know where I can learn more about this process?
    - janalsncm 3y ago
      Triplet loss. https://pytorch.org/docs/stable/generated/torch.nn.TripletMarginLoss.html https://pytorch.org/docs/stable/generated/torch.nn.TripletMa... Triplet loss takes an anchor, positive, and negative. In this case the anchor is your query, the positive is a similar doc, and the negative is a dissimilar doc. When you train, backpropagate the loss to both the doc and the query encoder.