3 ms·

Thanks... I also have another lingering doubt about the ability of RAG to make sense of "history", i.e. how to make sure that a more recent document on a given

by Pamar 2y ago

Thanks... I also have another lingering doubt about the ability of RAG to make sense of "history", i.e. how to make sure that a more recent document on a given topic has more "weight" than older documents on the same issue. I try to explain this a bit better here: https://pa-mar.net/Study/AiKiDo/VirtualBudoPass.html https://pa-mar.net/Study/AiKiDo/VirtualBudoPass.html

ankit219 2y ago
This is done at a reranking step. It's again custom. You have two variables - 1/ relevance (which most algos focus on) 2/ Date. Create a new score based on some combination of weights for relevance and date. Eg; Could be 50% of date. If the document has 70% relevance, but was published yesterday, it's overall score would be 85%. (A conceptual idea). This is similar to how you do weighted sorting anywhere.
- Pamar 2y ago
  Thank you! Btw, I notice only now that the link that was supposed to explain my question better is completely wrong. https://news.ycombinator.com/item?id=42589014 https://news.ycombinator.com/item?id=42589014 (But you still provided much needed clarification).
  - ankit219 2y ago
    No worries. If you need any specific help with the use case, please feel free to reach out on my email. ankit at clioapp.ai