3 ms·
I’m really interested in content explaining how to navigate graphs of embedded items for Q/A. Any resources on how to do this or arguments for why it’s a bad ap
by ckrapu 3y ago
I’m really interested in content explaining how to navigate graphs of embedded items for Q/A. Any resources on how to do this or arguments for why it’s a bad approach?
For example, if my top K docs aren’t answering the question but each are linked to neighbors, I’d want to know some folk wisdom or tricks for structuring the neighbor graph to cheaply expand the set of useful results.
- TruthWillHurt 3y agoNot using neighbour, but Autogen has a multi-agent pipeline where if a question is not answered by top K docs, the agent can request "UPDATE CONTEXT" and pull the next set of results. See example 5 here: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat.ipynb https://github.com/microsoft/autogen/blob/main/notebook/agen...
- jondwillis 3y agoYou could in theory create a tool/function like “Context Retrieval”, give it to an Agent, and instruct the Agent to paginate through it as needed. This would add some errors due to LLM usage and latency though. And then of course you would still need to design the graph structure. Maybe neo4j or similar graph dbs would be useful? I have seen a langchain integration for instance: https://python.langchain.com/docs/integrations/providers/neo4j https://python.langchain.com/docs/integrations/providers/neo...
- jarulraj 3y agoWouldn't more semantically related neighbors be retrieved by just increasing K?
- ckrapu 3y agoPotentially, yes! The scenario I am imagining is that context A as context for a query yields results 1, 2, and 3. Sometimes, finding neighbors of 1 (ie not necessarily in the top K w.r.t A) instead of going to results 4, 5, 6 might be better.