3 ms·
RAG will have a place in the LLM world, since it's a way to obtain data/facts/info for relevant queries. Since you asked about alternatives... (a) "World mode
by cl42 2y ago
RAG will have a place in the LLM world, since it's a way to obtain data/facts/info for relevant queries.
Since you asked about alternatives...
(a) "World models" where LLMs structure information into code, structured data, etc. and query those models will likely be a thing. AlphaGeometry uses this[1], and people have tried to abstract this in different ways[2].
(b) Depending on how you define RAG, knowledge graphs could be a form of RAG or alternatively an alternative to them. Companies like Elemental Cognition[3] are building distinct alternatives to RAG that use such graphs and give LLMs the ability to run queries on said graphs. Another approach here is to build "fact databases" where, you structure observations about the world into standalone concepts/ideas/observations and reference those[4]. Again, similar to RAG but not quite RAG as we know it today.
[1] https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/ https://deepmind.google/discover/blog/alphageometry-an-olymp...
[2] https://arxiv.org/abs/2306.12672 https://arxiv.org/abs/2306.12672
[3] https://ec.ai/ https://ec.ai/
[4] https://emergingtrajectories.com/ https://emergingtrajectories.com/
- grugagag 2y agoI hope it goes away because I have a real carpet to replace it with
- 0x008 2y agoI don't understand why knowledge graph would be an alternative to RAG? Knowledge graphs can (and are already) used as part of a RAG pipeline.
- ac2u 2y agoNot sure either... unless it means a workflow where instead of chunking your knowledgebase docs and generating embeddings, you do entity and relationship extractions on them instead and store an index of knowledge graph serialisations to source documents.
- kordlessagain 2y agoYou can do set overlap on the extracted terms and that gives similar relevancy to the chunks as done by their embedding vectors. It’s just a lower dimensionally of the relationships between the chunk and the query.