3 ms·

Are there any efforts to combining a knowledge base like Cyc together with LLMs and the like? Something like RAG could benefit I suppose. Have some vector for

Are there any efforts to combining a knowledge base like Cyc together with LLMs and the like? Something like RAG could benefit I suppose. Have some vector for a concept match a KB entry etc, IDK :).

mindcrime 3y ago
Are there any efforts to combining a knowledge base like Cyc together with LLMs and the like? Yes. It's something I've been working on, so there's at least 1 such effort. And I'm reasonably sure there are others. The idea is too obvious for there to not be other people pursuing it.
- ragebol 3y ago
  Too obvious indeed. Can I read anywhere about what you're working on? What other approaches exist?
  - mindcrime 3y ago
    > Can I read anywhere about what you're working on? Not yet. It's still early days. > What other approaches exist? Loosely speaking, I'd say this entire discussion falls into the general rubric of what people are calling "neuro-symbolic AI". Now within that there are a lot of different ways to try and combine different modalities. There are things like DeepProbLog, LogicTensorNetworks, etc. For anybody who wants to learn more, consider starting with: https://en.wikipedia.org/wiki/Neuro-symbolic_AI https://en.wikipedia.org/wiki/Neuro-symbolic_AI and the videos from the previous two "Neurosymbolic Summer School" events: https://neurosymbolic.github.io/nsss2023/ https://neurosymbolic.github.io/nsss2023/ https://www.neurosymbolic.org/summerschool.html https://www.neurosymbolic.org/summerschool.html (2022)
brendonwong 3y ago
This is a significant part of my vision for Web 10! https://www.web10.ai/p/web-10-in-under-10-minutes https://www.web10.ai/p/web-10-in-under-10-minutes One of the immediate things I'm working on is a text to knowledge graph system. Yohei (creator of BabyAGI) is also working on text to knowledge graphs: https://twitter.com/yoheinakajima/status/1769019899245158648 https://twitter.com/yoheinakajima/status/1769019899245158648. LlamaIndex has a basic implementation. This isn't quite connecting the system to an automated reasoner though. There is some research in this area, like: https://news.ycombinator.com/item?id=35735375 https://news.ycombinator.com/item?id=35735375 Cyc + LLMs is vaguely related to more advanced "cognitive architectures" for AI, for instance see the world model in Davidad's architecture, which LLMs can be used to help build: https://www.lesswrong.com/posts/jRf4WENQnhssCb6mJ/davidad-s-bold-plan-for-alignment-an-in-depth-explanation https://www.lesswrong.com/posts/jRf4WENQnhssCb6mJ/davidad-s-...

Are there any efforts to combining a knowledge base like Cyc together with LLMs and the like? Something like RAG could benefit I suppose. Have some vector for a concept match a KB entry etc, IDK :).

mindcrime 3y ago

Are there any efforts to combining a knowledge base like Cyc together with LLMs and the like? Yes. It's something I've been working on, so there's at least 1 such effort. And I'm reasonably sure there are others. The idea is too obvious for there to not be other people pursuing it.

ragebol 3y ago
Too obvious indeed. Can I read anywhere about what you're working on? What other approaches exist?
- mindcrime 3y ago
  > Can I read anywhere about what you're working on? Not yet. It's still early days. > What other approaches exist? Loosely speaking, I'd say this entire discussion falls into the general rubric of what people are calling "neuro-symbolic AI". Now within that there are a lot of different ways to try and combine different modalities. There are things like DeepProbLog, LogicTensorNetworks, etc. For anybody who wants to learn more, consider starting with: https://en.wikipedia.org/wiki/Neuro-symbolic_AI https://en.wikipedia.org/wiki/Neuro-symbolic_AI and the videos from the previous two "Neurosymbolic Summer School" events: https://neurosymbolic.github.io/nsss2023/ https://neurosymbolic.github.io/nsss2023/ https://www.neurosymbolic.org/summerschool.html https://www.neurosymbolic.org/summerschool.html (2022)

brendonwong 3y ago

This is a significant part of my vision for Web 10! https://www.web10.ai/p/web-10-in-under-10-minutes https://www.web10.ai/p/web-10-in-under-10-minutes One of the immediate things I'm working on is a text to knowledge graph system. Yohei (creator of BabyAGI) is also working on text to knowledge graphs: https://twitter.com/yoheinakajima/status/1769019899245158648 https://twitter.com/yoheinakajima/status/1769019899245158648. LlamaIndex has a basic implementation. This isn't quite connecting the system to an automated reasoner though. There is some research in this area, like: https://news.ycombinator.com/item?id=35735375 https://news.ycombinator.com/item?id=35735375 Cyc + LLMs is vaguely related to more advanced "cognitive architectures" for AI, for instance see the world model in Davidad's architecture, which LLMs can be used to help build: https://www.lesswrong.com/posts/jRf4WENQnhssCb6mJ/davidad-s-bold-plan-for-alignment-an-in-depth-explanation https://www.lesswrong.com/posts/jRf4WENQnhssCb6mJ/davidad-s-...