3 ms·

I'm glad to see the design of the knowledge base language is aware of context (here called a domain). This is better than other systems, like what is used to pa

I'm glad to see the design of the knowledge base language is aware of context (here called a domain). This is better than other systems, like what is used to parse / dump data from Wikipedia, where the context is basically "the real world right now". However, it doesn't seem like they wanted systems to be able to use and switch between domains easily, nor be able to backtrack (where you have inferred some knowledge, and you are trying to figure out what domain(s) you should be operating inside). Contextual reasoning is very important for many situations, because time is one of the most important contexts. You can have a fact like "the iPhone is the best selling phone". This fact is true or false based on the context. It is definitely false in the real world year 2006 and before, because the first iPhone was not released yet. While the fact may be true right now in a global context, if we are talking about a specific country or region, it may be false. For example, I would expect that satellite-capable phones may be more popular in a highly remote (non-urban) country or region.

TrueDuality 3y ago
I really want to see an open knowledge base project tackle exactly what you're describing. Cyc has a pretty solid foundation for accomplishing this using what they call microtheories (personally a weird term to me). It's a great step in the right direction but its a closed ecosystem with a huge amount of cognitive overhead to do anything against. A lot of other projects seem to do better with the ingestion and updating of information (albeit with less automated tooling). I've been refreshing myself on ontology design and knowledge representation lately. I'm kind of surprised that is hasn't moved very much in the last two decades. I'm guessing we're about to hit an inflection point there as it becomes a cool topic with the current AI trends.
- ansible 3y ago
  Formal knowledge representation and any sort of formal reasoning is not the way the industry is going right now with LLMs though. It occurs to me that if we want LLMs to be able to reason better, we may need to write texts that explicitly embody the reasoning we desire, and not just use a bunch of books or whatever scraped from the Internet.
  - spothedog1 3y ago
    My personal exploration has me moving in the direction of using LLMs as a Language Model and some sort of structured knowledge representation as a Knowledge Model. So using knowledge graphs to store facts and data and LLMs to interface with it. I wish there was more focus on this, right now I'm just using RDF graphs and SPARQL to try to create my own knowledge engine.
  - TrueDuality 3y ago
    While I agree with your statement I would add "yet" to the end of it. Language models are powerful tools on their own, but we're already augmenting their knowledge with external data stores. Personally I think the next stage is going to blend in knowledge graphs. Populating these Ontologies is very manual and time-consuming right now. LLMs without any additional training (I'm currently using a mix of the various size Lllama2 models, along with GPT 3.5 and 4 for this) are capable of few-shot generation of ontological classification. Extending this classification using fine-tuning is doing REALLY well. I'm also seeing a lot of value in using LLMs to query and interpret proofs from deductive reasoners against these knowledge graphs. I have been limiting the scope of my research around this to two domains that have a kind of eccentric mix of formal practices, explicit "correct" knowledge, and common sense rules of thumb to be successful. Queries can be quite onerous to build which a fine-tuned model can help with, and LLMs can both assist in interpreting those logic chains and doing knowledge maintenance to add in the missing common sense rules or remove bad or outdated rules. Even selecting among possible solutions produced by the reasoners is really solid when you include the task, desires, and constraints of what you're trying to accomplish in the prompt performing the selection. The formal process and knowledge is handled very well by knowledge graphs along with a deductive reasoning engine, but they produce very long winded logic chains to reach a positive or negative conclusion where a simpler chain might have sufficed (usually due to missing rules, or a lack of common sense rules) and are generally incapable of "leaps" in deduction. LLMs on their own are capable of (currently largely low-level) common sense reasoning, and some formal reasoning but are still very prone to hallucinations. A 20% failure rate when building rules that human lives may depend on is a non-starter. This will improve but I don't think a probabilistic approach will ever fully remove them. We can use all of our tools together in various blends to augment and verify knowledge, fully automatically, to make more capable systems.

TrueDuality 3y ago

I really want to see an open knowledge base project tackle exactly what you're describing. Cyc has a pretty solid foundation for accomplishing this using what they call microtheories (personally a weird term to me). It's a great step in the right direction but its a closed ecosystem with a huge amount of cognitive overhead to do anything against. A lot of other projects seem to do better with the ingestion and updating of information (albeit with less automated tooling). I've been refreshing myself on ontology design and knowledge representation lately. I'm kind of surprised that is hasn't moved very much in the last two decades. I'm guessing we're about to hit an inflection point there as it becomes a cool topic with the current AI trends.

ansible 3y ago
Formal knowledge representation and any sort of formal reasoning is not the way the industry is going right now with LLMs though. It occurs to me that if we want LLMs to be able to reason better, we may need to write texts that explicitly embody the reasoning we desire, and not just use a bunch of books or whatever scraped from the Internet.
- spothedog1 3y ago
  My personal exploration has me moving in the direction of using LLMs as a Language Model and some sort of structured knowledge representation as a Knowledge Model. So using knowledge graphs to store facts and data and LLMs to interface with it. I wish there was more focus on this, right now I'm just using RDF graphs and SPARQL to try to create my own knowledge engine.
- TrueDuality 3y ago
  While I agree with your statement I would add "yet" to the end of it. Language models are powerful tools on their own, but we're already augmenting their knowledge with external data stores. Personally I think the next stage is going to blend in knowledge graphs. Populating these Ontologies is very manual and time-consuming right now. LLMs without any additional training (I'm currently using a mix of the various size Lllama2 models, along with GPT 3.5 and 4 for this) are capable of few-shot generation of ontological classification. Extending this classification using fine-tuning is doing REALLY well. I'm also seeing a lot of value in using LLMs to query and interpret proofs from deductive reasoners against these knowledge graphs. I have been limiting the scope of my research around this to two domains that have a kind of eccentric mix of formal practices, explicit "correct" knowledge, and common sense rules of thumb to be successful. Queries can be quite onerous to build which a fine-tuned model can help with, and LLMs can both assist in interpreting those logic chains and doing knowledge maintenance to add in the missing common sense rules or remove bad or outdated rules. Even selecting among possible solutions produced by the reasoners is really solid when you include the task, desires, and constraints of what you're trying to accomplish in the prompt performing the selection. The formal process and knowledge is handled very well by knowledge graphs along with a deductive reasoning engine, but they produce very long winded logic chains to reach a positive or negative conclusion where a simpler chain might have sufficed (usually due to missing rules, or a lack of common sense rules) and are generally incapable of "leaps" in deduction. LLMs on their own are capable of (currently largely low-level) common sense reasoning, and some formal reasoning but are still very prone to hallucinations. A 20% failure rate when building rules that human lives may depend on is a non-starter. This will improve but I don't think a probabilistic approach will ever fully remove them. We can use all of our tools together in various blends to augment and verify knowledge, fully automatically, to make more capable systems.