7 ms·

Maybe good AI requires a combination of symbolic approaches for the knowledge representation, deep parsing for NLP and semantics/discourse models, and machine l

Maybe good AI requires a combination of symbolic approaches for the knowledge representation, deep parsing for NLP and semantics/discourse models, and machine learning for pattern recognition tasks of any kind. The currently trendy "AI" looks more like massive data mining with powerful ML to me, that's very good for certain tasks but brings us nowhere near real AI. The knowledge representation problem has not yet been solved. In order to get even just a convincing simulation of AI, let alone real AI, we need a large common sense knowledge base / computational ontology. One perhaps promising approach are geometric meaning theories/concept representations that allow for logical combinations. For example, Diederik Aerts works in this area. However, to be honest, I don't have the math skills to evaluate his approach. Generally speaking, logical modelling is too limited for good concept representations - especially classical 90s AI like in default reasoning and other nonmonotonic logics -, whereas traditionally geometric representations suffer from problems with representing logical inference and quantification. IMHO, that's a problem worth looking at. (Admittedly, I'm a bit biased towards symbolic AI like the people in this article.)

YeGoblynQueenne 8y ago
>> Generally speaking, logical modelling is too limited for good concept representations In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language. The problem is that in practice it is very hard to transfer into a logic language all the knowledge you might need for a useful system. In fact, this was one motivation for at least one branch of machine learning- the work from Ryszard Michalski, Claude Sammut, Ranan Banerji, Steven Vere, Brian Cohen and later rule-learners like Quinlan's decision tree learners and of course all the later work in Inductive Logic Programming by Plotkin, Shapiro, Muggleton etc. These are all rule-learning systems, that can avoid the knowledge acquisition bottleneck that probably did in for purely rule-based expert systems in the '90s (i.e. getting experts to transfer their knowledge into rules). I rather agree that further progress in AI will require a, let's say, synchretistic approach- like I say in another comment, the obvious thing for me is to use deep learning for perception, logic for inference and probabilistic modelling to deal with the noisy world. There are some people working in that sort of direction, with different amounts of emphasis on each of the three approaches. For instance, Josh Tenenbaum at MIT, Evans and Grefenstette at DeepMind, Luc De Raedt at KU Leuven, Kristian Kersting at Dortmund, Lise Getoor also at MIT, Pedro Domingos at Washington, and many others. Apologies for all the name-dropping without links. Let me know and I can provide them if required.
- TheOtherHobbes 8y ago
  I suspect you may have a hard time representing all the implications, subtexts, and allusions in a play by Shakespeare with First Order Logic. It seems more likely that First Order Logic can be represented by a subset of natural language. Otherwise, I agree. I don't understand why everyone has been concentrating so hard on probabilistic ML when there are many applications where inductive inference is more efficient and powerful.
  - garmaine 8y ago
    That’s not obvious. Do you have any reason for thinking so?
- paganel 8y ago
  > In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language. Is there any proof about an existing non-leaky abstraction between natural language and First Order Logic (or logic in general)? Afaik there is none, but I might me wrong. For example (one amongst thousands): “I don’t like you” can actually mean (and it often does) the exact opposite, and the same can be said of a simple statement like “I am ok” which also can mean the exact opposite. Natural language is a very hard problem.
  - narag 8y ago
    Does it matter, though? Understanding sarcasm or other forms of language twisting seems like only important for a very advanced and specific set of applications. There's another post in the front page about the death of a gorilla that was able to do signs language. I'm sure nobody cared if he could catch jokes.
  - YeGoblynQueenne 8y ago
    Well, going by formal language theory, FOL is equivalent to a universal Turing machine, which is probably also the full expressive power of human language. Therefore, everything that can be said in human language should in principle be possible to say in FOL. I don't think this is controversial. Notice again that I don't think this is feasible in practice. At least not with hand-crafted FOL expressions and certainly not with existing technology. I'm not sure what "non-leaky abstraction" means. Would formal language theory fit the bill?
wslh 8y ago
I think this is the perspective the "Society of the Mind" book by Marvin Minsky took at their time. When a software like Mathematica makes symbolic operations it is doing AI, the same for things like the Z3 solver.
- jryanwelsh 8y ago
  Very astute. Some folks here at Kyndi used to work with Minsky. One of our favorite quotes is: "what magical trick makes us intelligence? The trick is that there is no trick. The power of intelligence stems form our vast diversity, not from any single, perfect principle." What we see now in AI/ML is not diversity, but rather attempts at the single perfect principle. We don't believe that is the path forward. Rather, you need to figure out how to tightly couple different approaches (e.g., symbolic and statistical learning approaches) to form a more powerful, flexible, and explainable model of AI.
bigmanwalter 8y ago
The geometric approach is promising. I am working on a project that implements it. It is still a work in progress but things are moving along quite nicely. You can read more about IEML at https://pierrelevyblog.com/my-research-in-a-nutshell/the-basics-of-ieml/ https://pierrelevyblog.com/my-research-in-a-nutshell/the-bas... The researcher, Prof. Levy, is French :)
sharemywin 8y ago
AGI will need to be able to learn and represent any symbol system, just like we can. And it doesn't have to be that good at it(we arn't). There's millions of knowledge bases all around. You'll need something with a generative imagination. We model so much more than labels and relation ships. Imagine a speech, by president Obama. Where you can actually hear his words. or imagine a ball dropping. We just use words to convey the basic idea. How many hairs does a bald person have? And if we have a misunderstanding we just adjust our system. New word we use context, including social context etc.
tramGG 8y ago
> Maybe good AI requires a combination of symbolic approaches for the knowledge representation, deep parsing for NLP and semantics/discourse models, and machine learning for pattern recognition tasks of any kind. I totally agree.
- jryanwelsh 8y ago
  that's what we (Kyndi) think. Good AI requires ML, NLP, and KRR. ML to acquire the knowledge on which to reason, KRR to represent and reason, and NLP as the glue that holds everything together.
EGreg 8y ago
Let me suggest something that will turn everything upside down... How much of our “logical reasoning” about the world is just really strong correlations? Look, AlphaGo Zero beat the best chess programs by just precomputing MCTS. What does that say about finding the optimal solution to a problem? It seems MCTS beats even AlphaBeta search, by far. So when you talk about applying rules, you’re using a system that tried to reduce the system to a few rules and maybe recursively use them. That may work for some things. Certainly it seems to work amazingly for physics. But that’s the unreasonable effectiveness of mathematics. Ultimately what if I suggested that “understanding something” with huge vectors or monte carlo search is actually far deeper understanding than a few rules? I am talking about the actual territory not the map. Yes the model is neat and tidy but the model is just simple enough so that humans can understand! What if an AI can no more explain its solutions to a human than a human can explain human decisions to a cat?
- randcraw 8y ago
  That's like saying that causation doesn't matter; correlation will suffice. But the causal directionality of separating subject from object in a relation is essential to forming or sharing an idea usefully. In the synthetic game worlds you mention, causality is decided by play sequence. But at no point does the agent ever have to employ abstraction, or 'think about thinking', which diminishes the needed depth of thought greatly. No, probability alone simply isn't expressive enough to support higher cognition. At best it will deliver a purely reactive agent, no more cognizant than Kahnemann's knee-jerk level of thinking.
  - EGreg 8y ago
    Funny that you say that. David Hume said that you can’t meaningfully prove causation, only describe things and speak about about correlation. Also time is a local phenemenon. You can’t simply compare A and B time if A and B are really far apart. When you get to really small scales you would be hard pressed to find which event happened before the other, so you get uncaused events like Virtual Particles in quantum mechanics. And when you have tons of complexity then what do you mean by cause and effect instead of correlation? https://www.quantamagazine.org/omnigenic-model-suggests-that-all-genes-affect-every-complex-trait-20180620/ https://www.quantamagazine.org/omnigenic-model-suggests-that...

YeGoblynQueenne 8y ago

>> Generally speaking, logical modelling is too limited for good concept representations In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language. The problem is that in practice it is very hard to transfer into a logic language all the knowledge you might need for a useful system. In fact, this was one motivation for at least one branch of machine learning- the work from Ryszard Michalski, Claude Sammut, Ranan Banerji, Steven Vere, Brian Cohen and later rule-learners like Quinlan's decision tree learners and of course all the later work in Inductive Logic Programming by Plotkin, Shapiro, Muggleton etc. These are all rule-learning systems, that can avoid the knowledge acquisition bottleneck that probably did in for purely rule-based expert systems in the '90s (i.e. getting experts to transfer their knowledge into rules). I rather agree that further progress in AI will require a, let's say, synchretistic approach- like I say in another comment, the obvious thing for me is to use deep learning for perception, logic for inference and probabilistic modelling to deal with the noisy world. There are some people working in that sort of direction, with different amounts of emphasis on each of the three approaches. For instance, Josh Tenenbaum at MIT, Evans and Grefenstette at DeepMind, Luc De Raedt at KU Leuven, Kristian Kersting at Dortmund, Lise Getoor also at MIT, Pedro Domingos at Washington, and many others. Apologies for all the name-dropping without links. Let me know and I can provide them if required.

TheOtherHobbes 8y ago
I suspect you may have a hard time representing all the implications, subtexts, and allusions in a play by Shakespeare with First Order Logic. It seems more likely that First Order Logic can be represented by a subset of natural language. Otherwise, I agree. I don't understand why everyone has been concentrating so hard on probabilistic ML when there are many applications where inductive inference is more efficient and powerful.
- garmaine 8y ago
  That’s not obvious. Do you have any reason for thinking so?
paganel 8y ago
> In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language. Is there any proof about an existing non-leaky abstraction between natural language and First Order Logic (or logic in general)? Afaik there is none, but I might me wrong. For example (one amongst thousands): “I don’t like you” can actually mean (and it often does) the exact opposite, and the same can be said of a simple statement like “I am ok” which also can mean the exact opposite. Natural language is a very hard problem.
- narag 8y ago
  Does it matter, though? Understanding sarcasm or other forms of language twisting seems like only important for a very advanced and specific set of applications. There's another post in the front page about the death of a gorilla that was able to do signs language. I'm sure nobody cared if he could catch jokes.
- YeGoblynQueenne 8y ago
  Well, going by formal language theory, FOL is equivalent to a universal Turing machine, which is probably also the full expressive power of human language. Therefore, everything that can be said in human language should in principle be possible to say in FOL. I don't think this is controversial. Notice again that I don't think this is feasible in practice. At least not with hand-crafted FOL expressions and certainly not with existing technology. I'm not sure what "non-leaky abstraction" means. Would formal language theory fit the bill?

wslh 8y ago

I think this is the perspective the "Society of the Mind" book by Marvin Minsky took at their time. When a software like Mathematica makes symbolic operations it is doing AI, the same for things like the Z3 solver.

jryanwelsh 8y ago
Very astute. Some folks here at Kyndi used to work with Minsky. One of our favorite quotes is: "what magical trick makes us intelligence? The trick is that there is no trick. The power of intelligence stems form our vast diversity, not from any single, perfect principle." What we see now in AI/ML is not diversity, but rather attempts at the single perfect principle. We don't believe that is the path forward. Rather, you need to figure out how to tightly couple different approaches (e.g., symbolic and statistical learning approaches) to form a more powerful, flexible, and explainable model of AI.

bigmanwalter 8y ago

The geometric approach is promising. I am working on a project that implements it. It is still a work in progress but things are moving along quite nicely. You can read more about IEML at https://pierrelevyblog.com/my-research-in-a-nutshell/the-basics-of-ieml/ https://pierrelevyblog.com/my-research-in-a-nutshell/the-bas... The researcher, Prof. Levy, is French :)

sharemywin 8y ago

AGI will need to be able to learn and represent any symbol system, just like we can. And it doesn't have to be that good at it(we arn't). There's millions of knowledge bases all around. You'll need something with a generative imagination. We model so much more than labels and relation ships. Imagine a speech, by president Obama. Where you can actually hear his words. or imagine a ball dropping. We just use words to convey the basic idea. How many hairs does a bald person have? And if we have a misunderstanding we just adjust our system. New word we use context, including social context etc.

tramGG 8y ago

> Maybe good AI requires a combination of symbolic approaches for the knowledge representation, deep parsing for NLP and semantics/discourse models, and machine learning for pattern recognition tasks of any kind. I totally agree.

jryanwelsh 8y ago
that's what we (Kyndi) think. Good AI requires ML, NLP, and KRR. ML to acquire the knowledge on which to reason, KRR to represent and reason, and NLP as the glue that holds everything together.

EGreg 8y ago

Let me suggest something that will turn everything upside down... How much of our “logical reasoning” about the world is just really strong correlations? Look, AlphaGo Zero beat the best chess programs by just precomputing MCTS. What does that say about finding the optimal solution to a problem? It seems MCTS beats even AlphaBeta search, by far. So when you talk about applying rules, you’re using a system that tried to reduce the system to a few rules and maybe recursively use them. That may work for some things. Certainly it seems to work amazingly for physics. But that’s the unreasonable effectiveness of mathematics. Ultimately what if I suggested that “understanding something” with huge vectors or monte carlo search is actually far deeper understanding than a few rules? I am talking about the actual territory not the map. Yes the model is neat and tidy but the model is just simple enough so that humans can understand! What if an AI can no more explain its solutions to a human than a human can explain human decisions to a cat?

randcraw 8y ago
That's like saying that causation doesn't matter; correlation will suffice. But the causal directionality of separating subject from object in a relation is essential to forming or sharing an idea usefully. In the synthetic game worlds you mention, causality is decided by play sequence. But at no point does the agent ever have to employ abstraction, or 'think about thinking', which diminishes the needed depth of thought greatly. No, probability alone simply isn't expressive enough to support higher cognition. At best it will deliver a purely reactive agent, no more cognizant than Kahnemann's knee-jerk level of thinking.
- EGreg 8y ago
  Funny that you say that. David Hume said that you can’t meaningfully prove causation, only describe things and speak about about correlation. Also time is a local phenemenon. You can’t simply compare A and B time if A and B are really far apart. When you get to really small scales you would be hard pressed to find which event happened before the other, so you get uncaused events like Virtual Particles in quantum mechanics. And when you have tons of complexity then what do you mean by cause and effect instead of correlation? https://www.quantamagazine.org/omnigenic-model-suggests-that-all-genes-affect-every-complex-trait-20180620/ https://www.quantamagazine.org/omnigenic-model-suggests-that...