4 ms·

That doesn’t really make any sense though. Maybe I’m not fully understanding what you’re saying here? Can you clarify how exactly you would turn a connected gra

That doesn’t really make any sense though. Maybe I’m not fully understanding what you’re saying here? Can you clarify how exactly you would turn a connected graph of data points into something that can learn arbitrary new knowledge and assimilate it into its understanding? Because it seems like currently, this process is an entirely human driven manually done task wherein you have people making the decisions on how to connect and add new relations/knowledge to each other.

YeGoblynQueenne 8y ago
What you suggest, using previously acquired knowledge to learn new concepts, is par for the course in my field of study, Inductive Logic Programming. ILP is a set of algorithms and techniquest that learn logic programs from examples and background knowledge, where the examples and BK are themselves logic programs. Initially the BK comes from some existing source- it can be a hand-crafted database of a few predicates deemed relevant to the learning task or a large, automatically-acquired database mined from some text source, data from the CYC Project of course, etc. In any case, because of the unified representation, learned hypotheses (the "models") can be used immediately as background knowledge to learn new concepts. Edit: I don't know if the Cyc project uses ILP. But what the comment above says is doable.
brundolf 8y ago
In our current NLU experiments we use an off-the-shelf open source parser for the initial parse (Parsey McParseface, from Google). We then take the grammatically-tagged words and our system maps them to concepts it knows about. It theorizes about multiple possible interpretations of the sentence (pronouns in particular), judging their likelihood using real-world knowledge. A Winograd Schema can be solved because Cyc knows things about the concepts; it knows that a dog has four legs, and what it typically weighs, and that it's a mammal (and shares further traits of mammals), and knows that it couldn't interbreed with a cat because they're different species and the act of procreation requires two organisms of the same species. So far this information has been hand-encoded, but this has been going on for decades, and we have a lot now. Going past that: 1) We're able to map outside data from a DB into Cyc's knowledge format, rather than hand-encoding it. This knowledge is inherently not as rich as the rest, but it can obviously be useful anyway. 2) At some point we hope to reach a critical mass of knowledge that will allow Cyc to simply "import" a Wikipedia page by parsing and understanding it. It will interpret a given sentence into its own understanding, then assert it as true and do reasoning based on it down the line.
- lawrenceyan 8y ago
  An expert system will never be able to process or understand new things unless given explicit solutions. So you’re basically saying that it is possible to enumerate and solve for everything possible by manually iterating through every single possible edge case that exists. There’s a reason why Cyc has spent over 30 years and only has been able to get this far. You’re fundamentally limited by human constraints. The only realistic way of achieving a general purpose learning system is by teaching a system how to learn and then letting it figure things out on its own. Patently some method involving reinforcement learning. By the way, it’s not like RL is some newfangled thing. Much of it started during the 80’s, as it concurrently developed with the other purported method of developing intelligence, which was through expert systems. If you're interested, I highly recommend checking out a lecture that Demis Hassabis gives talking exactly about this issue: https://www.youtube.com/watch?v=3N9phq_yZP0 https://www.youtube.com/watch?v=3N9phq_yZP0
  - YeGoblynQueenne 8y ago
    >> An expert system will never be able to process or understand new things unless given explicit solutions. (Not the Cyc person). Your comment is arguing for an end-to-end machine learning (specifically, reinforcement learning) approach. However, modern statistical machine learning systems have demonstrated very clearly that, while they are very good at learning specific and narrow tasks, they are pretty rubbish at multi-task learning and, of course, at reasoning. For breadth of capabilities they are no match to expert systems that can generally both tie their shoelaces and chew gum at the same time. Couple this with the practical limitations of learning all of intelligence end-to-end from examples and it's obvious that statistical machine learning on its own is not going to get any much farther than rule-based systems on their own. Btw, reinforcement learning is rather older than the '80s. Donald Michie (my thesis advisor's thesis advisor) created MENACE, a reinforcement learning algorithm to play noughts-and-crosses in 1961 [1]. Machine learning in general is older still- Arthur Samuel baptised the field in the 1959 [2] but neural networks were first described in 1938 by Pitts and McCulloch [3]. Like Goeff Hinton has said, the current explosion of machine learning applications is due to large datasets and excesses of computing power- not because they're a new idea that people suddendly realised has potential. _______________ [1] https://rodneybrooks.com/forai-machine-learning-explained/ https://rodneybrooks.com/forai-machine-learning-explained/ [2] https://en.wikipedia.org/wiki/Arthur_Samuel https://en.wikipedia.org/wiki/Arthur_Samuel [3] https://en.wikipedia.org/wiki/Artificial_neuron https://en.wikipedia.org/wiki/Artificial_neuron

YeGoblynQueenne 8y ago

What you suggest, using previously acquired knowledge to learn new concepts, is par for the course in my field of study, Inductive Logic Programming. ILP is a set of algorithms and techniquest that learn logic programs from examples and background knowledge, where the examples and BK are themselves logic programs. Initially the BK comes from some existing source- it can be a hand-crafted database of a few predicates deemed relevant to the learning task or a large, automatically-acquired database mined from some text source, data from the CYC Project of course, etc. In any case, because of the unified representation, learned hypotheses (the "models") can be used immediately as background knowledge to learn new concepts. Edit: I don't know if the Cyc project uses ILP. But what the comment above says is doable.

brundolf 8y ago

In our current NLU experiments we use an off-the-shelf open source parser for the initial parse (Parsey McParseface, from Google). We then take the grammatically-tagged words and our system maps them to concepts it knows about. It theorizes about multiple possible interpretations of the sentence (pronouns in particular), judging their likelihood using real-world knowledge. A Winograd Schema can be solved because Cyc knows things about the concepts; it knows that a dog has four legs, and what it typically weighs, and that it's a mammal (and shares further traits of mammals), and knows that it couldn't interbreed with a cat because they're different species and the act of procreation requires two organisms of the same species. So far this information has been hand-encoded, but this has been going on for decades, and we have a lot now. Going past that: 1) We're able to map outside data from a DB into Cyc's knowledge format, rather than hand-encoding it. This knowledge is inherently not as rich as the rest, but it can obviously be useful anyway. 2) At some point we hope to reach a critical mass of knowledge that will allow Cyc to simply "import" a Wikipedia page by parsing and understanding it. It will interpret a given sentence into its own understanding, then assert it as true and do reasoning based on it down the line.

lawrenceyan 8y ago
An expert system will never be able to process or understand new things unless given explicit solutions. So you’re basically saying that it is possible to enumerate and solve for everything possible by manually iterating through every single possible edge case that exists. There’s a reason why Cyc has spent over 30 years and only has been able to get this far. You’re fundamentally limited by human constraints. The only realistic way of achieving a general purpose learning system is by teaching a system how to learn and then letting it figure things out on its own. Patently some method involving reinforcement learning. By the way, it’s not like RL is some newfangled thing. Much of it started during the 80’s, as it concurrently developed with the other purported method of developing intelligence, which was through expert systems. If you're interested, I highly recommend checking out a lecture that Demis Hassabis gives talking exactly about this issue: https://www.youtube.com/watch?v=3N9phq_yZP0 https://www.youtube.com/watch?v=3N9phq_yZP0
- YeGoblynQueenne 8y ago
  >> An expert system will never be able to process or understand new things unless given explicit solutions. (Not the Cyc person). Your comment is arguing for an end-to-end machine learning (specifically, reinforcement learning) approach. However, modern statistical machine learning systems have demonstrated very clearly that, while they are very good at learning specific and narrow tasks, they are pretty rubbish at multi-task learning and, of course, at reasoning. For breadth of capabilities they are no match to expert systems that can generally both tie their shoelaces and chew gum at the same time. Couple this with the practical limitations of learning all of intelligence end-to-end from examples and it's obvious that statistical machine learning on its own is not going to get any much farther than rule-based systems on their own. Btw, reinforcement learning is rather older than the '80s. Donald Michie (my thesis advisor's thesis advisor) created MENACE, a reinforcement learning algorithm to play noughts-and-crosses in 1961 [1]. Machine learning in general is older still- Arthur Samuel baptised the field in the 1959 [2] but neural networks were first described in 1938 by Pitts and McCulloch [3]. Like Goeff Hinton has said, the current explosion of machine learning applications is due to large datasets and excesses of computing power- not because they're a new idea that people suddendly realised has potential. _______________ [1] https://rodneybrooks.com/forai-machine-learning-explained/ https://rodneybrooks.com/forai-machine-learning-explained/ [2] https://en.wikipedia.org/wiki/Arthur_Samuel https://en.wikipedia.org/wiki/Arthur_Samuel [3] https://en.wikipedia.org/wiki/Artificial_neuron https://en.wikipedia.org/wiki/Artificial_neuron