7 ms·

And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more pow

And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants). Such architecture works great for differentiable data, such's images/audios, but the improvement on natural language tasks are only incremental. I was thinking maybe DeepMind's RL+DL is the way leads to AGI, since it does offer an elegant and complete framework. But seems like even DeepMind had trouble to get it working to more realistic scenarios, so maybe our modelling of intelligence is still hopelessly romantic.

skybrian 8y ago
Hmm, it seems like natural language translation has been getting quite a bit better with statistical techniques, though? I guess it depends what you mean by "only incremental".
- harperlee 8y ago
  Statistically, but it is a shallow translation, with no modelling of what is said. This works astonishly well for translation, but gives the false intuition that it is meaningful; whereas it is in fact orthogonal to advancing in systems that comprehend a text so as to be able to reason their way out of winograd problems.
- tanilama 8y ago
  I still remember sometime back Google's GNMT translate the Chinese text of 'I don't want to go to work' into English 'I want to go to work'. That example alone should be sufficient to showcase how most advanced machine learning model can fail at the simplest task. It didn't understand the source material, it is just very good at memorizing and faking.
- m0zg 8y ago
  It's been "statistical" since about mid-00's. What's new is that it's now neural.
- joe_the_user 8y ago
  Whenever I attempt to use a translate site to translate more than a paragraph (Facebook or Google), it comes out a garbled mess - doesn't mean some sentences are seemingly clear and meaningful. The big thing is it chokes on idioms, not understanding, not leaving them as is but guessing some clearly wrong meaning. I occasionally find single-sentence posts on Facebook apparently translated astonishingly well, in the sense of being literate English and seeming to reflect the original meaning. But my French or Spanish is quite rough so translate could have missed something big - as I know it does when you get into longer texts.
  - narag 8y ago
    Cada vez que intento usar un sitio de traducción para traducir más de un párrafo (Facebook o Google), sale un lío confuso - no significa que algunas oraciones sean aparentemente claras y significativas. Lo importante es que se atraganta con los modismos, no entendiéndolos, no dejándolos como están, sino adivinando algún significado claramente erróneo. Ocasionalmente encuentro que las publicaciones de una sola oración en Facebook aparentemente traducen sorprendentemente bien, en el sentido de que son en inglés y parecen reflejar el significado original. Pero mi francés o español es bastante duro, así que traducir podría haber pasado por alto algo grande, como sé que sucede cuando se trata de textos más largos. I'm a native and it seems almost good to me. "algunas oraciones no sean", OK. And "duro" should be "rudimentario", also "de qué son" lacks the accent. But the rest is acceptable and it's possible to get a decent translation, only modifying those bits. http://www.DeepL.com/Translator http://www.DeepL.com/Translator
    - joe_the_user 8y ago
      Oh, that just means I happen to write in clear, unidiomatic English ;-). Add just a smidgen of irregular usage, contemporary metaphors and such and things can go South pretty quickly.
      - barrkel 8y ago
        That'll just be a training problem (often translation is driven from example texts that have lots of translations, and we have many of these for multi-national orgs like EU, which necessarily don't include a lot of colloquialisms), and the inconsistency isn't found because the models built don't extend all the way out to real world experiences (training) and recognizing text as a real world experience narrative. I think a much deeper network could do better but we don't know how to train them, we take way too long as it is.
odyssey7 8y ago
Are there ways that an AI practitioner would be able to tell whether a neural network is doing human-style symbolic reasoning?
- karmasimida 8y ago
  Good question. I don't think we do actually. The only reason I am convinced it is NOT doing a good job, is how utterly difficult to apply NN to dialog generation/management domain of business, often time it behaves much worse than rule-based systems.
- bsaul 8y ago
  As mentioned in the original article, being able to reuse part of a network trained on one task, on another different task that shares a subset of concepts, would indicate something like the understanding of a concept has emerged.
- darkmighty 8y ago
  You'd be able to tell if every brain structure was replicated by a NN analogue (and we understood them sufficiently well). Otherwise you can only use behavioral replication (i.e. Turing tests) to infer it.
- antidesitter 8y ago
  IQ tests [1] and one-shot learning come to mind. [1] https://arxiv.org/abs/1807.04225 https://arxiv.org/abs/1807.04225
sdenton4 8y ago
Maaaaybe. I tend to think that symbolic reasoning is a learning tool, rather than a goalpost for general intelligence. For example, we use symbolic reasoning quite extensively when learning to read a new language, but once fluent can rely on something closer to raw processing - no more reading and sounding out character sequences. Similarly with chess - eventually we have good mnemonics for what make good plays, and can play blitz reasonably well. And - let's be real - a lot of human symbolic reasoning actually happens outside of the brain, on paper or computer screens. We painstakingly learn relatively simple transformations and feedback loops for manipulating this external memory, and then bootstrap it into short-term reaction via lots of practice. I tend to think that the problems are: a) Tightly defined / domain-specific loss functions. If all I ever do is ask you to identify pictures of bananas, you'll never get around to writing the great american novel. And we don't know how to train the kinds of adaptive or free form loss functions that would get us away from these domain-specific losses. b) Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs. We currently mostly build models that are only receptive (image, sound) or generative. Reinforcement learning is getting progress on feedback loops, but I have the sense that there's still a long way to go. c) I have the feeling that there's still a long way to go in understanding how to deal with time... d) As great as LSTMs are, there still seems to be some shortcoming in how to incorporate memory into networks. LSTMs seem to give a decent approximation of short-term memory, but still seems far from great. This might be the key to symbolic reasoning, though. Writing all that down, I gotta say I agree fundamentally with the DeepMind research priorities on reinforcement learning and multi-modal models.
- joe_the_user 8y ago
  Once someone is fluent in a language, the logical operations and judgements involved stop being overt and highly visible to the conscious mind. But that doesn't mean that one stops getting the benefits and results of logical operations. What you might see as logical operations "not mattering", I would see as logical operations integrated so deeply into reflexive operations that it's hard to see where one ends and the other begins. The contrast is that humans can do pattern recognition in a neural net fashion, taking something like the multidimensional average of a set of things. But a human can also receive a language-level input that some characteristic is or isn't important for recognizing a given thing and incorporate that input into their broad-average concepts. That kind of thing can't be done by deep learning currently - well, not a non-kludgey sort of way. Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs. It depends on how you want to mean that. A human can take inputs on one thing and apply them seamlessly to another thing. Neural nets tend to be very dependent on the task-focused content fed them.
kahoon 8y ago
I don't understand this fixation on symbolic reasoning. Do any other animals practice this? If the answer is no, then it is probably not the most important milestone to AGI or at least not the one we should be currently aiming for. Right now we can not replicate the cognition of a mouse. Feels like we want to go to Mars before figuring out how to build a rocket.
- evrydayhustling 8y ago
  Seconded. Even if animals do symbolic reasoning, they do it on top of hardware based on continuous physical dynamics, more similar to DNNs... So why not build on that platform? I don't think biological precedent is the only or even most valuable heuristic for deciding where to research intelligence... But I don't see where there is evidence that symbolic reasoning is either necessary or sufficient for AGI, except people describing how they think their brain works. Related, there are a lot of statements that symbolic or rule based systems do better / as well as / almost as well as neural methods. Citation please, I'd love a map of which ML problems are still best solved with symbolic systems. (Sincerely - it's not that I expect there aren't any.)
  - kahoon 8y ago
    > I don't think biological precedent is the only or even most valuable heuristic for deciding where to research intelligence... Good point, we wouldn't have AlphaZero now if we only relied on biological inspiration. Nature hardly ever performs Monte Carlo Tree Search (though I'm not sure this is entirely true, see slime mold searching for food: https://thumbs.gfycat.com/IdealisticThirdCalf-size_restricted.gif https://thumbs.gfycat.com/IdealisticThirdCalf-size_restricte...).
    - visarga 8y ago
      We're also good at trying out various ideas until one sticks. Isn't that MC tree search?
- Gibbon1 8y ago
  I think the the fixation on symbolic reasoning comes from ignorance at how hard classification is vs how hard pure mechanical symbolic operations are for humans. It's easy to make the mistake of thinking that since a computer can rapidly multiply two numbers together (hard for humans) that they were operating at a higher level than human brains. Turns out this is wrong. Human brains are very efficient.
darawk 8y ago
> And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants). You think symbolic reasoning is not a function? In what sense do you think 'symbolic reasoning' is a distinct thing from 'function approximation'?
Symmetry 8y ago
Something very interesting to me about the work Deep Mind has been doing is the way they've been combining neural network intuitions with tree search reasoning in Go, Chess, protein folding, etc.
cantthinkofone 8y ago
If humanlike reasoning is the destination for AGI, there's more than just symbolic reasoning to factor in. Emotions are a huge control on human reasoning. People essentially rely on emotions to make all their decisions. Emotions implicitly represent rapid-fire unconscious decision work. Again the current popular understanding of the mind separates emotion from thinking. They are not distinct. Emotional processing is another kind of thinking, and it drives the show.
- visarga 8y ago
  I see emotions as analogous to the value function in RL. It is essentially a prediction of future rewards based on current state and action plan. Artificial RL agents learn emotion as it is related to their tasks and environments.

skybrian 8y ago

Hmm, it seems like natural language translation has been getting quite a bit better with statistical techniques, though? I guess it depends what you mean by "only incremental".

harperlee 8y ago
Statistically, but it is a shallow translation, with no modelling of what is said. This works astonishly well for translation, but gives the false intuition that it is meaningful; whereas it is in fact orthogonal to advancing in systems that comprehend a text so as to be able to reason their way out of winograd problems.
tanilama 8y ago
I still remember sometime back Google's GNMT translate the Chinese text of 'I don't want to go to work' into English 'I want to go to work'. That example alone should be sufficient to showcase how most advanced machine learning model can fail at the simplest task. It didn't understand the source material, it is just very good at memorizing and faking.
m0zg 8y ago
It's been "statistical" since about mid-00's. What's new is that it's now neural.
joe_the_user 8y ago
Whenever I attempt to use a translate site to translate more than a paragraph (Facebook or Google), it comes out a garbled mess - doesn't mean some sentences are seemingly clear and meaningful. The big thing is it chokes on idioms, not understanding, not leaving them as is but guessing some clearly wrong meaning. I occasionally find single-sentence posts on Facebook apparently translated astonishingly well, in the sense of being literate English and seeming to reflect the original meaning. But my French or Spanish is quite rough so translate could have missed something big - as I know it does when you get into longer texts.
- narag 8y ago
  Cada vez que intento usar un sitio de traducción para traducir más de un párrafo (Facebook o Google), sale un lío confuso - no significa que algunas oraciones sean aparentemente claras y significativas. Lo importante es que se atraganta con los modismos, no entendiéndolos, no dejándolos como están, sino adivinando algún significado claramente erróneo. Ocasionalmente encuentro que las publicaciones de una sola oración en Facebook aparentemente traducen sorprendentemente bien, en el sentido de que son en inglés y parecen reflejar el significado original. Pero mi francés o español es bastante duro, así que traducir podría haber pasado por alto algo grande, como sé que sucede cuando se trata de textos más largos. I'm a native and it seems almost good to me. "algunas oraciones no sean", OK. And "duro" should be "rudimentario", also "de qué son" lacks the accent. But the rest is acceptable and it's possible to get a decent translation, only modifying those bits. http://www.DeepL.com/Translator http://www.DeepL.com/Translator
  - joe_the_user 8y ago
    Oh, that just means I happen to write in clear, unidiomatic English ;-). Add just a smidgen of irregular usage, contemporary metaphors and such and things can go South pretty quickly.
    - barrkel 8y ago
      That'll just be a training problem (often translation is driven from example texts that have lots of translations, and we have many of these for multi-national orgs like EU, which necessarily don't include a lot of colloquialisms), and the inconsistency isn't found because the models built don't extend all the way out to real world experiences (training) and recognizing text as a real world experience narrative. I think a much deeper network could do better but we don't know how to train them, we take way too long as it is.

odyssey7 8y ago

Are there ways that an AI practitioner would be able to tell whether a neural network is doing human-style symbolic reasoning?

karmasimida 8y ago
Good question. I don't think we do actually. The only reason I am convinced it is NOT doing a good job, is how utterly difficult to apply NN to dialog generation/management domain of business, often time it behaves much worse than rule-based systems.
bsaul 8y ago
As mentioned in the original article, being able to reuse part of a network trained on one task, on another different task that shares a subset of concepts, would indicate something like the understanding of a concept has emerged.
darkmighty 8y ago
You'd be able to tell if every brain structure was replicated by a NN analogue (and we understood them sufficiently well). Otherwise you can only use behavioral replication (i.e. Turing tests) to infer it.
antidesitter 8y ago
IQ tests [1] and one-shot learning come to mind. [1] https://arxiv.org/abs/1807.04225 https://arxiv.org/abs/1807.04225

sdenton4 8y ago

Maaaaybe. I tend to think that symbolic reasoning is a learning tool, rather than a goalpost for general intelligence. For example, we use symbolic reasoning quite extensively when learning to read a new language, but once fluent can rely on something closer to raw processing - no more reading and sounding out character sequences. Similarly with chess - eventually we have good mnemonics for what make good plays, and can play blitz reasonably well. And - let's be real - a lot of human symbolic reasoning actually happens outside of the brain, on paper or computer screens. We painstakingly learn relatively simple transformations and feedback loops for manipulating this external memory, and then bootstrap it into short-term reaction via lots of practice. I tend to think that the problems are: a) Tightly defined / domain-specific loss functions. If all I ever do is ask you to identify pictures of bananas, you'll never get around to writing the great american novel. And we don't know how to train the kinds of adaptive or free form loss functions that would get us away from these domain-specific losses. b) Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs. We currently mostly build models that are only receptive (image, sound) or generative. Reinforcement learning is getting progress on feedback loops, but I have the sense that there's still a long way to go. c) I have the feeling that there's still a long way to go in understanding how to deal with time... d) As great as LSTMs are, there still seems to be some shortcoming in how to incorporate memory into networks. LSTMs seem to give a decent approximation of short-term memory, but still seems far from great. This might be the key to symbolic reasoning, though. Writing all that down, I gotta say I agree fundamentally with the DeepMind research priorities on reinforcement learning and multi-modal models.

joe_the_user 8y ago
Once someone is fluent in a language, the logical operations and judgements involved stop being overt and highly visible to the conscious mind. But that doesn't mean that one stops getting the benefits and results of logical operations. What you might see as logical operations "not mattering", I would see as logical operations integrated so deeply into reflexive operations that it's hard to see where one ends and the other begins. The contrast is that humans can do pattern recognition in a neural net fashion, taking something like the multidimensional average of a set of things. But a human can also receive a language-level input that some characteristic is or isn't important for recognizing a given thing and incorporate that input into their broad-average concepts. That kind of thing can't be done by deep learning currently - well, not a non-kludgey sort of way. Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs. It depends on how you want to mean that. A human can take inputs on one thing and apply them seamlessly to another thing. Neural nets tend to be very dependent on the task-focused content fed them.

kahoon 8y ago

I don't understand this fixation on symbolic reasoning. Do any other animals practice this? If the answer is no, then it is probably not the most important milestone to AGI or at least not the one we should be currently aiming for. Right now we can not replicate the cognition of a mouse. Feels like we want to go to Mars before figuring out how to build a rocket.

evrydayhustling 8y ago
Seconded. Even if animals do symbolic reasoning, they do it on top of hardware based on continuous physical dynamics, more similar to DNNs... So why not build on that platform? I don't think biological precedent is the only or even most valuable heuristic for deciding where to research intelligence... But I don't see where there is evidence that symbolic reasoning is either necessary or sufficient for AGI, except people describing how they think their brain works. Related, there are a lot of statements that symbolic or rule based systems do better / as well as / almost as well as neural methods. Citation please, I'd love a map of which ML problems are still best solved with symbolic systems. (Sincerely - it's not that I expect there aren't any.)
- kahoon 8y ago
  > I don't think biological precedent is the only or even most valuable heuristic for deciding where to research intelligence... Good point, we wouldn't have AlphaZero now if we only relied on biological inspiration. Nature hardly ever performs Monte Carlo Tree Search (though I'm not sure this is entirely true, see slime mold searching for food: https://thumbs.gfycat.com/IdealisticThirdCalf-size_restricted.gif https://thumbs.gfycat.com/IdealisticThirdCalf-size_restricte...).
  - visarga 8y ago
    We're also good at trying out various ideas until one sticks. Isn't that MC tree search?
Gibbon1 8y ago
I think the the fixation on symbolic reasoning comes from ignorance at how hard classification is vs how hard pure mechanical symbolic operations are for humans. It's easy to make the mistake of thinking that since a computer can rapidly multiply two numbers together (hard for humans) that they were operating at a higher level than human brains. Turns out this is wrong. Human brains are very efficient.

darawk 8y ago

> And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants). You think symbolic reasoning is not a function? In what sense do you think 'symbolic reasoning' is a distinct thing from 'function approximation'?

Symmetry 8y ago

Something very interesting to me about the work Deep Mind has been doing is the way they've been combining neural network intuitions with tree search reasoning in Go, Chess, protein folding, etc.

cantthinkofone 8y ago

If humanlike reasoning is the destination for AGI, there's more than just symbolic reasoning to factor in. Emotions are a huge control on human reasoning. People essentially rely on emotions to make all their decisions. Emotions implicitly represent rapid-fire unconscious decision work. Again the current popular understanding of the mind separates emotion from thinking. They are not distinct. Emotional processing is another kind of thinking, and it drives the show.

visarga 8y ago
I see emotions as analogous to the value function in RL. It is essentially a prediction of future rewards based on current state and action plan. Artificial RL agents learn emotion as it is related to their tasks and environments.