5 ms·

I seem to have missed the Twitter spat that precipitated this essay, but I don't quite buy the larger argument he's making. We should judge approaches to AI bas

by shafte 8y ago

I seem to have missed the Twitter spat that precipitated this essay, but I don't quite buy the larger argument he's making. We should judge approaches to AI based on their results, not on their conformance to a (vague, incorrect, untested) model of human cognition. Symbolic AI fell out of favor primarily because it was not delivering results in impactful problem areas. Deep learning is currently popular because we are nowhere near the limit of what results it can produce. Can this change? Of course! The history of deep learning itself proves as much. But if you want to genuinely influence the direction of the field, you have to lead by example and produce novel/interesting research results, not by kvetching in The New Yorker that your favorite approach is not getting enough attention.

omeze 8y ago
Its a long article but he’s advocating for a hybrid approach to deal with problems that DNNs encounter today which he believes are fundamental to DNNs (reasoning, causality). An example of such an approach taken with success is given at the end: https://arxiv.org/pdf/1711.04574.pdf https://arxiv.org/pdf/1711.04574.pdf
- backpropaganda 8y ago
  How are you judging success here? The paper has no experiments at all, not even toy.
antidesitter 8y ago
> We should judge approaches to AI based on their results That's why the article discusses examples where currently popular approaches fail. > not on their conformance to a (vague, incorrect, untested) model of human cognition. What model of human cognition is "incorrect"? And is it the one presented by Marcus, or a strawman? > But if you want to genuinely influence the direction of the field, you have to lead by example and produce novel/interesting research results Are you claiming Marcus has produced no interesting research results? > not by kvetching in The New Yorker that your favorite approach is not getting enough attention Why use the term "kvetching"? I'm curious.
- _cs2017_ 8y ago
  > Are you claiming Marcus has produced no interesting research results? Not the OP, but I'm not familiar with Marcus' contributions. What would you consider his top contributions? Are they all purely theoretical, or is there something that's already been applied in the real world?
- shafte 8y ago
  > What model of human cognition are you claiming is "incorrect"? And is it the one presented by Marcus, or are you strawmanning? The model of human cognition I'm referring to is the hybrid connectionist-symbolic one that Marcus is well known for advocating (are YOU strawmanning? lol). I'm criticizing it for being more a theoretical model than one grounded in the physical realities of the brain, which of course no one really understands. Proposing a research program on that basis requires a high burden of proof. > Are you claiming Marcus has produced no interesting research results? Yes I am claiming that, if the benchmark for "interesting" is deep learning. There are indeed areas where deep learning is limited, and hybrid approaches could be superior. I would argue that there is not even close to enough evidence that a hybrid approach has improved generalizable power. > Why use the term "kvetching"? I'm curious. Huh? I guess it's the term my mother would use.
  - eli_gottlieb 8y ago
    >the physical realities of the brain, which of course no one really understands. No-one in machine learning, yeah, because machine learners mostly don't take neuroscience classes ;-).
  - antidesitter 8y ago
    > The model of human cognition I'm referring to is the hybrid connectionist-symbolic one that Marcus is well known for advocating > I'm criticizing it for being more a theoretical model than one grounded in the physical realities of the brain, which of course no one really understands You're contradicting yourself. On the one hand, you claim Marcus' model is "incorrect". On the other, you claim there's insufficient evidence either way. Which is it? > are YOU strawmanning? lol Do you know what the term "strawmanning" means? What could I possibly be strawmanning since I was asking for clarification? > Proposing a research program on that basis requires a high burden of proof. As opposed to...? > Yes I am claiming that, if the benchmark for "interesting" is deep learning. "Deep learning" isn't the correct benchmark since that's what Marcus is critiquing (to some extent) in the first place. > I would argue that there is not even close to enough evidence that a hybrid approach has improved generalizable power. Then you'd be wrong. Here's a good place to start your research: http://science.sciencemag.org/content/331/6022/1279 http://science.sciencemag.org/content/331/6022/1279 > Huh? I guess it's the term my mother would use. Your mother taught you to describe scientific debate as "kvetching"? That's disappointing.
YeGoblynQueenne 8y ago
>> Symbolic AI fell out of favor primarily because it was not delivering results in impactful problem areas. My understanding is instead that symbolic AI was working pretty damn well for its time. Expert systems routinely outperformed experts, for sure. The AI winter that killed them was brought on by political decisions taken by people who didn't really understand the field. Here's a good read on that historical period: Avoiding another AI winter, editorial in IEEE Intelligent Systems. https://www.computer.org/csdl/mags/ex/2008/02/mex2008020002.pdf https://www.computer.org/csdl/mags/ex/2008/02/mex2008020002.... Btw, "Intelligent Systems" is such a funny little expression. Basically, it was used by AI researchers during the 80's AI winter to be able to get funding for their work; because they wouldn't get any if they called it what it was, AI.
dreamcompiler 8y ago
Symbolic AI fell out of favor because it was overhyped. It was delivering quite impressive results--just not the promised results. Neural nets fell out of favor in the 90s for exactly the same reason. Both failures ultimately were caused by not enough computing power. Even though Deep Learning and Convolutional NNs look like major advances today, they never could have been practical before about 2005: There just wasn't enough computing power. If modern computer power were thrown at symbolic AI the same way it's been thrown at NNs, it highly likely symbolic AI would experience similarly-impressive gains.
- hodgesrm 8y ago
  > If modern computer power were thrown at symbolic AI the same way it's been thrown at NNs, it highly likely symbolic AI would experience similarly-impressive gains. What's the basis for this conjecture? Is there a mathematical model for symbolic manipulation that would benefit from parallel execution/GPUs the way ML applications do?
  - dreamcompiler 8y ago
    My conjecture is about e.g. the Rete algorithm for rule search and the likelihood that it could be made more scalable on multicore and distributed hardware with modern functional data structures allowing easy rule updates. I don't know whether rule search or logic unification could be mapped onto GPU or TPU operations; I suspect not, but it's worth looking into.
    - jeremyjh 8y ago
      You realize someone has to write all those rules - and correctly - right? Are you saying that we literally didn't have the computing power to run all the rules we could actually write? I think you need to support this idea that symbolic approaches failed to lack of computing power.
      - YeGoblynQueenne 8y ago
        Like I say in another comment, machine learning took off in part as a way to avoid having to hand-craft rules for expert systems' rule bases (although machine learning existed as a discipline from the early days of AI). So a lot of work on machine learning in the '80s and '90s went to learning rules. For instance (also in another comment) Decision Tree learners basically learn a set of If-Then-Else rules. They're one type of symbolic machine learning and there's more where they came from (e.g. Ross Quinlan's FOIL, for First-Order Inductive Learner, which is basically a first-order version of decision trees; Inductive Logic Programming which I study for my PhD; and many, many more). This work has dwindled, but it's still going. So, no, you don't have to write rules by hand, anymore than you need to set the weights of a neural net by hand. You can just learn them.

by shafte 8y ago

omeze 8y ago

Its a long article but he’s advocating for a hybrid approach to deal with problems that DNNs encounter today which he believes are fundamental to DNNs (reasoning, causality). An example of such an approach taken with success is given at the end: https://arxiv.org/pdf/1711.04574.pdf https://arxiv.org/pdf/1711.04574.pdf

backpropaganda 8y ago
How are you judging success here? The paper has no experiments at all, not even toy.

antidesitter 8y ago

> We should judge approaches to AI based on their results That's why the article discusses examples where currently popular approaches fail. > not on their conformance to a (vague, incorrect, untested) model of human cognition. What model of human cognition is "incorrect"? And is it the one presented by Marcus, or a strawman? > But if you want to genuinely influence the direction of the field, you have to lead by example and produce novel/interesting research results Are you claiming Marcus has produced no interesting research results? > not by kvetching in The New Yorker that your favorite approach is not getting enough attention Why use the term "kvetching"? I'm curious.

_cs2017_ 8y ago
> Are you claiming Marcus has produced no interesting research results? Not the OP, but I'm not familiar with Marcus' contributions. What would you consider his top contributions? Are they all purely theoretical, or is there something that's already been applied in the real world?
shafte 8y ago
> What model of human cognition are you claiming is "incorrect"? And is it the one presented by Marcus, or are you strawmanning? The model of human cognition I'm referring to is the hybrid connectionist-symbolic one that Marcus is well known for advocating (are YOU strawmanning? lol). I'm criticizing it for being more a theoretical model than one grounded in the physical realities of the brain, which of course no one really understands. Proposing a research program on that basis requires a high burden of proof. > Are you claiming Marcus has produced no interesting research results? Yes I am claiming that, if the benchmark for "interesting" is deep learning. There are indeed areas where deep learning is limited, and hybrid approaches could be superior. I would argue that there is not even close to enough evidence that a hybrid approach has improved generalizable power. > Why use the term "kvetching"? I'm curious. Huh? I guess it's the term my mother would use.
- eli_gottlieb 8y ago
  >the physical realities of the brain, which of course no one really understands. No-one in machine learning, yeah, because machine learners mostly don't take neuroscience classes ;-).
- antidesitter 8y ago
  > The model of human cognition I'm referring to is the hybrid connectionist-symbolic one that Marcus is well known for advocating > I'm criticizing it for being more a theoretical model than one grounded in the physical realities of the brain, which of course no one really understands You're contradicting yourself. On the one hand, you claim Marcus' model is "incorrect". On the other, you claim there's insufficient evidence either way. Which is it? > are YOU strawmanning? lol Do you know what the term "strawmanning" means? What could I possibly be strawmanning since I was asking for clarification? > Proposing a research program on that basis requires a high burden of proof. As opposed to...? > Yes I am claiming that, if the benchmark for "interesting" is deep learning. "Deep learning" isn't the correct benchmark since that's what Marcus is critiquing (to some extent) in the first place. > I would argue that there is not even close to enough evidence that a hybrid approach has improved generalizable power. Then you'd be wrong. Here's a good place to start your research: http://science.sciencemag.org/content/331/6022/1279 http://science.sciencemag.org/content/331/6022/1279 > Huh? I guess it's the term my mother would use. Your mother taught you to describe scientific debate as "kvetching"? That's disappointing.

YeGoblynQueenne 8y ago

>> Symbolic AI fell out of favor primarily because it was not delivering results in impactful problem areas. My understanding is instead that symbolic AI was working pretty damn well for its time. Expert systems routinely outperformed experts, for sure. The AI winter that killed them was brought on by political decisions taken by people who didn't really understand the field. Here's a good read on that historical period: Avoiding another AI winter, editorial in IEEE Intelligent Systems. https://www.computer.org/csdl/mags/ex/2008/02/mex2008020002.pdf https://www.computer.org/csdl/mags/ex/2008/02/mex2008020002.... Btw, "Intelligent Systems" is such a funny little expression. Basically, it was used by AI researchers during the 80's AI winter to be able to get funding for their work; because they wouldn't get any if they called it what it was, AI.

dreamcompiler 8y ago

Symbolic AI fell out of favor because it was overhyped. It was delivering quite impressive results--just not the promised results. Neural nets fell out of favor in the 90s for exactly the same reason. Both failures ultimately were caused by not enough computing power. Even though Deep Learning and Convolutional NNs look like major advances today, they never could have been practical before about 2005: There just wasn't enough computing power. If modern computer power were thrown at symbolic AI the same way it's been thrown at NNs, it highly likely symbolic AI would experience similarly-impressive gains.

hodgesrm 8y ago
> If modern computer power were thrown at symbolic AI the same way it's been thrown at NNs, it highly likely symbolic AI would experience similarly-impressive gains. What's the basis for this conjecture? Is there a mathematical model for symbolic manipulation that would benefit from parallel execution/GPUs the way ML applications do?
- dreamcompiler 8y ago
  My conjecture is about e.g. the Rete algorithm for rule search and the likelihood that it could be made more scalable on multicore and distributed hardware with modern functional data structures allowing easy rule updates. I don't know whether rule search or logic unification could be mapped onto GPU or TPU operations; I suspect not, but it's worth looking into.
  - jeremyjh 8y ago
    You realize someone has to write all those rules - and correctly - right? Are you saying that we literally didn't have the computing power to run all the rules we could actually write? I think you need to support this idea that symbolic approaches failed to lack of computing power.
    - YeGoblynQueenne 8y ago
      Like I say in another comment, machine learning took off in part as a way to avoid having to hand-craft rules for expert systems' rule bases (although machine learning existed as a discipline from the early days of AI). So a lot of work on machine learning in the '80s and '90s went to learning rules. For instance (also in another comment) Decision Tree learners basically learn a set of If-Then-Else rules. They're one type of symbolic machine learning and there's more where they came from (e.g. Ross Quinlan's FOIL, for First-Order Inductive Learner, which is basically a first-order version of decision trees; Inductive Logic Programming which I study for my PhD; and many, many more). This work has dwindled, but it's still going. So, no, you don't have to write rules by hand, anymore than you need to set the weights of a neural net by hand. You can just learn them.