13 ms·

Gary Marcus hasn't changed his tune since 2016. Yet a bunch of progress has been made since then! Is DL a reasonable path to strong AI? Probably not. DeepMind

Gary Marcus hasn't changed his tune since 2016. Yet a bunch of progress has been made since then! Is DL a reasonable path to strong AI? Probably not. DeepMind is making cool stuff in game playing, but it's still "dumb" AI. But Marcus has cornered himself as a professional skeptic of deep learning. A voice which is useful only at times.

9wzYQbTYsAIc 5y ago
> Is DL a reasonable path to strong AI? Arguably, it is the only path to strong AI (if you believe that it is even possible).
- karpierz 5y ago
  Of course strong AI is possible; most people can make one with 18 years and a willing partner. It just doesn't scale.
  - spyder 5y ago
    That's Natural Intelligence not Artificial Intelligence.
    - okaram 5y ago
      Well ... Artificial means created by humans, right? ;)
    - 9wzYQbTYsAIc 5y ago
      Naturally, but we’re going for artificial humor today.
    - deleted 5y ago
      [deleted]
  - gcheong 5y ago
    Also very nondeterministic
    - 9wzYQbTYsAIc 5y ago
      It’s a feature, not a bug.
  - 9wzYQbTYsAIc 5y ago
    > most people can make one with 18 years and a willing partner Well, there’s your scaling problem. It should only take an instant to create a Strong AI (if you believe that Strong AI can be created). What’s Weak about a Strong AI that takes a couple of years to make but pees on the floor all the time?
  - harry8 5y ago
    > It just doesn't scale. On what planet? I can give you about 7 billion reasons why it isn't this one.
    - samplatt 5y ago
      That's just 7 billion reasons why it can't (or at least, doesn't) scale larger than ~3lbs.
    - salawat 5y ago
      It doesn't scale in the "I want twice as many, right now, to use, abuse, and discard without guilt or strings attached". 9 women do not a baby in 1 month make. Once you have said bab(y|ies), you then have the inconvenience of them having a will of their own, you have to support and eventually pay them, and dear god, make too many of them and they might even just throw you away instead of going in the direction you want. That kind of not scaling. I mean... If no one else is willing to say it, I'll call a spade a spade; because that is precisely the major draw of the computer as ecpnomic/logistical lubricant. Doing jobs of people, but not getting paid or having to be accounted for as one.
      - izzygonzalez 5y ago
        If we manage to create an artificial sentience then we need to legislate and protect their rights. AGI won't be a free lunch.
        drdeca 5y ago
        Just because something is AGI probably needn't imply that it is sapient. It might have no internal experience, no feeling of desire, only a utility function which it optimizes.
        krageon 5y ago
        This reasoning is why it will need rights. If it doesn't, people thinking these thoughts (which are impossible to answer, like they are for every other potential sapience. Not just AGI) will abuse that reasoning to create yet another creative modern form of slavery.
  - mannykannot 5y ago
    That's the last step in a training process that took a half-billion years, if we count from the first neurons (this is not, of course, an argument against the feasibility of strong AI.)
    - 9wzYQbTYsAIc 5y ago
      One small step for bosons, one giant leap for consciousness.
    - Planktonne 5y ago
      Every DL system ever made is also the last step in a training process that took half a billion years. It's not reasonable to discount the training of parents/predecessors only when we're talking about machines; they still would not exist without them.
      - mannykannot 5y ago
        There is a slight ambiguity in 'last', which might mean either 'most recent' or 'final'. The difference here is that while karpierz's last step is, at least for now, the final one in a process producing a strong, generalized, self-awarely-conscious theory-of-mind-holding intelligent agent, the evolution of DL systems has not yet done so.
  - auntienomen 5y ago
    It can be partly parallelized, by staggering.
  - jahewson 5y ago
    It’s embarrassingly parallel.
- dkislyuk 5y ago
  What is the argument for it being the only path? We have a pretty poor understanding of how the human brain (our only "Strong AI" evidence) learns, but the one approach that has been pretty definitively ruled out is large-scale backpropagation in the style that most deep learning pursues today. Maybe backprop is a true alternative approach to AGI, but it's hard to be completely convinced of this.
  - 9wzYQbTYsAIc 5y ago
    Have you read Parallel Distributed Processing by McClelland and Rumelhart? I haven’t finished reading it, but they make the case that if humans are conscious, then PDP is the way Strong AI will be figured out. https://mitpress.mit.edu/books/parallel-distributed-processing-volume-1 https://mitpress.mit.edu/books/parallel-distributed-processi...
    - garymarcus 5y ago
      I link it to it in the Nautilus article at the top of thread, and to my thesis work which challenged PDP as a model of children’s learning of the English past tense; more generally the Nautilus article is in some sense my answer to that book.
      - 9wzYQbTYsAIc 5y ago
        Just finished reading the article, interesting points. I agree that there’s too much emphasis on the “robot”, but coming from an academic background in mathematics and psychology (and biology), I cannot take PDP as anything other than “this is how brains work”. Granted, half of PDP, the book, is a series of examples of “ways it might work” from 40 years ago or so, but the other half is about fundamentally tying the psychology to the brain through parallel distributed processing as a conceptual framework. In that sense, I don’t think PDP can be refuted any more than token economies or political polling can be refuted. Thoughts? In the meantime, I will bookmark your thesis for perusal.
giraffe_lady 5y ago
As I'm reading it, the argument isn't that progress can't be made or cool things can't be done with these tools. It's that for certain problems no amount of progress will be sufficient, yet we have entire domains of the tech and consumer products industries built around and selling the idea that just a little bit more will get us there. The argument is compelling to me and fits my experiences. I think the limitations of these tools has become more apparent to non-specialists the last couple years, so if he's been saying this since 2016 that's honestly more impressive. What do you expect him to change his tune to if he's already singing a good one?
- 9wzYQbTYsAIc 5y ago
  > It's that for certain problems no amount of progress will be sufficient In mathematics, you solve problems from both directions, at times. It seems that computer science has come full circle with psychology, finally. Dig into computational comparative psychology for leads on where deep learning experts need to turn to next. There are plenty of computational neuroscientists that are probably struggling with untwining what they are looking at.
idealmedtech 5y ago
I think he does offer some compelling arguments in this article, worth at least examining rather than dismissing based on his past skepticism. Particularly about the potential harmony between deep learning and symbolic manipulation, which is enormously difficult (a far cry from the set-and-forget style that has taken DL so far), but perhaps cracking AGI will require us to do a little more difficult work. I'd love to be proved wrong here, though! My depth of understanding on DL is pretty shallow.
- garymarcus 5y ago
  exactly - the article is a call to arms to do that difficult work.
  - 9wzYQbTYsAIc 5y ago
    Why not return to the roots of Parallel Distributed Processing?
  - p1esk 5y ago
    Lead by example. Instead of criticizing others.
- zmgsabst 5y ago
- naikrovek 5y ago
  We are a very long way from even having 0.0001% of the compute required to produce a weak AGI. like everything, it will take much more resources and time than we can predict today with our best estimates, partly because that's just how these things turn out, and mostly because a true AGI will likely require billions of neural networks all adapting and swapping neurons and communications pathways amongst themselves and training themselves at the same time they are doing useful work. we have no idea how to do anything like this at scale, yet.
  - p1esk 5y ago
29athrowaway 5y ago
DeepMind managed to rekt the best players in Go and StarCraft after a few years of founding. Go has been played for 3000 years, and it has influenced military strategy since ancient times. The strategies you can find in Sun Tzu's Art of War were influenced by the game of Go. Did you know what is the difference between the military simulators used by defense departments and strategy games like Go? Did you know both share a lot in common? What you dismiss as "game playing" has more present applications than you imagine and will make the difference between life and death in armed conflicts. The orders given by generals will likely come from a successor of MuZero. And that may expand to every key decision making process, like the equipment to be manufactured, logistics, etc. Combine that with the power asymmetry coming from drones and that "game playing" will become the most dominant force in this world.
- 9wzYQbTYsAIc 5y ago
  Has anyone pit DeepMind against a constraint satisfaction algorithm in StarCraft? I’ve heard constraint satisfaction can be good enough to simulate playing StarCraft.
  - 29athrowaway 5y ago
    StarCraft is not a complete information game. There's a fog of war. There's much that can be done to trick the opponent into making incorrect assumptions.
    - 9wzYQbTYsAIc 5y ago
      I think StarCraft I has a way to turn off fog of war.
      - 29athrowaway 5y ago
        Not in multiplayer games as far as I know.
        5y ago

9wzYQbTYsAIc 5y ago

> Is DL a reasonable path to strong AI? Arguably, it is the only path to strong AI (if you believe that it is even possible).

karpierz 5y ago
Of course strong AI is possible; most people can make one with 18 years and a willing partner. It just doesn't scale.
- spyder 5y ago
  That's Natural Intelligence not Artificial Intelligence.
  - okaram 5y ago
    Well ... Artificial means created by humans, right? ;)
  - 9wzYQbTYsAIc 5y ago
    Naturally, but we’re going for artificial humor today.
  - deleted 5y ago
    [deleted]
- gcheong 5y ago
  Also very nondeterministic
  - 9wzYQbTYsAIc 5y ago
    It’s a feature, not a bug.
- 9wzYQbTYsAIc 5y ago
  > most people can make one with 18 years and a willing partner Well, there’s your scaling problem. It should only take an instant to create a Strong AI (if you believe that Strong AI can be created). What’s Weak about a Strong AI that takes a couple of years to make but pees on the floor all the time?
- harry8 5y ago
  > It just doesn't scale. On what planet? I can give you about 7 billion reasons why it isn't this one.
  - samplatt 5y ago
    That's just 7 billion reasons why it can't (or at least, doesn't) scale larger than ~3lbs.
  - salawat 5y ago
    It doesn't scale in the "I want twice as many, right now, to use, abuse, and discard without guilt or strings attached". 9 women do not a baby in 1 month make. Once you have said bab(y|ies), you then have the inconvenience of them having a will of their own, you have to support and eventually pay them, and dear god, make too many of them and they might even just throw you away instead of going in the direction you want. That kind of not scaling. I mean... If no one else is willing to say it, I'll call a spade a spade; because that is precisely the major draw of the computer as ecpnomic/logistical lubricant. Doing jobs of people, but not getting paid or having to be accounted for as one.
    - izzygonzalez 5y ago
      If we manage to create an artificial sentience then we need to legislate and protect their rights. AGI won't be a free lunch.
      - drdeca 5y ago
        Just because something is AGI probably needn't imply that it is sapient. It might have no internal experience, no feeling of desire, only a utility function which it optimizes.
        krageon 5y ago
        This reasoning is why it will need rights. If it doesn't, people thinking these thoughts (which are impossible to answer, like they are for every other potential sapience. Not just AGI) will abuse that reasoning to create yet another creative modern form of slavery.
- mannykannot 5y ago
  That's the last step in a training process that took a half-billion years, if we count from the first neurons (this is not, of course, an argument against the feasibility of strong AI.)
  - 9wzYQbTYsAIc 5y ago
    One small step for bosons, one giant leap for consciousness.
  - Planktonne 5y ago
    Every DL system ever made is also the last step in a training process that took half a billion years. It's not reasonable to discount the training of parents/predecessors only when we're talking about machines; they still would not exist without them.
    - mannykannot 5y ago
      There is a slight ambiguity in 'last', which might mean either 'most recent' or 'final'. The difference here is that while karpierz's last step is, at least for now, the final one in a process producing a strong, generalized, self-awarely-conscious theory-of-mind-holding intelligent agent, the evolution of DL systems has not yet done so.
- auntienomen 5y ago
  It can be partly parallelized, by staggering.
- jahewson 5y ago
  It’s embarrassingly parallel.
dkislyuk 5y ago
What is the argument for it being the only path? We have a pretty poor understanding of how the human brain (our only "Strong AI" evidence) learns, but the one approach that has been pretty definitively ruled out is large-scale backpropagation in the style that most deep learning pursues today. Maybe backprop is a true alternative approach to AGI, but it's hard to be completely convinced of this.
- 9wzYQbTYsAIc 5y ago
  Have you read Parallel Distributed Processing by McClelland and Rumelhart? I haven’t finished reading it, but they make the case that if humans are conscious, then PDP is the way Strong AI will be figured out. https://mitpress.mit.edu/books/parallel-distributed-processing-volume-1 https://mitpress.mit.edu/books/parallel-distributed-processi...
  - garymarcus 5y ago
    I link it to it in the Nautilus article at the top of thread, and to my thesis work which challenged PDP as a model of children’s learning of the English past tense; more generally the Nautilus article is in some sense my answer to that book.
    - 9wzYQbTYsAIc 5y ago
      Just finished reading the article, interesting points. I agree that there’s too much emphasis on the “robot”, but coming from an academic background in mathematics and psychology (and biology), I cannot take PDP as anything other than “this is how brains work”. Granted, half of PDP, the book, is a series of examples of “ways it might work” from 40 years ago or so, but the other half is about fundamentally tying the psychology to the brain through parallel distributed processing as a conceptual framework. In that sense, I don’t think PDP can be refuted any more than token economies or political polling can be refuted. Thoughts? In the meantime, I will bookmark your thesis for perusal.

giraffe_lady 5y ago

As I'm reading it, the argument isn't that progress can't be made or cool things can't be done with these tools. It's that for certain problems no amount of progress will be sufficient, yet we have entire domains of the tech and consumer products industries built around and selling the idea that just a little bit more will get us there. The argument is compelling to me and fits my experiences. I think the limitations of these tools has become more apparent to non-specialists the last couple years, so if he's been saying this since 2016 that's honestly more impressive. What do you expect him to change his tune to if he's already singing a good one?

9wzYQbTYsAIc 5y ago
> It's that for certain problems no amount of progress will be sufficient In mathematics, you solve problems from both directions, at times. It seems that computer science has come full circle with psychology, finally. Dig into computational comparative psychology for leads on where deep learning experts need to turn to next. There are plenty of computational neuroscientists that are probably struggling with untwining what they are looking at.

idealmedtech 5y ago

I think he does offer some compelling arguments in this article, worth at least examining rather than dismissing based on his past skepticism. Particularly about the potential harmony between deep learning and symbolic manipulation, which is enormously difficult (a far cry from the set-and-forget style that has taken DL so far), but perhaps cracking AGI will require us to do a little more difficult work. I'd love to be proved wrong here, though! My depth of understanding on DL is pretty shallow.

garymarcus 5y ago
exactly - the article is a call to arms to do that difficult work.
- 9wzYQbTYsAIc 5y ago
  Why not return to the roots of Parallel Distributed Processing?
- p1esk 5y ago
  Lead by example. Instead of criticizing others.
zmgsabst 5y ago
naikrovek 5y ago
We are a very long way from even having 0.0001% of the compute required to produce a weak AGI. like everything, it will take much more resources and time than we can predict today with our best estimates, partly because that's just how these things turn out, and mostly because a true AGI will likely require billions of neural networks all adapting and swapping neurons and communications pathways amongst themselves and training themselves at the same time they are doing useful work. we have no idea how to do anything like this at scale, yet.
- p1esk 5y ago

29athrowaway 5y ago

DeepMind managed to rekt the best players in Go and StarCraft after a few years of founding. Go has been played for 3000 years, and it has influenced military strategy since ancient times. The strategies you can find in Sun Tzu's Art of War were influenced by the game of Go. Did you know what is the difference between the military simulators used by defense departments and strategy games like Go? Did you know both share a lot in common? What you dismiss as "game playing" has more present applications than you imagine and will make the difference between life and death in armed conflicts. The orders given by generals will likely come from a successor of MuZero. And that may expand to every key decision making process, like the equipment to be manufactured, logistics, etc. Combine that with the power asymmetry coming from drones and that "game playing" will become the most dominant force in this world.

9wzYQbTYsAIc 5y ago
Has anyone pit DeepMind against a constraint satisfaction algorithm in StarCraft? I’ve heard constraint satisfaction can be good enough to simulate playing StarCraft.
- 29athrowaway 5y ago
  StarCraft is not a complete information game. There's a fog of war. There's much that can be done to trick the opponent into making incorrect assumptions.
  - 9wzYQbTYsAIc 5y ago
    I think StarCraft I has a way to turn off fog of war.
    - 29athrowaway 5y ago
      Not in multiplayer games as far as I know.
      - 5y ago