6 ms·

Model-Free, Model-Based, and General Intelligence [pdf]

91 points by shurtler 8y ago

algorias 8y ago
I attended this talk at IJCAI, and I must say that the whole system 1 / system 2 analogy rubbed me the wrong way. A solver for e.g. 3-SAT is general only in a very narrow sense, namely that an entire class of problems can be reduced to the specific problem it solves. However, the solver itself is not doing the reducing, rather it is being spoon-fed instances generated by somebody, and that somebody is doing all the hard work of actually thinking. The solver is just doing a series of dumb steps very quickly, with lots of heuristics thrown in. How is that not also "system 1"? Anyway, the whole thing was just a fancy way of saying that you can either solve problems exactly, in the way that complexity theorists and algorithm designers do things, or statistically, in the way that learning theorists do things. No need to superimpose a strained analogy.
- nutjob2 8y ago
  I'm curious to know what your definition of "actually thinking" is. I suspect your argument is circular.
  - vinceguidry 8y ago
    All definitions of things get circular at some point. What defines a chair? The set of criteria you come up with to divide chairs from other things has to invariably turn in on itself, as the hilarious exercise to define a sandwich illustrates. All identity is ultimately an illusion. In other words, the more division you create in the world, the more 'specialness' you create. And in the immortal words of Syndrome, when everyone's special, no one is. With a fine enough definition of thought, anything can fit the definition.
- sophistication 8y ago
  Not to mention that there is no conclusive evidence of the dual process theory yet, see for example this experimental study finding that logical "type 2" answers are actually typically faster and that intuitive "type 1" answers are typically also logical: https://www.sciencedirect.com/science/article/pii/S0010027716302542 https://www.sciencedirect.com/science/article/pii/S001002771...
  - naasking 8y ago
    > Not to mention that there is no conclusive evidence of the dual process theory yet Define "conclusive". There is considerable evidence for this dual reasoning mode. As for your study, system 1 thinking is not inherently illogical. In fact, it's necessarily logical otherwise it would be maladaptive. The point is that it's logical in a "lossy" way that sometimes excludes pertinent information for speed of response, and so sometimes goes wildly wrong.
- eli_gottlieb 8y ago
  >I attended this talk at IJCAI, and I must say that the whole system 1 / system 2 analogy rubbed me the wrong way. It immediately rubs me in the wrong way because dual process theories of the brain are wrong and outmoded and need to die out of the public consciousness now! I am muttering angrily in cognitive science!
  - naasking 8y ago
    What's the current theory in vogue?
    - eli_gottlieb 8y ago
      Bayesian brain theories are more "in vogue", along with various other theories saying that the brain does some forms of statistical and causal learning and inference.
      - naasking 8y ago
        Sure, that seems reasonable. But I don't see why statistical and causal learning and inference really preclude the evolution of a system 1/2 dualism.
        eli_gottlieb 8y ago
        They don't preclude it, but they didn't happen to include it in our particular history. In particular, in the evolutionary history of the brain as an energy-optimizing controller of the body, a "System 1" would have been selected against extremely early on, when it directed the internal organs to act according to "heuristics" that wasted calories.
        naasking 8y ago
        > In particular, in the evolutionary history of the brain as an energy-optimizing controller of the body, a "System 1" would have been selected against extremely early on, when it directed the internal organs to act according to "heuristics" that wasted calories. How are the calories are wasted exactly? Our hind brain triggers autonomous reactions to various inputs. Clearly not all such reactions are adaptive, sometimes staying very still and bearing some pain or discomfort is better than death. And so we evolved higher level cognitive faculties to make better choices, just a little slower than the hind brain. This is system 1. I don't see why the exact same pressures couldn't work at this cognitive level as well. System 1 provided more adaptive reactions to a wider range of situations, but just a little slower than the hind brain. But even still, some metacognitive faculty would yield even better reactions in some circumstances, and so we evolved system 2. But system 1 still has tremendous significance, because it's much better than our hind brain, is sufficient for most daily scenarios, and is not as calorically expensive as system 2. The logic behind the efficiency gains is similar to the cache hierarchy in computers. We have more than one cache level because 2-3 cache levels is pretty close to optimal when trading off density, thermal considerations, and efficiency.
- dj-wonk 8y ago
  Yes, I know what you mean. In my opinion, the connection to the System 1 / System 2 theories did not add much depth to the paper. I think the intended purpose was to bolster the argument that both learners and solvers (operating in different ways) are both useful forms of intelligence. However, this point can be made in other ways as well. In any case, I look forward to more scholarship and experimentation at the intersection of these topics.
sgt101 8y ago
Was this an invited talk?
- odderik 8y ago
  Yep, it was.
nutjob2 8y ago
It seems the fundamental problem with bottom up/learning AI is that it is opaque and essentially unknowable. I find it all very hackish. We can develop systems now which we can test and seem to work, but we don't know exactly why they work (eg: what parts of the training data they are promoting) and when (or why) they will fail. The effectiveness of adversarial inputs to trained vision systems illustrates this. Zoom forward to a super-human AI that mimics our brains in its approach but exceeds its capacity. What is stopping it, for instance, learning that it can play the long game of being good until it has sufficient power at its disposal and then becoming evil? No matter what training data you present, you can't know exactly what the result will be. I get the feeling that learning systems will be combined with model systems with the former performing "low level" tasks and the latter providing a verifiable "executive" that guides high level goals or outcomes.
- nmca 8y ago
  One approach being considered is "AI Safety Via Debate"[0], which hopes to prevent deception by carefully constructing games in which a superhuman agent's best strategy is honesty. Note that this is the goal; much work to be done! [0] https://arxiv.org/abs/1805.00899 https://arxiv.org/abs/1805.00899
  - TeMPOraL 8y ago
    Do we know the entire game theory well enough so that we can structure such games with no theoretical way for AI to sneak out? I doubt that, but even so, funny things start happening when theory meets practice. I recall the example of quantum entanglement, which (I read) enables communications that cannot be spied upon without the intended parties knowing. Except, (I also read) it was attacked at the interface between quantum and classical domain. The world is complex, and superhuman AI is by definition better equipped to find loopholes than humans are.
  - nutjob2 8y ago
    Unfortunatley being dishonest or evil is just one example. Arguably the AI can develop new classes of deviancy, abuse or maladaptation that we haven't conceptualized yet. We supersize the ability, surely we supersize the problems. It leads to a scary question: what does a superhuman AI really want?
diminish 8y ago
"If we want good AI, we can’t look away from culture and politics." At the end AI will join our tense political atmosphere of parties fighting for ruling the world?
jeisc 8y ago
AGI will be a reflection of ourselves. First we must resolve the basic problems of the human condition (poverty, hunger, housing, war, ...) before developing AGI as it will surely amplify our worst nature as well as our best nature.

91 points by shurtler 8y ago

I attended this talk at IJCAI, and I must say that the whole system 1 / system 2 analogy rubbed me the wrong way. A solver for e.g. 3-SAT is general only in a very narrow sense, namely that an entire class of problems can be reduced to the specific problem it solves. However, the solver itself is not doing the reducing, rather it is being spoon-fed instances generated by somebody, and that somebody is doing all the hard work of actually thinking. The solver is just doing a series of dumb steps very quickly, with lots of heuristics thrown in. How is that not also "system 1"? Anyway, the whole thing was just a fancy way of saying that you can either solve problems exactly, in the way that complexity theorists and algorithm designers do things, or statistically, in the way that learning theorists do things. No need to superimpose a strained analogy.

nutjob2 8y ago
I'm curious to know what your definition of "actually thinking" is. I suspect your argument is circular.
- vinceguidry 8y ago
  All definitions of things get circular at some point. What defines a chair? The set of criteria you come up with to divide chairs from other things has to invariably turn in on itself, as the hilarious exercise to define a sandwich illustrates. All identity is ultimately an illusion. In other words, the more division you create in the world, the more 'specialness' you create. And in the immortal words of Syndrome, when everyone's special, no one is. With a fine enough definition of thought, anything can fit the definition.
sophistication 8y ago
Not to mention that there is no conclusive evidence of the dual process theory yet, see for example this experimental study finding that logical "type 2" answers are actually typically faster and that intuitive "type 1" answers are typically also logical: https://www.sciencedirect.com/science/article/pii/S0010027716302542 https://www.sciencedirect.com/science/article/pii/S001002771...
- naasking 8y ago
  > Not to mention that there is no conclusive evidence of the dual process theory yet Define "conclusive". There is considerable evidence for this dual reasoning mode. As for your study, system 1 thinking is not inherently illogical. In fact, it's necessarily logical otherwise it would be maladaptive. The point is that it's logical in a "lossy" way that sometimes excludes pertinent information for speed of response, and so sometimes goes wildly wrong.
eli_gottlieb 8y ago
>I attended this talk at IJCAI, and I must say that the whole system 1 / system 2 analogy rubbed me the wrong way. It immediately rubs me in the wrong way because dual process theories of the brain are wrong and outmoded and need to die out of the public consciousness now! I am muttering angrily in cognitive science!
- naasking 8y ago
  What's the current theory in vogue?
  - eli_gottlieb 8y ago
    Bayesian brain theories are more "in vogue", along with various other theories saying that the brain does some forms of statistical and causal learning and inference.
    - naasking 8y ago
      Sure, that seems reasonable. But I don't see why statistical and causal learning and inference really preclude the evolution of a system 1/2 dualism.
      - eli_gottlieb 8y ago
        They don't preclude it, but they didn't happen to include it in our particular history. In particular, in the evolutionary history of the brain as an energy-optimizing controller of the body, a "System 1" would have been selected against extremely early on, when it directed the internal organs to act according to "heuristics" that wasted calories.
        naasking 8y ago
        > In particular, in the evolutionary history of the brain as an energy-optimizing controller of the body, a "System 1" would have been selected against extremely early on, when it directed the internal organs to act according to "heuristics" that wasted calories. How are the calories are wasted exactly? Our hind brain triggers autonomous reactions to various inputs. Clearly not all such reactions are adaptive, sometimes staying very still and bearing some pain or discomfort is better than death. And so we evolved higher level cognitive faculties to make better choices, just a little slower than the hind brain. This is system 1. I don't see why the exact same pressures couldn't work at this cognitive level as well. System 1 provided more adaptive reactions to a wider range of situations, but just a little slower than the hind brain. But even still, some metacognitive faculty would yield even better reactions in some circumstances, and so we evolved system 2. But system 1 still has tremendous significance, because it's much better than our hind brain, is sufficient for most daily scenarios, and is not as calorically expensive as system 2. The logic behind the efficiency gains is similar to the cache hierarchy in computers. We have more than one cache level because 2-3 cache levels is pretty close to optimal when trading off density, thermal considerations, and efficiency.
dj-wonk 8y ago
Yes, I know what you mean. In my opinion, the connection to the System 1 / System 2 theories did not add much depth to the paper. I think the intended purpose was to bolster the argument that both learners and solvers (operating in different ways) are both useful forms of intelligence. However, this point can be made in other ways as well. In any case, I look forward to more scholarship and experimentation at the intersection of these topics.

sgt101 8y ago

Was this an invited talk?

odderik 8y ago
Yep, it was.

nutjob2 8y ago

It seems the fundamental problem with bottom up/learning AI is that it is opaque and essentially unknowable. I find it all very hackish. We can develop systems now which we can test and seem to work, but we don't know exactly why they work (eg: what parts of the training data they are promoting) and when (or why) they will fail. The effectiveness of adversarial inputs to trained vision systems illustrates this. Zoom forward to a super-human AI that mimics our brains in its approach but exceeds its capacity. What is stopping it, for instance, learning that it can play the long game of being good until it has sufficient power at its disposal and then becoming evil? No matter what training data you present, you can't know exactly what the result will be. I get the feeling that learning systems will be combined with model systems with the former performing "low level" tasks and the latter providing a verifiable "executive" that guides high level goals or outcomes.

nmca 8y ago
One approach being considered is "AI Safety Via Debate"[0], which hopes to prevent deception by carefully constructing games in which a superhuman agent's best strategy is honesty. Note that this is the goal; much work to be done! [0] https://arxiv.org/abs/1805.00899 https://arxiv.org/abs/1805.00899
- TeMPOraL 8y ago
  Do we know the entire game theory well enough so that we can structure such games with no theoretical way for AI to sneak out? I doubt that, but even so, funny things start happening when theory meets practice. I recall the example of quantum entanglement, which (I read) enables communications that cannot be spied upon without the intended parties knowing. Except, (I also read) it was attacked at the interface between quantum and classical domain. The world is complex, and superhuman AI is by definition better equipped to find loopholes than humans are.
- nutjob2 8y ago
  Unfortunatley being dishonest or evil is just one example. Arguably the AI can develop new classes of deviancy, abuse or maladaptation that we haven't conceptualized yet. We supersize the ability, surely we supersize the problems. It leads to a scary question: what does a superhuman AI really want?

diminish 8y ago

"If we want good AI, we can’t look away from culture and politics." At the end AI will join our tense political atmosphere of parties fighting for ruling the world?

jeisc 8y ago

AGI will be a reflection of ourselves. First we must resolve the basic problems of the human condition (poverty, hunger, housing, war, ...) before developing AGI as it will surely amplify our worst nature as well as our best nature.