5 ms·

> I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem. When all you

> I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem. When all you have is a hammer... I think we’re really thinking about the whole thing backwards, as if intelligence only comes from the cortex of the brain. I think human level intelligence is fundamentally an emotional state of being - our core lizard brain values aren’t something to be swept aside - they’re a fundamental part of real intelligence. It’s like trying to build a car starting from the body and electronics rather than the motor.

joe_the_user 8y ago
Well, if a computer scientist only has a bunch of data, is there any conceivable way to reason about that data that doesn't ultimately boil down to a data optimization problem? I hear that correlation does not imply causation but also that you can't distinguish correlation from causation merely with a stream of data. Is there any way out of this situation with just data?
- salty_biscuits 8y ago
  I think apart from the work Pearl and others did with the do-calculus, all you can do is suggest experiments. You can only ever have a true ai if it can interact and ask questions about the environment.
  - joe_the_user 8y ago
    Well, even if an AI can interact with it's environment, how can those interactions not simply also be correlations between input and output. We have for example, recurrent neural networks which seem to do this already. It seems like saying "it's not optimization" has to boil down to "it's not generic optimization, it's a special kind of optimization." But maybe there's some formalism I'm missing.
    - salty_biscuits 8y ago
      As a thought experiment, imagine an AI is trying to work out if a rooster crowing makes the sun rise or the other way around (because they are always correlated in its experience). It can do the experiment of putting the rooster in a bag and see if the sun doesn't rise. It can refine it's model after doing this experiment (all with optimization). It is more about untangling correlations that come from latent variables that the system doesn't know about yet.
      - halflings 8y ago
        Well, that sounds an awful lot like an optimization problem, and is actually reminescent of a Machine Learning technique called Active Learning: the model suggests data points for which it has low confidence (say images that resembles both a cat and dog according to its own understanding), and a human labels those (or in your case: the AI itself runs an experiment to get the ground truth) to gain the most information.
        salty_biscuits 8y ago
        Yes. I completely agree. I think he (Pearl) is being unfairly provocative and there has been a lot of progress in these areas.
        joe_the_user 8y ago
        The thing is that "not knowing casualty" along with "lack of understanding" feels very plausible for an explanation of what current AI is missing. And while AI has made lots of progress, it certainly feels like this progress is all of a certain kind (though that could the human reflex to discount any problem solved by robot). The problem might be that "understanding" isn't something we really have a good working definition of.
        imh 8y ago
        It's not quite the same as active learning. Experimentation and active learning are quite different in ways I didn't fully appreciate at the beginning of my career, and they come in the form of (conditional) independence relations, which you can get around with either structural assumptions or randomized intervention. It's the entire reason why we randomize in experiments. In active learning, the machine doesn't know what happens when the rooster is in a bag, so decides to try it. In a randomized experiment, the machine knows what happens when the rooster is in a bag, but thinks there might be other factors at work, so it decides to try it in a way that it can be sure that all other factors are equal (at least on average).
- mannykannot 8y ago
  We can call it a data optimization problem, but that does not tell us anything about how we go about solving it, or even what we are optimizing for. Alternatively, perhaps what's being optimized is so general that generalized AI is not, and cannot be, 'just' an optimization problem in any useful sense. Something like the biologically-inspired 'maximize lifetime production of descendants' does not get us very far.
  - joe_the_user 8y ago
    "We can call it a data optimization problem, but that does not tell us anything about how we go about solving it, or even what we are optimizing for." If you have a time-ordered stream of data, the problem of predicting the output given previous input naturally suggests itself. There are other approaches, of course. "Alternatively, perhaps what's being optimized is so general that generalized AI is not, and cannot be, 'just' an optimization problem in any useful sense." - Perhaps you mean this as "the problem is so complex that it can't be naively approached only with conventional, local optimization tools". Because a huge optimization problem is still an optimization problem in some meaningful senses, I mean I could consider my life as a "fun" or "flow" or "enlightenment" optimization problem. In general, it seems like a lot of reactions to particular AI methods go from "this approach isn't enough to get us all the way there" to "real AI is fundamentally, absolutely different from this, this approach is useless", a jump I don't think is justified.
- CuriouslyC 8y ago
  Your definition of data is too narrow. The result of any experiment is just data. All the human brain has to work with is just data. Either there is enough information to distinguish correlation from causation analytically, or humans are actually incapable of distinguishing the two. That doesn't mean current analytic techniques are advanced enough to do it, just that it is possible for machines if it is possible for man.
  - joe_the_user 8y ago
    Either there is enough information to distinguish correlation from causation analytically, or humans are actually incapable of distinguishing the two. Well, what I'd like to know is how you distinguish correlation from causation with any amount of data. What definition, in terms of data, do you use? Note, probabilistic and algorithmic theories of causation have a long history and many problems, see: https://plato.stanford.edu/entries/causation-probabilistic/ https://plato.stanford.edu/entries/causation-probabilistic/

joe_the_user 8y ago

Well, if a computer scientist only has a bunch of data, is there any conceivable way to reason about that data that doesn't ultimately boil down to a data optimization problem? I hear that correlation does not imply causation but also that you can't distinguish correlation from causation merely with a stream of data. Is there any way out of this situation with just data?

salty_biscuits 8y ago
I think apart from the work Pearl and others did with the do-calculus, all you can do is suggest experiments. You can only ever have a true ai if it can interact and ask questions about the environment.
- joe_the_user 8y ago
  Well, even if an AI can interact with it's environment, how can those interactions not simply also be correlations between input and output. We have for example, recurrent neural networks which seem to do this already. It seems like saying "it's not optimization" has to boil down to "it's not generic optimization, it's a special kind of optimization." But maybe there's some formalism I'm missing.
  - salty_biscuits 8y ago
    As a thought experiment, imagine an AI is trying to work out if a rooster crowing makes the sun rise or the other way around (because they are always correlated in its experience). It can do the experiment of putting the rooster in a bag and see if the sun doesn't rise. It can refine it's model after doing this experiment (all with optimization). It is more about untangling correlations that come from latent variables that the system doesn't know about yet.
    - halflings 8y ago
      Well, that sounds an awful lot like an optimization problem, and is actually reminescent of a Machine Learning technique called Active Learning: the model suggests data points for which it has low confidence (say images that resembles both a cat and dog according to its own understanding), and a human labels those (or in your case: the AI itself runs an experiment to get the ground truth) to gain the most information.
      - salty_biscuits 8y ago
        Yes. I completely agree. I think he (Pearl) is being unfairly provocative and there has been a lot of progress in these areas.
        joe_the_user 8y ago
        The thing is that "not knowing casualty" along with "lack of understanding" feels very plausible for an explanation of what current AI is missing. And while AI has made lots of progress, it certainly feels like this progress is all of a certain kind (though that could the human reflex to discount any problem solved by robot). The problem might be that "understanding" isn't something we really have a good working definition of.
      - imh 8y ago
        It's not quite the same as active learning. Experimentation and active learning are quite different in ways I didn't fully appreciate at the beginning of my career, and they come in the form of (conditional) independence relations, which you can get around with either structural assumptions or randomized intervention. It's the entire reason why we randomize in experiments. In active learning, the machine doesn't know what happens when the rooster is in a bag, so decides to try it. In a randomized experiment, the machine knows what happens when the rooster is in a bag, but thinks there might be other factors at work, so it decides to try it in a way that it can be sure that all other factors are equal (at least on average).
mannykannot 8y ago
We can call it a data optimization problem, but that does not tell us anything about how we go about solving it, or even what we are optimizing for. Alternatively, perhaps what's being optimized is so general that generalized AI is not, and cannot be, 'just' an optimization problem in any useful sense. Something like the biologically-inspired 'maximize lifetime production of descendants' does not get us very far.
- joe_the_user 8y ago
  "We can call it a data optimization problem, but that does not tell us anything about how we go about solving it, or even what we are optimizing for." If you have a time-ordered stream of data, the problem of predicting the output given previous input naturally suggests itself. There are other approaches, of course. "Alternatively, perhaps what's being optimized is so general that generalized AI is not, and cannot be, 'just' an optimization problem in any useful sense." - Perhaps you mean this as "the problem is so complex that it can't be naively approached only with conventional, local optimization tools". Because a huge optimization problem is still an optimization problem in some meaningful senses, I mean I could consider my life as a "fun" or "flow" or "enlightenment" optimization problem. In general, it seems like a lot of reactions to particular AI methods go from "this approach isn't enough to get us all the way there" to "real AI is fundamentally, absolutely different from this, this approach is useless", a jump I don't think is justified.
CuriouslyC 8y ago
Your definition of data is too narrow. The result of any experiment is just data. All the human brain has to work with is just data. Either there is enough information to distinguish correlation from causation analytically, or humans are actually incapable of distinguishing the two. That doesn't mean current analytic techniques are advanced enough to do it, just that it is possible for machines if it is possible for man.
- joe_the_user 8y ago
  Either there is enough information to distinguish correlation from causation analytically, or humans are actually incapable of distinguishing the two. Well, what I'd like to know is how you distinguish correlation from causation with any amount of data. What definition, in terms of data, do you use? Note, probabilistic and algorithmic theories of causation have a long history and many problems, see: https://plato.stanford.edu/entries/causation-probabilistic/ https://plato.stanford.edu/entries/causation-probabilistic/