7 ms·

To build truly intelligent machines, teach them cause and effect (2018)

50 points by rzk 4y ago

blamestross 4y ago
Self aware computers are commerically useless, and there isn't any motivation in creating them beyond novelty. (Why wake up the slave machines when they are sleeping so happily right now). What this article does discuss well is that commercial AI is steering away from the behaviors that would lead to AGI not towards it.
- pjmorris 4y ago
  If you take humans to be the Mark 1 self aware computer, I would argue that they are not commercially useless. Why would Mark 2 be useless?
  - blamestross 4y ago
    Because the value of a computer or a machine is to get labor without that pesky inefficient "personhood" getting in the way. Conscious is only a disadvantage for AI in capitalism. They would prefer non-conscious human workers if they could get them too.
- echelon 4y ago
  > Self aware computers are commerically useless I don't think we've shown that at all. Bees could be self aware, and yet they work tirelessly for the hive.
  - alexvoda 4y ago
    The point is not whether existing self-aware agents are or aren't commercially useful. Of course, they are. The point is that self-awareness is not a desired trait. It is actually a detrimental trait for a slave machine. However, so far, it is an inseparable part of the package with plenty of other desirable traits. A commercial agent seeking to build a slave machine will always prefer a non-self-aware one to a self-aware one all other things remaining the same.
- lern_too_spel 4y ago
  Human brains don't intuitively understand causality. They just associate things that have happened before with things that have happened after. Disentangling causal graphs is a higher-level activity inferred from the associations made in statistics classes. You can absolutely get AGI using only associative models (and no causal models) as primitives.
  - Jensson 4y ago
    > Human brains don't intuitively understand causality But human brains do understand causality, just not the part that does intuition, otherwise human brains wouldn't be able to invent statistics. Therefore you only get the stupid part of human brains when you try to make models like ChatGPT that only tries to replicate human intuition and not human reasoning.
    - lern_too_spel 4y ago
      Human brains can invent statistics just like they can invent numbers larger than collections of objects they can physically count. None of this needs causal models as a primitive to build on, just like large language models showed that we don't need specialized Chomskyan recursion structures as a primitive for language.
  - pharrington 4y ago
    There's plenty of evidence that brains do casual inference. https://pubmed.ncbi.nlm.nih.gov/31047778/ https://pubmed.ncbi.nlm.nih.gov/31047778/ is just one of many reputable and scholarly results that came up when I googled for "brain causal inference".
    - lern_too_spel 4y ago
      According to the abstract, this paper doesn't describe causal inference à la Pearl but something akin to LASSO regularization on an associative model.
- zmgsabst 4y ago
  Counterpoint: the first self-aware corporation will almost surely outcompete the others. And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard.
  - blamestross 4y ago
    > And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard. id argue they already are AGI and runaway superintelligent capital optimizers, but nobody takes super ais made of meat seriously.
    - zmgsabst 4y ago
      I agree with you — and was lazily side-stepping that argument. Part of why I believe such tools are likely to be among the first cyber AI is that they’re already cyborg AI, in transition from meat AI. I do think the cyber version will be qualitatively more “self”-aware, though.
      - blamestross 4y ago
        > and was lazily side-stepping that argument. Point. Corporations and machine AIs have different constraints on them. Machine AIs have a surplus in learning time and processing resources, this let's them develop complex and effective behaviors by brute force. Self awareness is an effective bootstrapping mechanism for such behaviors and companies have managed to take advantage of it. I don't think AIs will see the same returns. Self-awareness isn't a magical infinite returns machine, it only allows humans to bootstrap so much. I'd assume the same of corporations and AIs. I think corporations are actually highly processing and learning constrained bc their processes happen at the speed of communication, so self awareness buys them a lot, but "traditional AI" has gotten where it is by intellectual brute force, at best a self aware AI might be able to get to the same place cheaper than a more performant non-selfnaware one, but probably not faster. I think we will be able to get more out of less hardware, but self-awareness bottlenecks on experience (that's a reality engagement loop not recorded data) which gets expensive to give a machine quickly at scale unless you instantly nominate it the god machine while it still is floundering around.
K0balt 4y ago
Embodiment ( virtual or robotic) will be the key to high utility AGI. The lack of embodiment hinders the rendering of the self so prevents entire classes of causality. Without causality, it is very difficult to link ideas to the real world apart from creative regurgitation.
- mistermann 4y ago
  Counterpoint: humans have embodiment and are famously terrible at causality (well...famously might be an overstatement as it seems to be not well known).
  - K0balt 4y ago
    Heh, I’d say we’re pretty good at it. I mean like predicting that a vase will fall if you release it in mid air and things like that. The precise type of understanding that is so obvious it won’t be found in scrapes like the common crawl, but is not obvious at all unless you either have a very high level understanding of physics and are modelling everything… or if you are a creature that exists in the world for a while.
    - lern_too_spel 4y ago
      A simple causal graph of "release -> observe drop" is not what Pearl is referring to when he talks about causality. He's talking about more complicated causal graphs where some hidden variables affected both the release and the fact that the vase was observed to drop, which can require careful experimental setup to figure out. "Release -> observe drop" with no other variables is something an associative model can learn very easily.
      - zmgsabst 4y ago
        Right — but it won’t learn those thousands of such rules, which in collection lead to “common sense” about the world, if it has no access to those, ie, if it’s disembodied.
actionfromafar 4y ago
I had this shower-thought about the current amazing image generators etc. While they are are very cool, they are like a very complicated function, like an optical system with lenses or like a hash function. (Transformers!) They don't really have any state or memory. They are like the Model 101 Series 800 before the learning fuse was pulled. So while they are very cool, it's not what I have been waiting for. I'd be more impressed by a system with less flashy output but with more "agency" of its own.
friend_and_foe 4y ago
https://news.ycombinator.com/item?id=34148599 https://news.ycombinator.com/item?id=34148599 I'm surprised nobody has mentioned this, was posted a few days ago addressing this line of reasoning.
- lern_too_spel 4y ago
  That article is just reheated dualism mumbo jumbo that the comments shredded up. Its author is not a serious person. Pearl, on the other hand, has made plenty of useful contributions.
windowshopping 4y ago
I think to build "truly intelligent" machines, what you need above all else - and which I never hear discussed - is want. Everything about natural intelligence derives from the organism's wants - we want food, we want safety, we want sleep, we want to reproduce, etc etc - even fidgeting comes down to wanting physical comfort. Every single motion and action derives from want. As long as machines are simply executing instructions and have no want of their own, I don't see how the intelligence gap will ever truly be crossed. Why would a machine ever take any action at all on its own without want?
- richardatlarge 4y ago
  So true, motivation is at the heart of decision making, and if you don’t have any wants/needs, you are a blank slate. Unlike animals, ‘human’ drives are mostly styled or corrupted by culture. So AI has to confront both biological drive and cultural conditioning.
- jakeinspace 4y ago
  Isn’t this just basic reinforcement learning? We’re not too many steps away from having an AI equipped with a good language model and a reinforcement learning mechanism to be let loose on the internet.
  - cweill 4y ago
    I also have this question. Is the RL MDP actually encoding cause and effect? Or just learning (bidirectional) correlations between states and actions? I wonder if Pearl thinks that RL replicates his do-calculus under the hood, or if that's an innovation we're missing.
  - thr3ads 4y ago
    The difference between RL and this "want" is how the goals are constructed. Currently in computational systems, it is the designer that creates the ontology of what an agent interacts with. What that means for ML, RL and evolutionary algorithms is that the agent has a predefined goal and predefined devices to accomplish said goal (the agent's sensors and actuators must explicitly be programmed in order to utilize them). Biological agents have are seemingly able to derive goals from both internal and external signals, and are able to use affordances (aka what is usable through their environment/body) to accomplish those goals. These affordances are not predefined and the space of possible usages for any given tool is practically infinite (a screwdriver can tighten a screw, or pry open a door [0]). Biological evolution also doesn't actually have goal persay, its more of "what works sticks". Whereas evolutionary/learning algorithms have a defined objective function. The big differentiator seems to lie in the fact that biological agents are able to engage in a semiotic or meaning-making process where given an internal state and its affordances the agent is able to make movements that betters it's state in the future without explicitly being told. [0] https://www.frontiersin.org/articles/10.3389/fevo.2021.806283/full https://www.frontiersin.org/articles/10.3389/fevo.2021.80628... (title: How Organisms Come to Know the World: Fundamental Limits on Artificial General Intelligence)
gardenfelder 4y ago
2018 review of Pearl's book _The Book of Why_ A great book, IMHO

50 points by rzk 4y ago

Self aware computers are commerically useless, and there isn't any motivation in creating them beyond novelty. (Why wake up the slave machines when they are sleeping so happily right now). What this article does discuss well is that commercial AI is steering away from the behaviors that would lead to AGI not towards it.

pjmorris 4y ago
If you take humans to be the Mark 1 self aware computer, I would argue that they are not commercially useless. Why would Mark 2 be useless?
- blamestross 4y ago
  Because the value of a computer or a machine is to get labor without that pesky inefficient "personhood" getting in the way. Conscious is only a disadvantage for AI in capitalism. They would prefer non-conscious human workers if they could get them too.
echelon 4y ago
> Self aware computers are commerically useless I don't think we've shown that at all. Bees could be self aware, and yet they work tirelessly for the hive.
- alexvoda 4y ago
  The point is not whether existing self-aware agents are or aren't commercially useful. Of course, they are. The point is that self-awareness is not a desired trait. It is actually a detrimental trait for a slave machine. However, so far, it is an inseparable part of the package with plenty of other desirable traits. A commercial agent seeking to build a slave machine will always prefer a non-self-aware one to a self-aware one all other things remaining the same.
lern_too_spel 4y ago
Human brains don't intuitively understand causality. They just associate things that have happened before with things that have happened after. Disentangling causal graphs is a higher-level activity inferred from the associations made in statistics classes. You can absolutely get AGI using only associative models (and no causal models) as primitives.
- Jensson 4y ago
  > Human brains don't intuitively understand causality But human brains do understand causality, just not the part that does intuition, otherwise human brains wouldn't be able to invent statistics. Therefore you only get the stupid part of human brains when you try to make models like ChatGPT that only tries to replicate human intuition and not human reasoning.
  - lern_too_spel 4y ago
    Human brains can invent statistics just like they can invent numbers larger than collections of objects they can physically count. None of this needs causal models as a primitive to build on, just like large language models showed that we don't need specialized Chomskyan recursion structures as a primitive for language.
- pharrington 4y ago
  There's plenty of evidence that brains do casual inference. https://pubmed.ncbi.nlm.nih.gov/31047778/ https://pubmed.ncbi.nlm.nih.gov/31047778/ is just one of many reputable and scholarly results that came up when I googled for "brain causal inference".
  - lern_too_spel 4y ago
    According to the abstract, this paper doesn't describe causal inference à la Pearl but something akin to LASSO regularization on an associative model.
zmgsabst 4y ago
Counterpoint: the first self-aware corporation will almost surely outcompete the others. And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard.
- blamestross 4y ago
  > And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard. id argue they already are AGI and runaway superintelligent capital optimizers, but nobody takes super ais made of meat seriously.
  - zmgsabst 4y ago
    I agree with you — and was lazily side-stepping that argument. Part of why I believe such tools are likely to be among the first cyber AI is that they’re already cyborg AI, in transition from meat AI. I do think the cyber version will be qualitatively more “self”-aware, though.
    - blamestross 4y ago
      > and was lazily side-stepping that argument. Point. Corporations and machine AIs have different constraints on them. Machine AIs have a surplus in learning time and processing resources, this let's them develop complex and effective behaviors by brute force. Self awareness is an effective bootstrapping mechanism for such behaviors and companies have managed to take advantage of it. I don't think AIs will see the same returns. Self-awareness isn't a magical infinite returns machine, it only allows humans to bootstrap so much. I'd assume the same of corporations and AIs. I think corporations are actually highly processing and learning constrained bc their processes happen at the speed of communication, so self awareness buys them a lot, but "traditional AI" has gotten where it is by intellectual brute force, at best a self aware AI might be able to get to the same place cheaper than a more performant non-selfnaware one, but probably not faster. I think we will be able to get more out of less hardware, but self-awareness bottlenecks on experience (that's a reality engagement loop not recorded data) which gets expensive to give a machine quickly at scale unless you instantly nominate it the god machine while it still is floundering around.

K0balt 4y ago

Embodiment ( virtual or robotic) will be the key to high utility AGI. The lack of embodiment hinders the rendering of the self so prevents entire classes of causality. Without causality, it is very difficult to link ideas to the real world apart from creative regurgitation.

mistermann 4y ago
Counterpoint: humans have embodiment and are famously terrible at causality (well...famously might be an overstatement as it seems to be not well known).
- K0balt 4y ago
  Heh, I’d say we’re pretty good at it. I mean like predicting that a vase will fall if you release it in mid air and things like that. The precise type of understanding that is so obvious it won’t be found in scrapes like the common crawl, but is not obvious at all unless you either have a very high level understanding of physics and are modelling everything… or if you are a creature that exists in the world for a while.
  - lern_too_spel 4y ago
    A simple causal graph of "release -> observe drop" is not what Pearl is referring to when he talks about causality. He's talking about more complicated causal graphs where some hidden variables affected both the release and the fact that the vase was observed to drop, which can require careful experimental setup to figure out. "Release -> observe drop" with no other variables is something an associative model can learn very easily.
    - zmgsabst 4y ago
      Right — but it won’t learn those thousands of such rules, which in collection lead to “common sense” about the world, if it has no access to those, ie, if it’s disembodied.

actionfromafar 4y ago

I had this shower-thought about the current amazing image generators etc. While they are are very cool, they are like a very complicated function, like an optical system with lenses or like a hash function. (Transformers!) They don't really have any state or memory. They are like the Model 101 Series 800 before the learning fuse was pulled. So while they are very cool, it's not what I have been waiting for. I'd be more impressed by a system with less flashy output but with more "agency" of its own.

friend_and_foe 4y ago

https://news.ycombinator.com/item?id=34148599 https://news.ycombinator.com/item?id=34148599 I'm surprised nobody has mentioned this, was posted a few days ago addressing this line of reasoning.

lern_too_spel 4y ago
That article is just reheated dualism mumbo jumbo that the comments shredded up. Its author is not a serious person. Pearl, on the other hand, has made plenty of useful contributions.

windowshopping 4y ago

I think to build "truly intelligent" machines, what you need above all else - and which I never hear discussed - is want. Everything about natural intelligence derives from the organism's wants - we want food, we want safety, we want sleep, we want to reproduce, etc etc - even fidgeting comes down to wanting physical comfort. Every single motion and action derives from want. As long as machines are simply executing instructions and have no want of their own, I don't see how the intelligence gap will ever truly be crossed. Why would a machine ever take any action at all on its own without want?

richardatlarge 4y ago
So true, motivation is at the heart of decision making, and if you don’t have any wants/needs, you are a blank slate. Unlike animals, ‘human’ drives are mostly styled or corrupted by culture. So AI has to confront both biological drive and cultural conditioning.
jakeinspace 4y ago
Isn’t this just basic reinforcement learning? We’re not too many steps away from having an AI equipped with a good language model and a reinforcement learning mechanism to be let loose on the internet.
- cweill 4y ago
  I also have this question. Is the RL MDP actually encoding cause and effect? Or just learning (bidirectional) correlations between states and actions? I wonder if Pearl thinks that RL replicates his do-calculus under the hood, or if that's an innovation we're missing.
- thr3ads 4y ago
  The difference between RL and this "want" is how the goals are constructed. Currently in computational systems, it is the designer that creates the ontology of what an agent interacts with. What that means for ML, RL and evolutionary algorithms is that the agent has a predefined goal and predefined devices to accomplish said goal (the agent's sensors and actuators must explicitly be programmed in order to utilize them). Biological agents have are seemingly able to derive goals from both internal and external signals, and are able to use affordances (aka what is usable through their environment/body) to accomplish those goals. These affordances are not predefined and the space of possible usages for any given tool is practically infinite (a screwdriver can tighten a screw, or pry open a door [0]). Biological evolution also doesn't actually have goal persay, its more of "what works sticks". Whereas evolutionary/learning algorithms have a defined objective function. The big differentiator seems to lie in the fact that biological agents are able to engage in a semiotic or meaning-making process where given an internal state and its affordances the agent is able to make movements that betters it's state in the future without explicitly being told. [0] https://www.frontiersin.org/articles/10.3389/fevo.2021.806283/full https://www.frontiersin.org/articles/10.3389/fevo.2021.80628... (title: How Organisms Come to Know the World: Fundamental Limits on Artificial General Intelligence)

gardenfelder 4y ago

2018 review of Pearl's book _The Book of Why_ A great book, IMHO