4 ms·

I think to build "truly intelligent" machines, what you need above all else - and which I never hear discussed - is want. Everything about natural intelligence

I think to build "truly intelligent" machines, what you need above all else - and which I never hear discussed - is want. Everything about natural intelligence derives from the organism's wants - we want food, we want safety, we want sleep, we want to reproduce, etc etc - even fidgeting comes down to wanting physical comfort. Every single motion and action derives from want. As long as machines are simply executing instructions and have no want of their own, I don't see how the intelligence gap will ever truly be crossed. Why would a machine ever take any action at all on its own without want?

richardatlarge 4y ago
So true, motivation is at the heart of decision making, and if you don’t have any wants/needs, you are a blank slate. Unlike animals, ‘human’ drives are mostly styled or corrupted by culture. So AI has to confront both biological drive and cultural conditioning.
jakeinspace 4y ago
Isn’t this just basic reinforcement learning? We’re not too many steps away from having an AI equipped with a good language model and a reinforcement learning mechanism to be let loose on the internet.
- cweill 4y ago
  I also have this question. Is the RL MDP actually encoding cause and effect? Or just learning (bidirectional) correlations between states and actions? I wonder if Pearl thinks that RL replicates his do-calculus under the hood, or if that's an innovation we're missing.
- thr3ads 4y ago
  The difference between RL and this "want" is how the goals are constructed. Currently in computational systems, it is the designer that creates the ontology of what an agent interacts with. What that means for ML, RL and evolutionary algorithms is that the agent has a predefined goal and predefined devices to accomplish said goal (the agent's sensors and actuators must explicitly be programmed in order to utilize them). Biological agents have are seemingly able to derive goals from both internal and external signals, and are able to use affordances (aka what is usable through their environment/body) to accomplish those goals. These affordances are not predefined and the space of possible usages for any given tool is practically infinite (a screwdriver can tighten a screw, or pry open a door [0]). Biological evolution also doesn't actually have goal persay, its more of "what works sticks". Whereas evolutionary/learning algorithms have a defined objective function. The big differentiator seems to lie in the fact that biological agents are able to engage in a semiotic or meaning-making process where given an internal state and its affordances the agent is able to make movements that betters it's state in the future without explicitly being told. [0] https://www.frontiersin.org/articles/10.3389/fevo.2021.806283/full https://www.frontiersin.org/articles/10.3389/fevo.2021.80628... (title: How Organisms Come to Know the World: Fundamental Limits on Artificial General Intelligence)
grugagag 4y ago
It is possible that machines can be programmed with a set of main (higher) goals and can then set themselves sub-goals in order to achieve those higher goals.. The question is whether us humans would be left out to compete for resources, AI machines may at some point become able to out-compete us for resources. Do we really need to satisfy our desire to play God? Another less than perfect possibility to come out of this could be AI based war machines - goal driven self sufficient entities that have two goals, to survive and to kill.
Misdicorl 4y ago
I think you have this backwards. The missing magic is consequences. From there, AIs that avoid bad consequences (reduced compute budget?) and seek good consequences (??) may develop a 'want' heuristic that favors those things. Or rather, the ones that do will be more successful and the ones that don't will die out.
- pdimitar 4y ago
  You might be partially correct but you seem to omit built-in wants e.g. hunger or thirst or lust. Furthermore, if a machine has no wants/needs then it won't take any action at all. It will just stand there and won't even experiment to find the good or bad outcomes.
  - ahefner 4y ago
    Perhaps, while slowly learning, the machine will cry.
    - dunefox 4y ago
      The machine is me.
  - ithkuil 4y ago
    If not doing anything has bad consequences, then it will do something to avoid bad consequences. It seems to me that "want" and "avoid bad consequences" are quite isomorphic
    - alexvoda 4y ago
      It would need "to want" "to avoid bad consequences". It is rather circular because of the word "bad" which implies "do not want". If it is indifferent to all consequences, you are back to square 1, so no, consequences are not the missing magic. Consequences are a function that transform one "want" into a different "want".
      - 4y ago
vintermann 4y ago
You're right that want is important. But what I think you miss (I may be wrong) is that "want" is not an empirical category. Does a cloud want to rain? Or does it just rain because it obeys physical laws as part of a huge complex system? Do I want to have a cup of coffee? Or a job? Or a girlfriend? Or am I likewise just doing things because my atoms obey physical laws as part of a huge complex system? This is not a scientific question. It is a teleological question. That doesn't make it less important, on the contrary. It's the answers to teleological questions which make anything important, including the scientific questions which happen to be important. So the question is not, "what can we do to make a machine want things", it's "When should we ascribe what a machine does to its own wants". And my answer is "never". Not as long as it's a product of our wants, which they always will be.
- itworkslikethat 4y ago
  Yes, you want to work, you want to have a cup of coffee. You may not like it, but you decided to do it and then executed a very complex series of steps to accomplish the goal. You're not a cloud, working/drinking is not an inevitable and direct physical process you're undergoing by yourself.
  - vintermann 4y ago
    I obviously agree that I have both will and wants. But I can't say that the physical processes in me are in any fundamental way different, than physical processes anywhere else in nature. Most are deterministic, some may be inscrutable, and some may even be random or unknowable - but all those things can be said about the weather, too. You cannot derive my wants from my capabilities (unless you're prepared you ascribe wants to clouds as well, I guess).
    - itworkslikethat 4y ago
      The actual physical process of your body using the water once you ingest it is not much different, sure. But you can choose not to drink (and die some days later). A cloud can't choose not to rain.
xtiansimon 4y ago
You first need individual embodiment, then you will want all sorts of things.

richardatlarge 4y ago

So true, motivation is at the heart of decision making, and if you don’t have any wants/needs, you are a blank slate. Unlike animals, ‘human’ drives are mostly styled or corrupted by culture. So AI has to confront both biological drive and cultural conditioning.

jakeinspace 4y ago

Isn’t this just basic reinforcement learning? We’re not too many steps away from having an AI equipped with a good language model and a reinforcement learning mechanism to be let loose on the internet.

cweill 4y ago
I also have this question. Is the RL MDP actually encoding cause and effect? Or just learning (bidirectional) correlations between states and actions? I wonder if Pearl thinks that RL replicates his do-calculus under the hood, or if that's an innovation we're missing.
thr3ads 4y ago
The difference between RL and this "want" is how the goals are constructed. Currently in computational systems, it is the designer that creates the ontology of what an agent interacts with. What that means for ML, RL and evolutionary algorithms is that the agent has a predefined goal and predefined devices to accomplish said goal (the agent's sensors and actuators must explicitly be programmed in order to utilize them). Biological agents have are seemingly able to derive goals from both internal and external signals, and are able to use affordances (aka what is usable through their environment/body) to accomplish those goals. These affordances are not predefined and the space of possible usages for any given tool is practically infinite (a screwdriver can tighten a screw, or pry open a door [0]). Biological evolution also doesn't actually have goal persay, its more of "what works sticks". Whereas evolutionary/learning algorithms have a defined objective function. The big differentiator seems to lie in the fact that biological agents are able to engage in a semiotic or meaning-making process where given an internal state and its affordances the agent is able to make movements that betters it's state in the future without explicitly being told. [0] https://www.frontiersin.org/articles/10.3389/fevo.2021.806283/full https://www.frontiersin.org/articles/10.3389/fevo.2021.80628... (title: How Organisms Come to Know the World: Fundamental Limits on Artificial General Intelligence)

grugagag 4y ago

It is possible that machines can be programmed with a set of main (higher) goals and can then set themselves sub-goals in order to achieve those higher goals.. The question is whether us humans would be left out to compete for resources, AI machines may at some point become able to out-compete us for resources. Do we really need to satisfy our desire to play God? Another less than perfect possibility to come out of this could be AI based war machines - goal driven self sufficient entities that have two goals, to survive and to kill.

Misdicorl 4y ago

I think you have this backwards. The missing magic is consequences. From there, AIs that avoid bad consequences (reduced compute budget?) and seek good consequences (??) may develop a 'want' heuristic that favors those things. Or rather, the ones that do will be more successful and the ones that don't will die out.

pdimitar 4y ago
You might be partially correct but you seem to omit built-in wants e.g. hunger or thirst or lust. Furthermore, if a machine has no wants/needs then it won't take any action at all. It will just stand there and won't even experiment to find the good or bad outcomes.
- ahefner 4y ago
  Perhaps, while slowly learning, the machine will cry.
  - dunefox 4y ago
    The machine is me.
- ithkuil 4y ago
  If not doing anything has bad consequences, then it will do something to avoid bad consequences. It seems to me that "want" and "avoid bad consequences" are quite isomorphic
  - alexvoda 4y ago
    It would need "to want" "to avoid bad consequences". It is rather circular because of the word "bad" which implies "do not want". If it is indifferent to all consequences, you are back to square 1, so no, consequences are not the missing magic. Consequences are a function that transform one "want" into a different "want".
    - 4y ago

vintermann 4y ago

You're right that want is important. But what I think you miss (I may be wrong) is that "want" is not an empirical category. Does a cloud want to rain? Or does it just rain because it obeys physical laws as part of a huge complex system? Do I want to have a cup of coffee? Or a job? Or a girlfriend? Or am I likewise just doing things because my atoms obey physical laws as part of a huge complex system? This is not a scientific question. It is a teleological question. That doesn't make it less important, on the contrary. It's the answers to teleological questions which make anything important, including the scientific questions which happen to be important. So the question is not, "what can we do to make a machine want things", it's "When should we ascribe what a machine does to its own wants". And my answer is "never". Not as long as it's a product of our wants, which they always will be.

itworkslikethat 4y ago
Yes, you want to work, you want to have a cup of coffee. You may not like it, but you decided to do it and then executed a very complex series of steps to accomplish the goal. You're not a cloud, working/drinking is not an inevitable and direct physical process you're undergoing by yourself.
- vintermann 4y ago
  I obviously agree that I have both will and wants. But I can't say that the physical processes in me are in any fundamental way different, than physical processes anywhere else in nature. Most are deterministic, some may be inscrutable, and some may even be random or unknowable - but all those things can be said about the weather, too. You cannot derive my wants from my capabilities (unless you're prepared you ascribe wants to clouds as well, I guess).
  - itworkslikethat 4y ago
    The actual physical process of your body using the water once you ingest it is not much different, sure. But you can choose not to drink (and die some days later). A cloud can't choose not to rain.

xtiansimon 4y ago

You first need individual embodiment, then you will want all sorts of things.