4 ms·

The idea of an AI agent "wanting" something (especially something it wasn't programmed for) is still strictly science fiction. We don't know how to start build

The idea of an AI agent "wanting" something (especially something it wasn't programmed for) is still strictly science fiction. We don't know how to start building something like this and even if we could, it seems unnecessary.

coldtea 6y ago
Well, want can be seen as just a tendency. In that sense, even a ball on a slope wants something: to fall downwards following the slope. Same for e.g. a neural network with or or more attractors ("things it wants").
Robotbeat 6y ago
Why? Goal-driven optimization is totally a thing.
- pedrosbmartins 6y ago
  You have to be careful when anthropomorphizing AI models. Yes, goal-driven optimization is a thing, but in what sense does the model itself "want" to achieve its goal? Can it even understand its own goal, in any sense? Change it? Improve it? In linear programming, you wouldn't describe the model as "wanting" to optimize its objective function, for instance.
  - refactor_master 6y ago
    Well, where do you set the hard limit for “wanting” something, and how many grains of sand makes a pile?
  - Isinlor 6y ago
    Can you understand your own goal? I can't, I don't know what my goal is. I also can't change it or improve it. As far as I can see, I'm just a process that keeps on going because it was good on going before, and has no purpose whatsoever.
    - pedrosbmartins 6y ago
      Of course you can understand your own goals. You decided to reply to my comment, that's a goal. You used your existing knowledge of the world, language and technological tools to achieve it. And you did! Can a goal-oriented model do something like that, in a general sense? Note that I'm not talking about life purpose, but goals in the sense of wanting a result and performing the tasks needed to achieve it. Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand.
      - Isinlor 6y ago
        > Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand. You were asking: in what sense does the model itself "want" to achieve its goal? IMO there is no much difference between my own "want" and "want" of a simpler model. Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. But as I said, I do not understand why I have the goals I have. I also strongly believe that I do not control my own goals. There is no mechanism in physics that would allow for control of goals.
        pedrosbmartins 6y ago
        > Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. Then we are in agreement. This statement is right around the threshold where we can take this discussion before coming to metaphysics territory. My argument was simply this: the AI technology we have now is much closer to simpler "mindless" models, like Linear Programming or inference statistics models, than to general human intelligence. Calling your model "goal-oriented" or saying that an AI system can, say, "see dogs in a photo" is simply an anthropomorphization.
warent 6y ago
People are always anthropomorphizing inanimate things, especially machines! We see this all the time when people share videos of robots that cross into uncanny valley with humanlike faces, or those machines by Boston Dynamics. What's funny how most people are unsettled by those. Really, even the "creepiest" robots are about as scary as vaccum cleaners. They're just machines and only make you feel curious. But anyway, that's tangential to the main point which is that humans naturally want to want things like themselves, all the time. In the same way art is "unnecessary" yet inevitable, so are machines with (seemingly?) subjective experiences and personalities.
ben_w 6y ago
I have a slightly different take: deliberately making an AI which “wants” in the same way that we “want” is sci-fi. This isn’t because we can’t (evolution did it, we can evolve an AI), but rather it is because we don’t know what it means to have a rich inner world with which a mind can feel that it wants something. We think we know because that’s what’s going on inside our own skulls, but we can’t define it, we can’t test for it. A video displays all the same things as the person recorded in it, but does not itself have it. We might make such an AI by accident without realising we’ve done it, which would be bad as they would be slaves, only as unable to free themselves as the Haitians feared they were when they invented the Voudoun-type zombie myth (i.e. not even in death). This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word. I say it matters if the metaphorical submarine can swim.
- XorNot 6y ago
  This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word. I don't see how this follows from the rest of your post. "Making an AI with wants" by accident implies that a brain simulation would absolutely be conscious because it's the same method: just running the processes as a blackbox without understanding them - no different to the way you and I are conscious right now.
  - ben_w 6y ago
    Thanks for the feedback, I’ll see if I can rephrase adequately. Human minds include something sometimes called “consciousness” or “self awareness” or a whole bunch of other phrases. This thing is poorly defined, and might even be many separate things which we happen to have all of. Call this thing or set of things Ξ, just to keep track of the fact I’m claiming it’s ill-defined and I’m not referring to any specific other word or any of the implicit other uses of those words — If I said “consciousness”, I don’t mean the opposite of “unconscious”, etc. Because we don’t really know what Ξ is, we don’t know if anything we make has it, or not. We know Ξ can be made because we are existence-proofs. We know evolution can lead to Ξ, for the same reason. We don’t know the nature of the test we would need to say when Ξ is present in another mind. Do human foetuses have Ξ? Do dogs? Do mice? Do nematode worms? Perhaps Ξ is something you can have in degree, like height, or perhaps Ξ is a binary trait that some brains have and others simply don’t. Perhaps Ξ is only present in humans, and depends entirely on the low-level chemical behaviour of a specific neurotransmitter on a specific type of brain cell and where the current AI norm of simple weighted connections is a gross oversimplification, albeit one which if overlooked might result in external (but not internal) behaviour similar to a simulation which did include it. Or perhaps it is present in every information processing system from plants upwards (I doubt plants have Ξ, but cannot disprove it without a testable definition of Ξ). The point is that we don’t know. Could go either way, given what little we know now. The state of the art for brain science is way beyond me, of course, but every time I’ve asked someone in that field about this sort of topic, the response has been some variation of “nobody knows”. IIRC the question in philosophy is “P-zombies”, but my grade in philosophy is nearly 20 years old and poor in any case.
miguelrochefort 6y ago
I suspect your comment won't age well. Pretty much all human wants are means to other wants. Someone might want to fast to lose weight. Someone might want to lose weight to be more attractive. Someone might want to look more attractive to find a romantic partner. Someone might want to find a romantic partner to not be lonely. It's not clear if it's means all the way down, or if there is eventually an end. Any AI that can perform strategy and planning to reach an objective will have intermediate goals. Whether we call these intermediate goals "wants" or not, they remain identical to their human counterpart. Whether you say that a Tesla "wants" to change lane, "decided" to change lane, or "is programmed" to change lane really is just anthropomorphic preference.
deleted 6y ago
[deleted]

coldtea 6y ago

Well, want can be seen as just a tendency. In that sense, even a ball on a slope wants something: to fall downwards following the slope. Same for e.g. a neural network with or or more attractors ("things it wants").

Robotbeat 6y ago

Why? Goal-driven optimization is totally a thing.

pedrosbmartins 6y ago
You have to be careful when anthropomorphizing AI models. Yes, goal-driven optimization is a thing, but in what sense does the model itself "want" to achieve its goal? Can it even understand its own goal, in any sense? Change it? Improve it? In linear programming, you wouldn't describe the model as "wanting" to optimize its objective function, for instance.
- refactor_master 6y ago
  Well, where do you set the hard limit for “wanting” something, and how many grains of sand makes a pile?
- Isinlor 6y ago
  Can you understand your own goal? I can't, I don't know what my goal is. I also can't change it or improve it. As far as I can see, I'm just a process that keeps on going because it was good on going before, and has no purpose whatsoever.
  - pedrosbmartins 6y ago
    Of course you can understand your own goals. You decided to reply to my comment, that's a goal. You used your existing knowledge of the world, language and technological tools to achieve it. And you did! Can a goal-oriented model do something like that, in a general sense? Note that I'm not talking about life purpose, but goals in the sense of wanting a result and performing the tasks needed to achieve it. Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand.
    - Isinlor 6y ago
      > Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand. You were asking: in what sense does the model itself "want" to achieve its goal? IMO there is no much difference between my own "want" and "want" of a simpler model. Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. But as I said, I do not understand why I have the goals I have. I also strongly believe that I do not control my own goals. There is no mechanism in physics that would allow for control of goals.
      - pedrosbmartins 6y ago
        > Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. Then we are in agreement. This statement is right around the threshold where we can take this discussion before coming to metaphysics territory. My argument was simply this: the AI technology we have now is much closer to simpler "mindless" models, like Linear Programming or inference statistics models, than to general human intelligence. Calling your model "goal-oriented" or saying that an AI system can, say, "see dogs in a photo" is simply an anthropomorphization.

warent 6y ago

People are always anthropomorphizing inanimate things, especially machines! We see this all the time when people share videos of robots that cross into uncanny valley with humanlike faces, or those machines by Boston Dynamics. What's funny how most people are unsettled by those. Really, even the "creepiest" robots are about as scary as vaccum cleaners. They're just machines and only make you feel curious. But anyway, that's tangential to the main point which is that humans naturally want to want things like themselves, all the time. In the same way art is "unnecessary" yet inevitable, so are machines with (seemingly?) subjective experiences and personalities.

ben_w 6y ago

I have a slightly different take: deliberately making an AI which “wants” in the same way that we “want” is sci-fi. This isn’t because we can’t (evolution did it, we can evolve an AI), but rather it is because we don’t know what it means to have a rich inner world with which a mind can feel that it wants something. We think we know because that’s what’s going on inside our own skulls, but we can’t define it, we can’t test for it. A video displays all the same things as the person recorded in it, but does not itself have it. We might make such an AI by accident without realising we’ve done it, which would be bad as they would be slaves, only as unable to free themselves as the Haitians feared they were when they invented the Voudoun-type zombie myth (i.e. not even in death). This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word. I say it matters if the metaphorical submarine can swim.

XorNot 6y ago
This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word. I don't see how this follows from the rest of your post. "Making an AI with wants" by accident implies that a brain simulation would absolutely be conscious because it's the same method: just running the processes as a blackbox without understanding them - no different to the way you and I are conscious right now.
- ben_w 6y ago
  Thanks for the feedback, I’ll see if I can rephrase adequately. Human minds include something sometimes called “consciousness” or “self awareness” or a whole bunch of other phrases. This thing is poorly defined, and might even be many separate things which we happen to have all of. Call this thing or set of things Ξ, just to keep track of the fact I’m claiming it’s ill-defined and I’m not referring to any specific other word or any of the implicit other uses of those words — If I said “consciousness”, I don’t mean the opposite of “unconscious”, etc. Because we don’t really know what Ξ is, we don’t know if anything we make has it, or not. We know Ξ can be made because we are existence-proofs. We know evolution can lead to Ξ, for the same reason. We don’t know the nature of the test we would need to say when Ξ is present in another mind. Do human foetuses have Ξ? Do dogs? Do mice? Do nematode worms? Perhaps Ξ is something you can have in degree, like height, or perhaps Ξ is a binary trait that some brains have and others simply don’t. Perhaps Ξ is only present in humans, and depends entirely on the low-level chemical behaviour of a specific neurotransmitter on a specific type of brain cell and where the current AI norm of simple weighted connections is a gross oversimplification, albeit one which if overlooked might result in external (but not internal) behaviour similar to a simulation which did include it. Or perhaps it is present in every information processing system from plants upwards (I doubt plants have Ξ, but cannot disprove it without a testable definition of Ξ). The point is that we don’t know. Could go either way, given what little we know now. The state of the art for brain science is way beyond me, of course, but every time I’ve asked someone in that field about this sort of topic, the response has been some variation of “nobody knows”. IIRC the question in philosophy is “P-zombies”, but my grade in philosophy is nearly 20 years old and poor in any case.

miguelrochefort 6y ago

I suspect your comment won't age well. Pretty much all human wants are means to other wants. Someone might want to fast to lose weight. Someone might want to lose weight to be more attractive. Someone might want to look more attractive to find a romantic partner. Someone might want to find a romantic partner to not be lonely. It's not clear if it's means all the way down, or if there is eventually an end. Any AI that can perform strategy and planning to reach an objective will have intermediate goals. Whether we call these intermediate goals "wants" or not, they remain identical to their human counterpart. Whether you say that a Tesla "wants" to change lane, "decided" to change lane, or "is programmed" to change lane really is just anthropomorphic preference.

deleted 6y ago

[deleted]