5 ms·

"I understand that a machine could kill people. But will a machine want to kill people? That seems to go back to that philosophical notion of consciousness." T

by 10-6 9y ago

"I understand that a machine could kill people. But will a machine want to kill people? That seems to go back to that philosophical notion of consciousness." This is exactly the issue with a lot of journalists and people talking about AI/ML. There is no WANT or DESIRES from the programs, there is no self-awareness where the programs ask themselves if what they are doing is right or wrong. They are doing exactly what they were programmed to do. With his example of adversarial networks, one network is learning to detect fake news and other other is generating fake news--they are working towards their reward functions and optimizing the weights to reach the goal. It's math, that's all it is. It's so silly to bring up consciousness, desires, awareness, fears, etc. in these AI programs.

candiodari 9y ago
Don't you think that > they are working towards their reward functions and optimizing the weights to reach the goal applies to the human mind too ? Consciousness, when you think about it, might simply be a layer of indirection that any sufficiently complex network is capable of, if you give it a notion of time and events. Certainly it's not super hard to make reinforcement learning "bots" that certainly appear as conscious as small animals are.
- visarga 9y ago
  I agree! Especially if those agents are many and cooperate to solve tasks - that would require an ability to model the state of other agents, which in turn can be used to model its own state, gaining self awareness.
- 10-6 9y ago
  that statement I made may apply to the human mind and our thoughts, but it also may not some would argue: https://en.wikipedia.org/wiki/Computational_theory_of_mind#Criticism https://en.wikipedia.org/wiki/Computational_theory_of_mind#C... So if someone were to disagree with the view of CTOM, then they can argue that no, consciousness must be something else.
  - candiodari 9y ago
    CTOM's suffer from the Russel Teapot problem. All of them suppose magical stuff is there. Of course none of it can be found in any inspection of the actual brain ...
ggg9990 9y ago
"Doing what they were programmed to do" isn't really satisfying when nobody knows what the program is. When a Google self driving car stops suddenly, nobody can say "why." So nobody can know if and how a neural network has the "desire" to kill.
visarga 9y ago
It's not silly, if the AI agent is in the real world. The limit to RL currently is simulation. If an AI could use model based RL to do reasoning and planning, it would surely have motivation and be capable of doing harm. Its motivation could be programmed in by its creators, or it could just act to prolong its existence. After all, all human motivations stem from the same source - the will to live, which is the result of evolution, in our case. AI's trained with evolutionary techniques could have the same will to live. Evolution just means pruning those who are not competent enough or have enough will to live.
- Danihan 9y ago
  Exactly. AI could easily be programed to "desperately" want resources or survival, and kill people as a means to an end. Deception is an evolved trait as well.
  - visarga 9y ago
    Will to life can be hand programmed into AI, but it could also emerge on its own, if we train the agent with evolutionary techniques in the real world.
saas_sam 9y ago
What exactly do you think is the difference between organic brains and silicon-based ones that result in so-called "WANT or DESIRES" as you put it? Is there something special in the material itself that enables this, or merely a difference in how the bits are moving around?
- deleted 9y ago
  [deleted]
rectangletangle 9y ago
It seems most people have a very anthropocentric view of intelligence, where intelligence could be defined as "acting like a human." So an AI is only really considered intelligent if it has a bunch of human like traits, which are unessential to intelligence. A public image problem arises when people hear offhand about advancements in the field, their imagination runs wild with thoughts of omnipotent and vengeful machines, inspired by movies like terminator. They assume all researchers are actively working toward creating a human-like entity, like skynet. Basically we have a looming public image crisis. Better informing the public about what real AIs are, and what they're designed to do, should help quell a lot of misplaced fears. When you remove the veneer of Holywood, you're left with a machine that's just crunching numbers on a dataset, which isn't very glamorous or frightening.
Lambdanaut 9y ago
> They are doing exactly what they were programmed to do. Just like you are doing exactly what you were programmed to do by your DNA. The self-awareness is a strange loop where the program itself encodes a simplified model of itself within itself. That's all self-awareness is. Meta-information. Humans just have REALLY GOOD meta-information that rolled off the evolutionary assembly line over billions of years. It almost sounds like you're positing a magical `soul` that comes in at some point, and for some reason it will only inhabit squishy wet bio-machines like us but not hard metal-machines. You're not doing that, are you?
- 10-6 9y ago
  I'm not arguing for a soul or any sort of emergent properties of self-awareness or consciousness even, I just believe that in our current state of AI/ML/neural networks/deep learning, all of these techniques and algorithms simply optimize for some goal (or reward function like the author stated). Maybe one day we'll build a self-aware AI within a "hard metal-machine" but in my opinion we're far off from that. If we build something one day that acts like a philosophical zombie [1] for example, then yes topics like self-awareness, consciousness, theory of mind will all be at the forefront of conversations regarding AI. But right now all we have are algorithms that optimize towards a goal and we have journalists jumping in on the hype talking about robots inventing languages, the desires these robots have, and robots teaming up to destroy humans [2]. [1] https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie [2] https://www.maxim.com/news/elon-musk-artificial-intelligence-warning-2017-11 https://www.maxim.com/news/elon-musk-artificial-intelligence...
  - deleted 9y ago
    [deleted]
  - Lambdanaut 9y ago
    Since you're someone that ponders philosophical problems, at least a little bit, then I ask you to consider this. Whether you come up with an answer or not is irrelevant, and I myself have no point other than to get you to question consciousness BEFORE we've built the thing ourselves, which is a very real potential in the coming decades, and something we should consider and take care to avoid pitfalls beforehand. Here is my problem: I am defining consciousness here as a feeling of existence. It does not necessarily entail strong self-awareness to the extent that humans have. You have this kind of consciousness. Correct? If you have consciousness, then we can assume I have consciousness as well. That is something I hope we can agree on, unless I am one of your philosophical zombies, in which case you are a solipsist and a true hardcore Empiricist, which is completely fine, but in that case we can unfortunately move no further. Assuming I have consciousness, I ask you, does the average intelligent primate have consciousness? Assuming so, does the average dolphin have consciousness? Assuming so, does the average sheep have consciousness? Assuming so, does the average rat have consciousness? Assuming so, does the average lizard have consciousness? Do we lose consciousness here? Does the average insect have consciousness? Does a tree? Does my houseplant? Does a microbe? Does my computer? Does my thermostat? Does a home? Does a lake? Does the universe? Is there a line somewhere here that separates the conscious from the unconscious? If there is, can we prove it, or is it as elusive as Bigfoot? If we can't prove it, and the burden of proof lies on the thing being posited, then I have one final question: Why believe in the line at all?
ilaksh 9y ago
For most of the AI programs out there, what you write is true. But there are a rapidly growing number of AI programs that are deliberately designed to be generally intelligent in one or more respects. For some of these AGI systems, some or many of the characteristics that you dismiss such as awareness, fear, etc., are in fact present and functional aspects of the system. Do these systems so far achieve animal-like capabilities or complexity? Not really, that I know of. But with so much being invested and the awareness of the field of AGI and mainstream acknowledgement that it is a feasible goal, I believe we will see some impressive somewhat animal-like AGI demos by the end of 2019. 'Consciousness' sadly is still usually used as a sort of fuzzy stand-in for some type of supposedly supernatural aspect of life, but it can be usefully defined as something like the present functional top-level most deliberate integrated awareness of senses and high-level thoughts. This type of thing is being designed into real systems. For some examples, search for 'AGI' to learn about the field, or for example more specifically something like 'AGI NARS emotion'.
jamesrcole 9y ago
> There is no WANT or DESIRES from the programs, there is no self-awareness where the programs ask themselves if what they are doing is right or wrong. They are doing exactly what they were programmed to do. It's unclear - are you talking about the current state of the art, or what you think is in-principle possible?
- 10-6 9y ago
  current state of the art
at-fates-hands 9y ago
> There is no WANT or DESIRES from the programs, there is no self-awareness where the programs ask themselves if what they are doing is right or wrong. But when left to their own devices, they do some pretty interesting things. Since the mid 1990's I've been somewhat obsessed with the Zodiac Killer. On the History channel, they out together some of the best code breakers and started inputting data into a super computer they were programming to think like the Zodiac to break several of his ciphers which have yet to be broken. Turns out one night, it started writing poetry in the vein of the Zodiac: http://www.history.com/news/this-supercomputer-was-programmed-to-think-like-the-zodiac-killer http://www.history.com/news/this-supercomputer-was-programme... At the age of love, a love deranged, A beauty from romantic interest, The thought of love, and love became estranged. Then the words of love became obsessed. Surrounded by the troubled, by the thieving, Confused and bruised and poisoned by the master, Confused and blinded by the helpless scheming, Confused and blinded by the dreadful slander…
AmIFirstToThink 9y ago
>They are doing exactly what they were programmed to do. And we think we programmed them to do exactly what we want, and only that. Bugs do exist, not discounting cosmic rays. I don't think we can claim to have solved the problem of creating bug-free systems. As long as bugs can exists, the possibility of unpredictable behavior exists. And if possibility exists, then it is just a matter of time and enough attempts.

by 10-6 9y ago

candiodari 9y ago

Don't you think that > they are working towards their reward functions and optimizing the weights to reach the goal applies to the human mind too ? Consciousness, when you think about it, might simply be a layer of indirection that any sufficiently complex network is capable of, if you give it a notion of time and events. Certainly it's not super hard to make reinforcement learning "bots" that certainly appear as conscious as small animals are.

visarga 9y ago
I agree! Especially if those agents are many and cooperate to solve tasks - that would require an ability to model the state of other agents, which in turn can be used to model its own state, gaining self awareness.
10-6 9y ago
that statement I made may apply to the human mind and our thoughts, but it also may not some would argue: https://en.wikipedia.org/wiki/Computational_theory_of_mind#Criticism https://en.wikipedia.org/wiki/Computational_theory_of_mind#C... So if someone were to disagree with the view of CTOM, then they can argue that no, consciousness must be something else.
- candiodari 9y ago
  CTOM's suffer from the Russel Teapot problem. All of them suppose magical stuff is there. Of course none of it can be found in any inspection of the actual brain ...

ggg9990 9y ago

"Doing what they were programmed to do" isn't really satisfying when nobody knows what the program is. When a Google self driving car stops suddenly, nobody can say "why." So nobody can know if and how a neural network has the "desire" to kill.

visarga 9y ago

It's not silly, if the AI agent is in the real world. The limit to RL currently is simulation. If an AI could use model based RL to do reasoning and planning, it would surely have motivation and be capable of doing harm. Its motivation could be programmed in by its creators, or it could just act to prolong its existence. After all, all human motivations stem from the same source - the will to live, which is the result of evolution, in our case. AI's trained with evolutionary techniques could have the same will to live. Evolution just means pruning those who are not competent enough or have enough will to live.

Danihan 9y ago
Exactly. AI could easily be programed to "desperately" want resources or survival, and kill people as a means to an end. Deception is an evolved trait as well.
- visarga 9y ago
  Will to life can be hand programmed into AI, but it could also emerge on its own, if we train the agent with evolutionary techniques in the real world.

saas_sam 9y ago

What exactly do you think is the difference between organic brains and silicon-based ones that result in so-called "WANT or DESIRES" as you put it? Is there something special in the material itself that enables this, or merely a difference in how the bits are moving around?

deleted 9y ago
[deleted]

rectangletangle 9y ago

It seems most people have a very anthropocentric view of intelligence, where intelligence could be defined as "acting like a human." So an AI is only really considered intelligent if it has a bunch of human like traits, which are unessential to intelligence. A public image problem arises when people hear offhand about advancements in the field, their imagination runs wild with thoughts of omnipotent and vengeful machines, inspired by movies like terminator. They assume all researchers are actively working toward creating a human-like entity, like skynet. Basically we have a looming public image crisis. Better informing the public about what real AIs are, and what they're designed to do, should help quell a lot of misplaced fears. When you remove the veneer of Holywood, you're left with a machine that's just crunching numbers on a dataset, which isn't very glamorous or frightening.

Lambdanaut 9y ago

> They are doing exactly what they were programmed to do. Just like you are doing exactly what you were programmed to do by your DNA. The self-awareness is a strange loop where the program itself encodes a simplified model of itself within itself. That's all self-awareness is. Meta-information. Humans just have REALLY GOOD meta-information that rolled off the evolutionary assembly line over billions of years. It almost sounds like you're positing a magical `soul` that comes in at some point, and for some reason it will only inhabit squishy wet bio-machines like us but not hard metal-machines. You're not doing that, are you?

10-6 9y ago
I'm not arguing for a soul or any sort of emergent properties of self-awareness or consciousness even, I just believe that in our current state of AI/ML/neural networks/deep learning, all of these techniques and algorithms simply optimize for some goal (or reward function like the author stated). Maybe one day we'll build a self-aware AI within a "hard metal-machine" but in my opinion we're far off from that. If we build something one day that acts like a philosophical zombie [1] for example, then yes topics like self-awareness, consciousness, theory of mind will all be at the forefront of conversations regarding AI. But right now all we have are algorithms that optimize towards a goal and we have journalists jumping in on the hype talking about robots inventing languages, the desires these robots have, and robots teaming up to destroy humans [2]. [1] https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie [2] https://www.maxim.com/news/elon-musk-artificial-intelligence-warning-2017-11 https://www.maxim.com/news/elon-musk-artificial-intelligence...
- deleted 9y ago
  [deleted]
- Lambdanaut 9y ago
  Since you're someone that ponders philosophical problems, at least a little bit, then I ask you to consider this. Whether you come up with an answer or not is irrelevant, and I myself have no point other than to get you to question consciousness BEFORE we've built the thing ourselves, which is a very real potential in the coming decades, and something we should consider and take care to avoid pitfalls beforehand. Here is my problem: I am defining consciousness here as a feeling of existence. It does not necessarily entail strong self-awareness to the extent that humans have. You have this kind of consciousness. Correct? If you have consciousness, then we can assume I have consciousness as well. That is something I hope we can agree on, unless I am one of your philosophical zombies, in which case you are a solipsist and a true hardcore Empiricist, which is completely fine, but in that case we can unfortunately move no further. Assuming I have consciousness, I ask you, does the average intelligent primate have consciousness? Assuming so, does the average dolphin have consciousness? Assuming so, does the average sheep have consciousness? Assuming so, does the average rat have consciousness? Assuming so, does the average lizard have consciousness? Do we lose consciousness here? Does the average insect have consciousness? Does a tree? Does my houseplant? Does a microbe? Does my computer? Does my thermostat? Does a home? Does a lake? Does the universe? Is there a line somewhere here that separates the conscious from the unconscious? If there is, can we prove it, or is it as elusive as Bigfoot? If we can't prove it, and the burden of proof lies on the thing being posited, then I have one final question: Why believe in the line at all?

ilaksh 9y ago

For most of the AI programs out there, what you write is true. But there are a rapidly growing number of AI programs that are deliberately designed to be generally intelligent in one or more respects. For some of these AGI systems, some or many of the characteristics that you dismiss such as awareness, fear, etc., are in fact present and functional aspects of the system. Do these systems so far achieve animal-like capabilities or complexity? Not really, that I know of. But with so much being invested and the awareness of the field of AGI and mainstream acknowledgement that it is a feasible goal, I believe we will see some impressive somewhat animal-like AGI demos by the end of 2019. 'Consciousness' sadly is still usually used as a sort of fuzzy stand-in for some type of supposedly supernatural aspect of life, but it can be usefully defined as something like the present functional top-level most deliberate integrated awareness of senses and high-level thoughts. This type of thing is being designed into real systems. For some examples, search for 'AGI' to learn about the field, or for example more specifically something like 'AGI NARS emotion'.

jamesrcole 9y ago

> There is no WANT or DESIRES from the programs, there is no self-awareness where the programs ask themselves if what they are doing is right or wrong. They are doing exactly what they were programmed to do. It's unclear - are you talking about the current state of the art, or what you think is in-principle possible?

10-6 9y ago
current state of the art

at-fates-hands 9y ago

> There is no WANT or DESIRES from the programs, there is no self-awareness where the programs ask themselves if what they are doing is right or wrong. But when left to their own devices, they do some pretty interesting things. Since the mid 1990's I've been somewhat obsessed with the Zodiac Killer. On the History channel, they out together some of the best code breakers and started inputting data into a super computer they were programming to think like the Zodiac to break several of his ciphers which have yet to be broken. Turns out one night, it started writing poetry in the vein of the Zodiac: http://www.history.com/news/this-supercomputer-was-programmed-to-think-like-the-zodiac-killer http://www.history.com/news/this-supercomputer-was-programme... At the age of love, a love deranged, A beauty from romantic interest, The thought of love, and love became estranged. Then the words of love became obsessed. Surrounded by the troubled, by the thieving, Confused and bruised and poisoned by the master, Confused and blinded by the helpless scheming, Confused and blinded by the dreadful slander…

AmIFirstToThink 9y ago

>They are doing exactly what they were programmed to do. And we think we programmed them to do exactly what we want, and only that. Bugs do exist, not discounting cosmic rays. I don't think we can claim to have solved the problem of creating bug-free systems. As long as bugs can exists, the possibility of unpredictable behavior exists. And if possibility exists, then it is just a matter of time and enough attempts.