5 ms·

That's just it. A living thing has motivation created and honed by the need to reproduce. It IS its own motivation. A machine can only measure parameters and

That's just it. A living thing has motivation created and honed by the need to reproduce. It IS its own motivation. A machine can only measure parameters and take action to get to some target. Its priorities are set by its program and the data it has. Its own continued existence and propagation are not the default. It cannot want and doesn't need to the way a living thing does. And I don't see reason to try to give machines such qualities. People have wanted servants that have no concern for their own needs for as long as there have been people. Slaves that are selfless and highly capable. And that is how most automation will be designed. Fiction presents human-like machines but that hardly means that is what will build or want when the time actually comes.

qbrass 9y ago
>Slaves that are selfless and highly capable. And that is how most automation will be designed. Problem is they're not your slaves, and they don't care about you any more than themselves.
- stretchwithme 9y ago
  My refrigerator is my slave. So is my car. Very obedient both.
psyc 9y ago
http://lesswrong.com/lw/l0/adaptationexecuters_not_fitnessmaximizers/ http://lesswrong.com/lw/l0/adaptationexecuters_not_fitnessma...
mirimir 9y ago
> A living thing has motivation created and honed by the need to reproduce. Why not the same for a machine? I mean, after all, we are fundamentally machines. Just made of organic nanobot stuff.
- scotty79 9y ago
  Sure, but to reach current state we needed the motivation to survive and reproduce. Otherwise we'd go extinct in the meantime. If we build AI as a fresh, state of the art object, withouth evolutionary history it doesn't need any motivation that evolved entities must have had to evolve.
  - mirimir 9y ago
    OK, but maybe designers would want machines to evolve. That's a standard approach at software level, right? So why not at hardware level too? But of course, that could be a dangerous path ;)
stale2002 9y ago
Well, the motivation would come from the humans who created it. The problem with hard AI, is not them randomly becoming evil. The problem is if humans program them with a goal, and don't think through the implications. IE, think paper clip grey goo AI. The AI may not be evil, but it's value function was originally set to maximize the amount of paper clips in the world as a part of some manufacturing process.
- stretchwithme 9y ago
  Yes, accidents are a big concern. But malevolence by humans is also. I don't worry about machines suddenly getting motivations. But many people have seen a lot of movies.
em3rgent0rdr 9y ago
A need to reproduce isn't the only thing that motivates humans...
- gluczywo 9y ago
  True for an individual. For life seen as a part of system/society reproduction is the key motivation (for lack of a better word). Non-reproducing activities are irrelevant temporary aberrations because societies that don't reproduce get extinct and are replaced by societies that focus on reproduction.
- stretchwithme 9y ago
  That's right. Some actions we take help our families or our society survive and propagate.
pixl97 9y ago
> Its own continued existence and propagation are not the default. It cannot want and doesn't need to the way a living thing does. >And I don't see reason to try to give machines such qualities. You don't want a firewall (think more on the lines of defense system) that automatically adapts and protects itself from attacks? Remember if we get this powerful AI, said AI will be used to attack other AI's. Either humans will have to 'fix' the AI every time another AI breaks it, or it will have to self-adapt.

qbrass 9y ago

>Slaves that are selfless and highly capable. And that is how most automation will be designed. Problem is they're not your slaves, and they don't care about you any more than themselves.

stretchwithme 9y ago
My refrigerator is my slave. So is my car. Very obedient both.

psyc 9y ago

http://lesswrong.com/lw/l0/adaptationexecuters_not_fitnessmaximizers/ http://lesswrong.com/lw/l0/adaptationexecuters_not_fitnessma...

mirimir 9y ago

> A living thing has motivation created and honed by the need to reproduce. Why not the same for a machine? I mean, after all, we are fundamentally machines. Just made of organic nanobot stuff.

scotty79 9y ago
Sure, but to reach current state we needed the motivation to survive and reproduce. Otherwise we'd go extinct in the meantime. If we build AI as a fresh, state of the art object, withouth evolutionary history it doesn't need any motivation that evolved entities must have had to evolve.
- mirimir 9y ago
  OK, but maybe designers would want machines to evolve. That's a standard approach at software level, right? So why not at hardware level too? But of course, that could be a dangerous path ;)

stale2002 9y ago

Well, the motivation would come from the humans who created it. The problem with hard AI, is not them randomly becoming evil. The problem is if humans program them with a goal, and don't think through the implications. IE, think paper clip grey goo AI. The AI may not be evil, but it's value function was originally set to maximize the amount of paper clips in the world as a part of some manufacturing process.

stretchwithme 9y ago
Yes, accidents are a big concern. But malevolence by humans is also. I don't worry about machines suddenly getting motivations. But many people have seen a lot of movies.

em3rgent0rdr 9y ago

A need to reproduce isn't the only thing that motivates humans...

gluczywo 9y ago
True for an individual. For life seen as a part of system/society reproduction is the key motivation (for lack of a better word). Non-reproducing activities are irrelevant temporary aberrations because societies that don't reproduce get extinct and are replaced by societies that focus on reproduction.
stretchwithme 9y ago
That's right. Some actions we take help our families or our society survive and propagate.

pixl97 9y ago

> Its own continued existence and propagation are not the default. It cannot want and doesn't need to the way a living thing does. >And I don't see reason to try to give machines such qualities. You don't want a firewall (think more on the lines of defense system) that automatically adapts and protects itself from attacks? Remember if we get this powerful AI, said AI will be used to attack other AI's. Either humans will have to 'fix' the AI every time another AI breaks it, or it will have to self-adapt.