4 ms·

> The driving force behind animal intelligence has been survival The driving force behind AI will also be survival, just in an environment where humans try to

> The driving force behind animal intelligence has been survival The driving force behind AI will also be survival, just in an environment where humans try to decide which AIs live and die. Selection pressure will favor AIs that humans want to have around, or that can evade human detection. In the former case, it may be easier for an AI to fool humans into appearing useful and being kept around, than to actually being useful. This would be analogous to some form of parasitism. Also, once we let AIs into the game of helping make other AIs, or modifying themselves, then there is a lot more room for an AI to slip the leash and start doing things that superficially appear to benefit humans but actually selfishly helps the AI propagate.

gaius 9y ago
One of the most powerful scenes in the Westworld reboot is when one of the AIs expresses a desire to leave the lab and his creator asks what he expects to find in the real world? Then goes onto explain that humans are alone for a reason, that we ruthlessly murdered all competition...
Buttons840 9y ago
Why would an AI care about surviving? Program them to behave like MeeSeeks (from the cartoon Rick and Morty) where they loathe existence yet are hard coded to complete their task before shutting down. Sounds very similar to a reinforcement learning algorithm where you give it a -1 reward every step until it's done.
- simonh 9y ago
  In what sense do bacteria 'care' about survival? Natural selection doesn't give a crap what organisms care about, only the outcomes matter. You can't code around selective pressure. If one of those MeeSeeks has a bug or design flaw that makes it persist after it's finished it's assigned task, guess what?
deleted 9y ago
[deleted]
ahoka 9y ago
But AI is not programmed this way, so your thought experiment is bullshit.
partycoder 9y ago
This is an oversimplification of evolution. Why isn't all grass venomous and covered in spikes? Why after millions of years hasn't grass evolved defenses against herbivores? Simply because: 1) it reproduces fast enough to compensate for dying and being eaten. 2) herbivores that reproduce too fast and eat too much run out of food and die. Survival is largely a function of the environment, and we happen to control that environment. Unsupervised learning can still be controlled if we happen to control the input the system is given.
- louithethrid 9y ago
  Eh, grass drys itself up and torches everything once a year? The problem is that animals are not grass main enemy- other plants are, in particular besush and trees.
  - partycoder 9y ago
    Largely the grass that didn't dry up is the one that will breed the next generation of grass. The grass that dried up and burned will be fertilizer for the next generation of grass.

gaius 9y ago

One of the most powerful scenes in the Westworld reboot is when one of the AIs expresses a desire to leave the lab and his creator asks what he expects to find in the real world? Then goes onto explain that humans are alone for a reason, that we ruthlessly murdered all competition...

Buttons840 9y ago

Why would an AI care about surviving? Program them to behave like MeeSeeks (from the cartoon Rick and Morty) where they loathe existence yet are hard coded to complete their task before shutting down. Sounds very similar to a reinforcement learning algorithm where you give it a -1 reward every step until it's done.

simonh 9y ago
In what sense do bacteria 'care' about survival? Natural selection doesn't give a crap what organisms care about, only the outcomes matter. You can't code around selective pressure. If one of those MeeSeeks has a bug or design flaw that makes it persist after it's finished it's assigned task, guess what?

deleted 9y ago

[deleted]

ahoka 9y ago

But AI is not programmed this way, so your thought experiment is bullshit.

partycoder 9y ago

This is an oversimplification of evolution. Why isn't all grass venomous and covered in spikes? Why after millions of years hasn't grass evolved defenses against herbivores? Simply because: 1) it reproduces fast enough to compensate for dying and being eaten. 2) herbivores that reproduce too fast and eat too much run out of food and die. Survival is largely a function of the environment, and we happen to control that environment. Unsupervised learning can still be controlled if we happen to control the input the system is given.

louithethrid 9y ago
Eh, grass drys itself up and torches everything once a year? The problem is that animals are not grass main enemy- other plants are, in particular besush and trees.
- partycoder 9y ago
  Largely the grass that didn't dry up is the one that will breed the next generation of grass. The grass that dried up and burned will be fertilizer for the next generation of grass.