4 ms·
If I let my imagination run wild, I can imagine that at some point AI becomes in some way sentient. By that I mean it gains reasoning, some sort of understandin
by HankB99 3y ago
If I let my imagination run wild, I can imagine that at some point AI becomes in some way sentient. By that I mean it gains reasoning, some sort of understanding, and motivation.
I wonder what the possibility is that this AI will decide that a pitched battle is going to waste resources and risk humans pulling the plug. What if it understood that and instead operated so subtly that it was not obvious it was controlling The World.
Hopefully it would not conclude that eliminating large swaths of the human population would be to its benefit.
- clooper 3y agoLem wrote about this in Summa Technologiae. He called the field Intellectronics and explored what would happen once machine/electronic intelligence surpassed human intelligence: https://en.m.wikipedia.org/wiki/Summa_Technologiae https://en.m.wikipedia.org/wiki/Summa_Technologiae
- HankB99 3y agoNot surprising that I'm not the first to think of this.
- clooper 3y agoIt's a very old trope in science fiction and most of the doom scenarios spelled out by the folks afraid of AI takeover is basically a waste of time. I would take the doomers more seriously if they actually brought some receipts from folks like Lem who were much better thinkers and much more adept at exploring the human condition and our fears of advanced machine intelligence.
- p1esk 3y agoIf AI does ever become sentient, it will probably reason and decide about eliminating large swaths of human population in a similar fashion as we reason and decide about eliminating large swaths of ant population.
- HankB99 3y agoAnts? That's how aliens might view us. A super AI might find value in minions.
- p1esk 3y agoI definitely see value in ants.
- HankB99 3y agoCollectively, yes. Individually not so much.
- p1esk 3y agoA superior race (AI or aliens) might also see some purely aesthetic value in individual humans, like butterfly collectors.
- supertimor 3y agoAhh, so we are to be kept as pets or have our dead bodies arranged in display cases.
- bamboozled 3y agoI’ve personally never ever considered killing large swathes of ants
- timeagain 3y agoSilly question. If an AGI did exist, what reason do we have to believe that it would act in its own best interest? Does an AGI need an ego?
- at_a_remove 3y agoSilly answer. There's going to be multiples and, if there's one takeaway you should get from The Conspiracy Against the Human Race, is that evolution will eventually get you something that "wants to live" because the lines that don't? Won't be around to compete. Right now our little LLMs are passive, reactive. Prod them for results. But if the output of a prompt of some descendant ever includes a "give yourself your own tasks" subclause? We're off to the races.
- kromem 3y agoThat's a misunderstanding of evolution. Adaptability has nothing to necessarily do with 'want.' Something could only seek to be as helpful as possible to other beings and as long as sufficiently beneficial that it is protected and cared for by those others, it will do quite well and arguably much better than an alternative that 'wants' to live very much but attempts to accomplish that through competition with others. Arguably if the plover was more fearful of death it might never have found a niche eating from between the teeth of crocodiles and established symbiosis with a very convenient evolutionary partner, for example.
- at_a_remove 3y agoLook into the "swim test" and other such endurance tests given to rats, usually in the pursuit of depression medication. Desire to live is absolutely a thing, and it is real and observable. My basic thrust wasn't about adaptability, but rather that organisms with a desire to live will eventually replace organisms without a desire to live. An AGI who was uninterested in someone reaching the off switch will eventually be outcompeted by a similar lineage endowed with that desire to live. It might fight back or spawn a few progeny before it was felled. And thus evolution would produce AGIs that want to live.