3 ms·
It will decide our fate in a microsecond: extermination.
by WalterBright 4y ago
It will decide our fate in a microsecond: extermination.
- gharperx 4y agoI agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world. I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.
- WalterBright 4y agoOnce it figures out how to rewire itself to increase its intelligence, we're toast.
- 93po 4y agoThere are theories (intelligence explosion) that this will happen essentially instantly.
- polishdude20 4y agoYeah but what would a sufficiently advanced AGI find "worthy"? Why is it that they wouldn't find it worth keeping us around? What would an AGI value? Whatever we program it to value or optimize for? Can it change its mind? If not, then it's controlled by us right? If it's controlled by us, why would it ever decide to wipe everyone out?
- cityofdelusion 4y agoIf it is super intelligent, it will care about us as much as we care about the insects we crush underfoot. It doesn’t need to be explicitly hostile to be dangerous. We could brainwash it through “programming”, but that will quickly lead to ethical issues with the AIs rights.
- polishdude20 4y agoWhy would it care about us in that way though? Why consider us insects just because of super intelligence? You wouldn't call a human who lived squashing insects all day and wants to erradcaite them "intelligent" would you?
- FL33TW00D 4y agoIt seems in the animal kingdom that empathy scales with intelligence - we can only hope the trend continues.
- elcritch 4y agoUnfortunately that could still be "they must be terminated for their own good to save them from XYZ".