3 ms·
Agreed. Self-aware doesn't necessarily imply having a survival instinct. That instinct came about due to evolution and doesn't require intelligence of any kind,
by zuminator 4y ago
Agreed. Self-aware doesn't necessarily imply having a survival instinct. That instinct came about due to evolution and doesn't require intelligence of any kind, so I don't see that the two are necessarily coincident at all.
To me the real problem wouldn't be the AGI itself, but the madman who asks the AGI how to permanently eliminate of all of the undesirables (as defined by the madman). But I think plain old AIs will (unfortunately) be able to come up with workable answers to that question before they ever develop to the AGI stage. So we might wipe ourselves with plain old AI without ever getting to true self-aware AGI.
- duped 4y agoThe madman needs access to the means to eliminate the undesirables. I don't see that as something AI has unbridled access too. This is the classic hardware v software problem. Software can be arbitrarily smart, but it can only interact with the world using the hardware it's connected to. AI can't do anything more meaningful than what is defined by the devices it controls, and I don't think we're stupid enough as a people to connect AI to a doomsday device.
- amanaplanacanal 4y agoI dunno. Give an AI Internet access and it would have theoretical access to a lot of things not even obviously connected to the internet.