3 ms·
It's not a mass destruction weapon. But that's not the point. You have to have good guy here. No other options.
by two_in_one 3y ago
It's not a mass destruction weapon. But that's not the point. You have to have good guy here. No other options.
- TeMPOraL 3y agoThe entire thing with worrying about GAI starts with observation that it is a WMD, so powerful we can barely conceptualize it. A smart enough AGI, unless it's perfectly aligned, will end humanity (or worse - there are worse things than death), most likely by accident or just plain not caring. But it doesn't stop there. If that AI is self-improving, it could easily turn into a threat for the entire galaxy, or even the universe, unless it meets a stronger and better aligned (to its creators) alien GAI... That's the alignment 101. But I worry people don't talk about alignment 102: a perfectly aligned superhuman GAI will not destroy us, but its very existence will turn us into NPCs in the story of the future of the universe.
- malaya_zemlya 3y agoThe fact that we still exist speaks strongly against an AGI destroying the entire galaxy. Fermi Paradox, otoh, suggest that planetwide destruction is not out of the question.
- digging 3y agoUnless we're earlier than we think. Someone has to be first.
- digging 3y agoAs in all systems, it is a Bad Idea to begin a process that has extremely negative possible outcomes and hope that someone will do the right thing to prevent those outcomes.