3 ms·
The entire thing with worrying about GAI starts with observation that it is a WMD, so powerful we can barely conceptualize it. A smart enough AGI, unless it's
by TeMPOraL 4y ago
The entire thing with worrying about GAI starts with observation that it is a WMD, so powerful we can barely conceptualize it.
A smart enough AGI, unless it's perfectly aligned, will end humanity (or worse - there are worse things than death), most likely by accident or just plain not caring. But it doesn't stop there. If that AI is self-improving, it could easily turn into a threat for the entire galaxy, or even the universe, unless it meets a stronger and better aligned (to its creators) alien GAI...
That's the alignment 101. But I worry people don't talk about alignment 102: a perfectly aligned superhuman GAI will not destroy us, but its very existence will turn us into NPCs in the story of the future of the universe.
- malaya_zemlya 4y agoThe fact that we still exist speaks strongly against an AGI destroying the entire galaxy. Fermi Paradox, otoh, suggest that planetwide destruction is not out of the question.
- digging 4y agoUnless we're earlier than we think. Someone has to be first.