3 ms·
"AI alarmists" are worried because the worst-case outcomes of AGI are mistakes you cannot ever fix. All the rest of these are bad, but they are problems we can
by bpodgursky 5y ago
"AI alarmists" are worried because the worst-case outcomes of AGI are mistakes you cannot ever fix.
All the rest of these are bad, but they are problems we can fix given time and thought, because we will still exist. Extinction-level events decrease all future human utility to zero, and so should be treated with extraordinary care.
- est31 5y agoAs you are talking about extinction level events, I'm not very confident that if humans have ultimate say over nuclear weapons, we will continue to not end our species with them. It might in fact be a good idea to establish an AGI overlord which watches over humans and enforces nuclear non proliferation policies. If you look through history, it's full with war, genocide, and similar. Human societies are bound for change, and while it's been a peaceful few decades in which we had the nuclear button, it's basically ensured that we'll press it in the next 10 thousand years. How will technological civilization become million years old if not with the help of an AGI that enforces basic rules like "don't nuke each other"?
- samatman 5y ago> It might in fact be a good idea to establish an AGI overlord which Does whatever it damn well pleases.