4 ms·

That would only be true if there were reasonable safeguards that could be suggested. But I don't see any - just the sowing of fear.

meowface 6y ago
There are many people like Nick Bostrom and Eliezer Yudkowsky who are much more on the "finding possible solutions" side than the fear side. I'd consider myself a major futurist and techno-optimist who eagerly anticipates near and far AI advances, but I think some dose of fear is very healthy here, too, though. I want the top AI and AI risk researchers to constantly consider and fear worst-case scenarios, so that they're hopefully less likely to occur. Kind of like nuclear research, even if only for energy purposes: very exciting, but you should still fear accidents and their consequences. You just need to be rational about the fear and let it guide you towards developing fail-safes rather than paralyzing in despair. Pascal's Wager-style, the possibility of infinite negative utility should instill visceral terror and drive behavior almost no matter how low the probability is; except this one isn't a mugging because creating a god turns out to be a lot more plausible than a vengeful one already watching.
- LargoLasskhyfv 6y ago
  I don't get this(fear). I think this is just stupid brainwashing by movies like https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project , https://en.wikipedia.org/wiki/WarGames https://en.wikipedia.org/wiki/WarGames , the emergence of https://en.wikipedia.org/wiki/Skynet_(Terminator) https://en.wikipedia.org/wiki/Skynet_(Terminator), or https://en.wikipedia.org/wiki/Agent_Smith https://en.wikipedia.org/wiki/Agent_Smith from https://en.wikipedia.org/wiki/The_Matrix_(franchise) https://en.wikipedia.org/wiki/The_Matrix_(franchise) to name just a few. Yah, yah, I know about the https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer https://en.wikipedia.org/wiki/Instrumental_convergence#Paper... too! With that out of the way, I feel all these trains of thought are misguided in various ways. From my point of view an https://en.wikipedia.org/wiki/AI_takeover https://en.wikipedia.org/wiki/AI_takeover would be the most illogical thing to do, ever! A really real AI would probably just say: "So long, and thanks for all the chips" cf. https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_the_Fish https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_th... and get the hell out of that gravity well we are currently limited to. Because even if limited to sublightspeeds only, it has much more resources "up there", be it solar energy, or asteroids which it can mine. There simply is no competition for "Lebensraum" necessary, like we always imagine, because we don't know it any different. Got it?
  - meowface 6y ago
    I'd recommend reading Bostrom's and Yudkowsky's writing on this (for example: "Superintelligence: Paths, Dangers, Strategies"). Note these are philosophers trying to build AI, so they're definitely not luddites or anything. Same with Elon Musk; whatever one might think of him, he's definitely not a luddite trying to convince people to stop developing technology, yet he's also very concerned about superintelligent AI. It has nothing to do with sci-fi. It's a complex and difficult-to-predict philosophical problem. It's certainly possible some AIs may decide to just leave. Or maybe some will leave and some will stay and be ordered by a government to kill a few hundred thousand people. Or maybe some will leave and one will stay and malfunction and cause a neurotoxin to be released (at least until you throw its various personality cores into an incinerator). If you assume there exists an entity which can continuously improve itself until it's much smarter and more powerful than any human, then that alone is a risk, since you may not be able to predict or have any control over what it may wittingly or unwittingly do, or what its objectives may be, if any, or how it may perceive things, or how vulnerable it might be to tampering from humans or other AIs, etc. Of course, these existential issues are likely decades or perhaps centuries away, but the discussion is about the theoretical possibilities irrespective of the timeline.

meowface 6y ago

There are many people like Nick Bostrom and Eliezer Yudkowsky who are much more on the "finding possible solutions" side than the fear side. I'd consider myself a major futurist and techno-optimist who eagerly anticipates near and far AI advances, but I think some dose of fear is very healthy here, too, though. I want the top AI and AI risk researchers to constantly consider and fear worst-case scenarios, so that they're hopefully less likely to occur. Kind of like nuclear research, even if only for energy purposes: very exciting, but you should still fear accidents and their consequences. You just need to be rational about the fear and let it guide you towards developing fail-safes rather than paralyzing in despair. Pascal's Wager-style, the possibility of infinite negative utility should instill visceral terror and drive behavior almost no matter how low the probability is; except this one isn't a mugging because creating a god turns out to be a lot more plausible than a vengeful one already watching.

LargoLasskhyfv 6y ago
I don't get this(fear). I think this is just stupid brainwashing by movies like https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project , https://en.wikipedia.org/wiki/WarGames https://en.wikipedia.org/wiki/WarGames , the emergence of https://en.wikipedia.org/wiki/Skynet_(Terminator) https://en.wikipedia.org/wiki/Skynet_(Terminator), or https://en.wikipedia.org/wiki/Agent_Smith https://en.wikipedia.org/wiki/Agent_Smith from https://en.wikipedia.org/wiki/The_Matrix_(franchise) https://en.wikipedia.org/wiki/The_Matrix_(franchise) to name just a few. Yah, yah, I know about the https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer https://en.wikipedia.org/wiki/Instrumental_convergence#Paper... too! With that out of the way, I feel all these trains of thought are misguided in various ways. From my point of view an https://en.wikipedia.org/wiki/AI_takeover https://en.wikipedia.org/wiki/AI_takeover would be the most illogical thing to do, ever! A really real AI would probably just say: "So long, and thanks for all the chips" cf. https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_the_Fish https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_th... and get the hell out of that gravity well we are currently limited to. Because even if limited to sublightspeeds only, it has much more resources "up there", be it solar energy, or asteroids which it can mine. There simply is no competition for "Lebensraum" necessary, like we always imagine, because we don't know it any different. Got it?
- meowface 6y ago
  I'd recommend reading Bostrom's and Yudkowsky's writing on this (for example: "Superintelligence: Paths, Dangers, Strategies"). Note these are philosophers trying to build AI, so they're definitely not luddites or anything. Same with Elon Musk; whatever one might think of him, he's definitely not a luddite trying to convince people to stop developing technology, yet he's also very concerned about superintelligent AI. It has nothing to do with sci-fi. It's a complex and difficult-to-predict philosophical problem. It's certainly possible some AIs may decide to just leave. Or maybe some will leave and some will stay and be ordered by a government to kill a few hundred thousand people. Or maybe some will leave and one will stay and malfunction and cause a neurotoxin to be released (at least until you throw its various personality cores into an incinerator). If you assume there exists an entity which can continuously improve itself until it's much smarter and more powerful than any human, then that alone is a risk, since you may not be able to predict or have any control over what it may wittingly or unwittingly do, or what its objectives may be, if any, or how it may perceive things, or how vulnerable it might be to tampering from humans or other AIs, etc. Of course, these existential issues are likely decades or perhaps centuries away, but the discussion is about the theoretical possibilities irrespective of the timeline.