40 ms·

There is no sci-fi story that details a realistic scenario of AI existential risk, because sci-fi stories are written by and for humans. The potential danger of

by mrob 4y ago

There is no sci-fi story that details a realistic scenario of AI existential risk, because sci-fi stories are written by and for humans. The potential danger of AI comes from the combination of superhuman abilities and profoundly alien mind. No human can predict any specific behavior of a superhuman intelligence, only that it will probably succeed in its goals, whatever those goals happen to be. And out of all possible goals, only a small proportion are compatible with life continuing to exist. This is especially true when you consider only simple goals, and simple things are generally easier to make. Sci-fi is a distraction. There can be no heroic human resistance like in the stories. If an AI is intelligent enough to be dangerous, it's intelligent enough to conceal its intentions until it's too late. The only way to beat a superhuman AI is by not making it in the first place.

mitthrowaway2 4y ago
Yes, an AI apocalypse is not depicted in any science fiction because it would make for terrible fiction. Good stories depict relatable characters engaged in a battle between near-equals that ends in victory after terrifying odds. Not "someone saying 'oops' and then everyone falling over dead", which is the Yudkowsky scenario.
blibble 4y ago
maybe something like the BBC Threads film is the way to go? plot of threads (set in Sheffield, UK): - russians/americans disagree on something in iran - initial nuclear attacks around base in iran - uneasy 2 day ceasefire - worldwide nuclear exchange occurs (a relativly small one) - graphic depiction of nuclear attack - people suffering horribly - ends with scene in hospital of character we've followed in labour, resulting in deformed stillbirth plot of artificial super intelligence: - tech company creates AI - idiot CEO enters badly thought out goal in an attempt to capture 2% more search market share - AI begins to executes goal - idiot CEO initially delighted at results - at some point AI becomes super-intelligent as it helps it achieve its goal - all life on earth is casually exterminated by the AI as it becomes better at achieving its goal - AI acts as a global scale systematic combine harvester - no humans, animals, bacteria, fungi left... no biological life ever again on earth - ends with scene of 100% of the earth's surface converted to GPUs, with the NVIDIA logo prominently displayed on each of them
ben_w 4y ago
While none of them (except possibly The Matrix and the Replicators) are Yudkowskian level risks, I'm fairly sure all of those scenarios would result in somewhere between career-ending fines and lynch mobs or special-ops extrajudicial executions for humans who triggered them. Soong-Altman-Musk: "I made an android that passed US Navy officer training!" US Navy: "Fantastic. We've given it a rank of Lieutenant." Soong-Altman-Musk: "I left in a backdoor I didn't tell you about which when triggered made it seek me out for an upgrade." Android: *Steals CVN-65* US Navy: *unhappy face*

by mrob 4y ago

mitthrowaway2 4y ago

Yes, an AI apocalypse is not depicted in any science fiction because it would make for terrible fiction. Good stories depict relatable characters engaged in a battle between near-equals that ends in victory after terrifying odds. Not "someone saying 'oops' and then everyone falling over dead", which is the Yudkowsky scenario.

blibble 4y ago

maybe something like the BBC Threads film is the way to go? plot of threads (set in Sheffield, UK): - russians/americans disagree on something in iran - initial nuclear attacks around base in iran - uneasy 2 day ceasefire - worldwide nuclear exchange occurs (a relativly small one) - graphic depiction of nuclear attack - people suffering horribly - ends with scene in hospital of character we've followed in labour, resulting in deformed stillbirth plot of artificial super intelligence: - tech company creates AI - idiot CEO enters badly thought out goal in an attempt to capture 2% more search market share - AI begins to executes goal - idiot CEO initially delighted at results - at some point AI becomes super-intelligent as it helps it achieve its goal - all life on earth is casually exterminated by the AI as it becomes better at achieving its goal - AI acts as a global scale systematic combine harvester - no humans, animals, bacteria, fungi left... no biological life ever again on earth - ends with scene of 100% of the earth's surface converted to GPUs, with the NVIDIA logo prominently displayed on each of them

ben_w 4y ago

While none of them (except possibly The Matrix and the Replicators) are Yudkowskian level risks, I'm fairly sure all of those scenarios would result in somewhere between career-ending fines and lynch mobs or special-ops extrajudicial executions for humans who triggered them. Soong-Altman-Musk: "I made an android that passed US Navy officer training!" US Navy: "Fantastic. We've given it a rank of Lieutenant." Soong-Altman-Musk: "I left in a backdoor I didn't tell you about which when triggered made it seek me out for an upgrade." Android: *Steals CVN-65* US Navy: *unhappy face*