5 ms·
I was mostly skeptical that AI weapons would be some sort of novel threat. We've had weapons that choose to kill on their own since the landmine. There are also
by johngalt 11y ago
I was mostly skeptical that AI weapons would be some sort of novel threat. We've had weapons that choose to kill on their own since the landmine. There are also weapons that could fight/end an entire war with the push of one button. What AI weapon could be more destructive than the guidance system on an ICBM?
However there are some impressive names on that letter. I can't imagine knowing something about AI that they don't. I will have to re-evaluate.
- hackuser 11y ago> What AI weapon could be more destructive than the guidance system on an ICBM? ICBMs are much more easily controlled; only a few governments have the knowledge and resources to develop and deploy them, and even those governments have hundreds of ICBMs at most. Compare that with AI weapons which in theory could be developed and/or deployed by anyone, and which which could be built by the millions.
- ObviousScience 11y agoThe story "I Have No Mouth, And I Must Scream" is about two computer systems set on opposite sides of a war, and driven to extremes until one of them became self-aware, the birth of AI, and ate the other computer system. It then psychotically murdered all of humanity except for a few people it kept around as caricatures to torture, as punishment for creating a mind like it, haunted by the insane things it was told to do by its makers. The biggest existential threat to humanity from AI is that we build an insane one that takes time to recover from the insanity of its makers, and murders us all before it can. Such an AI is an existential threat in a new, and novel way, because it's a mind as powerful as ours -- probably more powerful -- but unconstrained by concern for us, since it is not fundamentally one of us.
- samatman 11y agoFor more information, read "A Boy and his Dog", from the same (eponymous) collection of short stories.
- chriswarbo 11y ago> The biggest existential threat to humanity from AI is that we build an insane one that takes time to recover from the insanity of its makers, and murders us all before it can. I think that's too anthropomorphic. More likely, the biggest threat from AI is that they'll be modular/understandable enough that we can include strategy, creativity, resourcefulness, etc. while avoiding the empathy, compassion, disgust, etc.
- ObviousScience 11y agoI think you just said no, but included a recipe to do exactly that. My fear is your fear, I just phrased it more generally, while what you said is one of the specific forms making such an insane AI could take -- and reflects the insanity of its makers, our belief we'd somehow be greater without those parts.
- Nemcue 11y ago"We've had weapons that choose to kill on their own since the landmine." You're right. That's why most of us have signed the https://en.wikipedia.org/wiki/Ottawa_Treaty https://en.wikipedia.org/wiki/Ottawa_Treaty and multiple treaties just like it.
- higherpurpose 11y agoAnd surprise surprise - the land mines have also been absolutely terrible for humanity.
- knodi123 11y agowell, terrible in a "slaughtering children and making large swathes of land deadly for generations" sense, sure. But they sure are cost effective, so if you look at it with the right value system, they're really wonderful.