5 ms·
Paper says it's focus is on all ET communications, but instead focuses on malicious AI code. Secure isolation review of code is dismissed because AIs can trick
by alex_young 9y ago
Paper says it's focus is on all ET communications, but instead focuses on malicious AI code.
Secure isolation review of code is dismissed because AIs can trick people into freeing them.
Finding states: "[M]essage cannot be decontaminated with certainty, and technical risks remain which can pose an existential threat. Complex messages would need to be destroyed in the risk averse case."
Couldn't we simply agree not to execute un-audited code from unknown third parties? Seems like the same threat vector as unknown executable code delivered over email to me.
- deleted 9y ago[deleted]
- lkrubner 9y agoI have something urgent I need to tell you. Your life could be in danger. You need to contact me immediately.
- IntronExon 9y agoOr more effective and sinister, appeal to one person in a posotion to run the code. “Want be rich/immortal/powerful beyond human limits?”
- perl4ever 9y agoFurthermore, that strategy doesn't have to be a one shot deal, where the message says "believe me" before it is proven trustworthy. You can have little pieces of information that demonstrate neat tricks, leading to progressively increasing trust until the full payload is unveiled, for better or worse.
- lkrubner 9y agoI mean, that's an example. There are many ways they could try to trick us. Presumably we are talking about hyper-intelligent beings.
- biesnecker 9y agoThis civilization colonized half the galaxy thanks to this one weird trick. Execute the rest of the message to find out more.
- jwfxpr 9y ago"Hello mister sentient being! I am the informational treasurer for the dying civilization of Nigerium, with access to great technological and cultural wealth that is about to be lost forever. I wish your help in establishing a vast database for our future descendents to recover our advancements, and would like to send you the entirety of our knowledge in a format that will be accessible to you and our great-great-grandspawn. All I need is an example of the informational structures used by your civilization, with a complete and robust list of all known vulnerabilities that may have an impact on the safety and integrity of our heritage." The simplest of tricks have been proven to work again and again.
- pavel_lishin 9y ago"They're following this signal at 0.98c. You might be able to defeat them, but you must act immediately or face utter annihilation."
- IntronExon 9y agoIf they’re following at that velocity, then we’re already dead, and “they” are a relativistic bombardment we won’t even see coming.
- wging 9y agoDepends on how far away the signal came from. 10^9 light years away, and your .98c attackers won't be here for 20 million years. Even 10000 light years away, you have 200 years to play with. Though it's likely pretty hard to get a signal 10000 light years, especially if you don't know the direction to send it.
- pavel_lishin 9y agoThen the promise of safety is all the more tempting, isn't it?
- IntronExon 9y agoMaybe, but if they’re sending us EM signals then we know we’re doomed. Physics as we know it, with c as a limit, strongly implies that it is simply impossible to defend against a relativistic strike.
- shkkmo 9y ago> Couldn't we simply agree not to execute un-audited code from unknown third parties? Seems like the same threat vector as unknown executable code delivered over email to me. There are a couple of problems with this. First, limiting the spread and ensuring that nobody executes the message is non-trivial (especially given that currently the protocols dictate "These recordings should be made available to the international institutions listed above and to members of the scientific community for further objective analysis and interpretation". Also, unless all instances of the message were destroyed AND transmission ceased permanently, humanity is left with a literal Pandora's Box. It's hard to imagine that this box wouldn't be opened eventually by someone with more curiosity that caution. Second, there are messages that cannot be safely audited without running. A self-bootstrapping decompressor/compiler that modifies it's own code as it parses itself into existence could be impossible to evaluate. Committing ourselves to never executing un-audited code means accepting there are certain types of messages we would never understand. So while your solution is a possibility, it is not an easy one and it is not one without costs. Maybe I'm not risk averse enough but I (like authors) think the potential gains of executing ET code outweigh the risks associated
- perl4ever 9y agoI think the epidemics attributed to European explorers encountering the Americas are literally an example of unleashing unaudited alien codes on people. I used to read old Analog magazines in high school, and I distinctly recall a short story about people picking up a transmission with a radio telescope, which told them how to build a computer - basically a form of alien invasion based on the fact that radio transmissions are the only practical way of interstellar travel.
- Coincoin 9y ago>Couldn't we simply agree not to execute un-audited code from unknown third parties? How exactly would we achieve that? How can we prevent every single human being on the planet from executing some malicious code coming from open space?
- Dylan16807 9y agoAll ten of them that have the requisite equipment to pick up the message, yes.
- EvanAnderson 9y agoI immediately thought of Vernor Vinge's "A Fire Upon The Deep" (https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep). Half of the plot is an AI tricking people into freeing it and the subsequent havoc it wreaks.
- mirimir 9y agoIt is indeed a devious AI.
- pdonis 9y ago> Couldn't we simply agree not to execute un-audited code from unknown third parties? The ability of humans to abide by such a restriction is highly doubtful. Destroying the message would be much safer.