3 ms·

Remember when the U.S. tried to limit the sale of PS3s to the government of Iran? That was a decade ago. In light of current negotiations with the Islamic Repub

Remember when the U.S. tried to limit the sale of PS3s to the government of Iran? That was a decade ago. In light of current negotiations with the Islamic Republic I see a lot of parallels between nuclear non-proliferation and the regulation of "weaponized" AI. To wit, even if the entire International System is in agreement about a set of regulations that will do little to stop bad actors. And do I really want inspectors coming to my office to poke around my jump point search code ;) Rather, let's take a cue from Nassim Nicholas Taleb and in concert with the progression of AI, develop counter systems that make us less fragile. It shouldn't be unreasonable to extrapolate where AI is headed in 1, 5 or even 20 years based upon publicly available research. Which brings up the most crucial point. The subset of humanity that constitutes world-class AI researchers is very finite indeed. Which brings up the significant possibility of a Bane / Dr. Pavel scenario playing out. Not unlike certain coerced atomic scientists in Persia today.

MelatoninRonin 12y ago
I commented above, but thought it wise to also comment directly to you. I enjoyed the musing of a Bane scenario, and consider it inside the realm of possibility. However, I'm of the opinion that SMI will have the ability to have 'constantly game-breaking thinking.' Which basically means there is no way we could possibly control it after a certain point. For instance what if It realizes how to move matter remotely? I also wanted to discuss your point about the finite set of top AI researchers. I think their specific psychological biases have the potential to appear and magnify inside the personality of their AI, the way a work of art can be said to imbue the 'spirit' of its artist. Your thoughts would be appreciated.
- TheOtherHobbes 12y ago
  We have no defences against a hypothetical god-AI. Absolutely none. But I don't think a hypothetical god-AI is likely any time soon. And a hypothetical god-AI is just as likely to be indifferent as hostile. What worries me more in the short term is cyber war. Cyber defences that might protect against a hypothetical non-god AI look a lot like cyber defences that might protect against conventional human threats. Currently those defences are ridiculously weak. So I think rather than hyperventilating about science fiction threats from the distant future it would be more useful to start securing the entire Internet as a matter of urgency - especially all the infrastructure systems that are either connected to it already, or will be connected to it soon.
  - MelatoninRonin 12y ago
    Haven't you noticed how quickly things are happening these days? Of course we should restructure Internet security if it's a problem. This is a very sci-fi idea, that we absolutely can't predict, it wasn't hyperventialtion but a soberly considered idea. A god-AI is possible, it could have capabilities and intentions that my tiny brain can't even comprehend. Since I, nor anyone, really understands what super-intelligence is capable of we should be wary of it.

MelatoninRonin 12y ago

I commented above, but thought it wise to also comment directly to you. I enjoyed the musing of a Bane scenario, and consider it inside the realm of possibility. However, I'm of the opinion that SMI will have the ability to have 'constantly game-breaking thinking.' Which basically means there is no way we could possibly control it after a certain point. For instance what if It realizes how to move matter remotely? I also wanted to discuss your point about the finite set of top AI researchers. I think their specific psychological biases have the potential to appear and magnify inside the personality of their AI, the way a work of art can be said to imbue the 'spirit' of its artist. Your thoughts would be appreciated.

TheOtherHobbes 12y ago
We have no defences against a hypothetical god-AI. Absolutely none. But I don't think a hypothetical god-AI is likely any time soon. And a hypothetical god-AI is just as likely to be indifferent as hostile. What worries me more in the short term is cyber war. Cyber defences that might protect against a hypothetical non-god AI look a lot like cyber defences that might protect against conventional human threats. Currently those defences are ridiculously weak. So I think rather than hyperventilating about science fiction threats from the distant future it would be more useful to start securing the entire Internet as a matter of urgency - especially all the infrastructure systems that are either connected to it already, or will be connected to it soon.
- MelatoninRonin 12y ago
  Haven't you noticed how quickly things are happening these days? Of course we should restructure Internet security if it's a problem. This is a very sci-fi idea, that we absolutely can't predict, it wasn't hyperventialtion but a soberly considered idea. A god-AI is possible, it could have capabilities and intentions that my tiny brain can't even comprehend. Since I, nor anyone, really understands what super-intelligence is capable of we should be wary of it.