2 ms·

I commented above, but thought it wise to also comment directly to you. I enjoyed the musing of a Bane scenario, and consider it inside the realm of possibility

I commented above, but thought it wise to also comment directly to you. I enjoyed the musing of a Bane scenario, and consider it inside the realm of possibility. However, I'm of the opinion that SMI will have the ability to have 'constantly game-breaking thinking.' Which basically means there is no way we could possibly control it after a certain point. For instance what if It realizes how to move matter remotely? I also wanted to discuss your point about the finite set of top AI researchers. I think their specific psychological biases have the potential to appear and magnify inside the personality of their AI, the way a work of art can be said to imbue the 'spirit' of its artist. Your thoughts would be appreciated.

TheOtherHobbes 12y ago
We have no defences against a hypothetical god-AI. Absolutely none. But I don't think a hypothetical god-AI is likely any time soon. And a hypothetical god-AI is just as likely to be indifferent as hostile. What worries me more in the short term is cyber war. Cyber defences that might protect against a hypothetical non-god AI look a lot like cyber defences that might protect against conventional human threats. Currently those defences are ridiculously weak. So I think rather than hyperventilating about science fiction threats from the distant future it would be more useful to start securing the entire Internet as a matter of urgency - especially all the infrastructure systems that are either connected to it already, or will be connected to it soon.
- MelatoninRonin 12y ago
  Haven't you noticed how quickly things are happening these days? Of course we should restructure Internet security if it's a problem. This is a very sci-fi idea, that we absolutely can't predict, it wasn't hyperventialtion but a soberly considered idea. A god-AI is possible, it could have capabilities and intentions that my tiny brain can't even comprehend. Since I, nor anyone, really understands what super-intelligence is capable of we should be wary of it.

TheOtherHobbes 12y ago

We have no defences against a hypothetical god-AI. Absolutely none. But I don't think a hypothetical god-AI is likely any time soon. And a hypothetical god-AI is just as likely to be indifferent as hostile. What worries me more in the short term is cyber war. Cyber defences that might protect against a hypothetical non-god AI look a lot like cyber defences that might protect against conventional human threats. Currently those defences are ridiculously weak. So I think rather than hyperventilating about science fiction threats from the distant future it would be more useful to start securing the entire Internet as a matter of urgency - especially all the infrastructure systems that are either connected to it already, or will be connected to it soon.

MelatoninRonin 12y ago
Haven't you noticed how quickly things are happening these days? Of course we should restructure Internet security if it's a problem. This is a very sci-fi idea, that we absolutely can't predict, it wasn't hyperventialtion but a soberly considered idea. A god-AI is possible, it could have capabilities and intentions that my tiny brain can't even comprehend. Since I, nor anyone, really understands what super-intelligence is capable of we should be wary of it.