3 ms·

I agree more with this essay: http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm >What it really is is a form of

I agree more with this essay: http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm >What it really is is a form of religion. People have called a belief in a technological Singularity the "nerd Apocalypse", and it's true. > >It's a clever hack, because instead of believing in God at the outset, you imagine yourself building an entity that is functionally identical with God. This way even committed atheists can rationalize their way into the comforts of faith. > >The AI has all the attributes of God: it's omnipotent, omniscient, and either benevolent (if you did your array bounds-checking right), or it is the Devil and you are at its mercy. > >Like in any religion, there's even a feeling of urgency. You have to act now! The fate of the world is in the balance! > >And of course, they need money!

AgentME 9y ago
I feel like you could substitute AI for any existential threat in that to try to make people look silly. "Climate change ... really is a form of religion" ... add some analogies to Catholic punishment fantasies and doomsday prophecies to make it sound just like the type of thing humans have been talking about forever, criticize scientists for guilting people into giving them money to study their apocalypse fantasy, etc.
- crooked-v 9y ago
  The difference is that environmental protection evangelists never literally just reinvented Pascal's wager except with an all-powerful AI instead. https://rationalwiki.org/wiki/Roko's_basilisk https://rationalwiki.org/wiki/Roko's_basilisk
  - AgentME 9y ago
    I thought the point of Roko's basilisk was a paradox to use to test different decision theories and to explore the nature of identity (Should the decision theory allow your choices to be influenced on possible outcomes to possible future copies of yourself?), and maybe a little about whether dangerous thoughts could exist in theory. Not an actual prediction or argument for or against AI research, any more than say the twin paradox is an argument for or against splitting up twins to send one on a rocket.
    - Filligree 9y ago
      If it has any use at all, then it'd have to be something like that. GP's choice of link is extremely unfortunate, given that rationalwiki has a vendetta going against the AI risk movement in general. I would recommend https://wiki.lesswrong.com/wiki/Roko's_basilisk https://wiki.lesswrong.com/wiki/Roko's_basilisk instead.
  - Filligree 9y ago
    You're not doing yourself any favours by quoting Rationalwiki on that subject. The guy who wrote that article appears to have an axe to grind with the AI risk movement in general, and Eliezer Yudkowsky in particular; as a result, it's full of misleading statements and outright lies. Try this one instead, for a less biased take on it: https://wiki.lesswrong.com/wiki/Roko's_basilisk https://wiki.lesswrong.com/wiki/Roko's_basilisk

AgentME 9y ago

I feel like you could substitute AI for any existential threat in that to try to make people look silly. "Climate change ... really is a form of religion" ... add some analogies to Catholic punishment fantasies and doomsday prophecies to make it sound just like the type of thing humans have been talking about forever, criticize scientists for guilting people into giving them money to study their apocalypse fantasy, etc.

crooked-v 9y ago
The difference is that environmental protection evangelists never literally just reinvented Pascal's wager except with an all-powerful AI instead. https://rationalwiki.org/wiki/Roko's_basilisk https://rationalwiki.org/wiki/Roko's_basilisk
- AgentME 9y ago
  I thought the point of Roko's basilisk was a paradox to use to test different decision theories and to explore the nature of identity (Should the decision theory allow your choices to be influenced on possible outcomes to possible future copies of yourself?), and maybe a little about whether dangerous thoughts could exist in theory. Not an actual prediction or argument for or against AI research, any more than say the twin paradox is an argument for or against splitting up twins to send one on a rocket.
  - Filligree 9y ago
    If it has any use at all, then it'd have to be something like that. GP's choice of link is extremely unfortunate, given that rationalwiki has a vendetta going against the AI risk movement in general. I would recommend https://wiki.lesswrong.com/wiki/Roko's_basilisk https://wiki.lesswrong.com/wiki/Roko's_basilisk instead.
- Filligree 9y ago
  You're not doing yourself any favours by quoting Rationalwiki on that subject. The guy who wrote that article appears to have an axe to grind with the AI risk movement in general, and Eliezer Yudkowsky in particular; as a result, it's full of misleading statements and outright lies. Try this one instead, for a less biased take on it: https://wiki.lesswrong.com/wiki/Roko's_basilisk https://wiki.lesswrong.com/wiki/Roko's_basilisk