4 ms·

A good example of LW going off the deep end is "Roko's basilisk" and Yudkowsky's response http://kruel.co/2013/01/12/rokos-basilisk-everything-you-need-to-know

A good example of LW going off the deep end is "Roko's basilisk" and Yudkowsky's response http://kruel.co/2013/01/12/rokos-basilisk-everything-you-need-to-know/ http://kruel.co/2013/01/12/rokos-basilisk-everything-you-nee... http://rationalwiki.org/wiki/Roko%27s_basilisk http://rationalwiki.org/wiki/Roko%27s_basilisk

Tichy 14y ago
That sounds scary, but not crazy?
- henrikschroder 14y ago
  The basilisk argument, in a nutshell, is this: At some point in the future, a near omniscient and omnipotent being, whose existence is not a certainty, will judge you and may punish you if it finds that the actions you take today are not pleasing, according to its undiscoverable morals, and the fact that you know about this increases the risk of you being punished. The parallells to religion should be obvious to anyone with the ability to see the bigger picture, but it's somehow reassuring that even the most rational of humans can transform into a religious nutjob and not realize it. The road to hell is paved with good intentions, etc. :-)
  - Tichy 14y ago
    It sounds like Pascal's wager to me. I haven't really thought it through in much detail, though - for safety reasons...
gwern 14y ago
That's not LW's problem, that's pretty much solely Yudkowsky's problem. Very few LW regulars agree with his response.
- FeepingCreature 14y ago
  He himself doesn't even agree with his response. In the Reddit open thread on the topic he said he reacted rashly because the idea that somebody would be smart enough to come up with it and then stupid enough to post it publicly (his words, paraphrased) did not occur to him in advance. I think currently he's primarily deleting posts for it because once you've started that trend, there's not really that much to make up by stopping.
davidgerard 14y ago
I started the RW article because LW readers distressed over the basilisk were emailing me (and other RW contributors) for help, because the matter is banned from discussion at LW. Even though the idea is utterly foolish, it would be inhuman to just tell them to go away; but individual emails don't scale. So, thinking that since secrecy hadn't worked the way out would be the way through, we quickly put that together. Then Warren Ellis and Charlie Stross publicised it and I've spent a coupla days getting it into fit condition for hits from the entire world ... Frankly, the article is still insane. (Look at the talk page for my girlfriend's reaction. She felt like throwing my laptop across the room for wasting a second of her time on this inanity.) This is because this shit is almost impossible to explain to normal humans. I have found it easier explaining Scientology to normal humans. Pretty much no-one on LessWrong endorses or believes in the basilisk ... they just advocate most of its component parts. LessWrong are generally a pretty good bunch, but this will be their Xenu. Unfair as that may be. And they did it to themselves. (gwern - if you're wondering why Dmytry can be quite such a dick at LW about this stuff, it's because he gets email from LW's distressed children too. He does his best to be a helpful, decent person.)

Tichy 14y ago

That sounds scary, but not crazy?

henrikschroder 14y ago
The basilisk argument, in a nutshell, is this: At some point in the future, a near omniscient and omnipotent being, whose existence is not a certainty, will judge you and may punish you if it finds that the actions you take today are not pleasing, according to its undiscoverable morals, and the fact that you know about this increases the risk of you being punished. The parallells to religion should be obvious to anyone with the ability to see the bigger picture, but it's somehow reassuring that even the most rational of humans can transform into a religious nutjob and not realize it. The road to hell is paved with good intentions, etc. :-)
- Tichy 14y ago
  It sounds like Pascal's wager to me. I haven't really thought it through in much detail, though - for safety reasons...

gwern 14y ago

That's not LW's problem, that's pretty much solely Yudkowsky's problem. Very few LW regulars agree with his response.

FeepingCreature 14y ago
He himself doesn't even agree with his response. In the Reddit open thread on the topic he said he reacted rashly because the idea that somebody would be smart enough to come up with it and then stupid enough to post it publicly (his words, paraphrased) did not occur to him in advance. I think currently he's primarily deleting posts for it because once you've started that trend, there's not really that much to make up by stopping.

davidgerard 14y ago

I started the RW article because LW readers distressed over the basilisk were emailing me (and other RW contributors) for help, because the matter is banned from discussion at LW. Even though the idea is utterly foolish, it would be inhuman to just tell them to go away; but individual emails don't scale. So, thinking that since secrecy hadn't worked the way out would be the way through, we quickly put that together. Then Warren Ellis and Charlie Stross publicised it and I've spent a coupla days getting it into fit condition for hits from the entire world ... Frankly, the article is still insane. (Look at the talk page for my girlfriend's reaction. She felt like throwing my laptop across the room for wasting a second of her time on this inanity.) This is because this shit is almost impossible to explain to normal humans. I have found it easier explaining Scientology to normal humans. Pretty much no-one on LessWrong endorses or believes in the basilisk ... they just advocate most of its component parts. LessWrong are generally a pretty good bunch, but this will be their Xenu. Unfair as that may be. And they did it to themselves. (gwern - if you're wondering why Dmytry can be quite such a dick at LW about this stuff, it's because he gets email from LW's distressed children too. He does his best to be a helpful, decent person.)