3 ms·

Anthropic is a company full of basilisk believers.

epihelix 24d ago
Yes, but the really weird thing is that they seem to: a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)? That seems to be why this individual resigned, but I'm surprised it's not all of them. The cakeism is strong in that company.
- jenadine 24d ago
  He was referring to this basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk In short, this is the believe that a god-like AI could punish them retroactively, for not having done all that was in their power to create this AI. (A bit similar to some religious believe that a god could punish you after your death if you did not spend your live "pleasing" said god during your life)
- danaris 24d ago
  With Roko's Basilisk, if you believe in it, the most rational thing to do is to put forth every effort to bring it into being. Because if you don't, then you will be one of its targets when it does, inevitably, come into being. (I am not a basilisk believer. I think this is all absolute horseshit. But to understand someone's motivations, one must think like them.)
  - vyyy 24d ago
    [flagged]
    - krapp 24d ago
      It will never not be funny that a community of self-described "rationalists" scared themselves into believing in their own version of a Slenderman creepypasta.
  - SepiaSapient 24d ago
    >Because if you don't, then you will be one of its targets when it does, inevitably, come into being. Only if you don't spend 5 minutes thinking about the Self as a concept. Resurrecting someone by perfect simulation is a cool plot device for a novel like Altered Carbon (which ironically is really about wealth inequality) but not a serious hypothetical. Even if we ignore that, no, super intelligent MechaGrok17 cannot make a perfect copy of you posthumously from your embarrassing Character.AI chats and IoT toilet data, the copy it still wouldn't be YOU. It would (supposedly) think and behave like you, but for all purposes it would be the Basilisk torturing a future stranger, or your descendants. I'm very much a fan of people not being potentiality tortured, so the rational action is to stop the creation Basilisk, or even just delay it so you aren't around for it. There's a bunch of other big assumptions that makes Roko's Basilisk really stupid, but they are unimportant. It's just theory crafting, like talking about the lore of video games or marvel comics. It doesn't help you understand their real motivations. Roko's Basilisk is just recontextualized protestant Hell in a sci-fi coat of paint. It's the childhood anxieties of American nerds about hell and sin. The problem is that they are still culturally protestant. God doesn't exist or he's hands off, but in their minds there's still a open position for the wrathful and proactive God. If you think that you actually can create God for realsies, your very rational™ prediction of its behavior will coincidentally line up with your childhood fears. Why is the spooky AI mad at you? Because you didn't worship it correctly. Why do you obey it? Because if I don't it will torture me for eternity. How will it do that? God is omnipotent, don't think about it. Why don't try to stop it's creation? “I am the Alpha and the Omega,” says the Lord God, “who is, and who was, and who is to come, the Almighty.” All the scifi scaffolding and stuff like acausal trade is paint over Timmy's first panic attack on Sunday school 30 years ago.
    - danaris 24d ago
      There are a couple of different formulations of the Basilisk I've heard: I agree with you that the version that says the AI will create a simulation of you to torture for eternity makes no sense as an actual incentive for you to help bring it into being. However, the version that says AGI—as in, the Basilisk—will be created within our lifetimes proposes that it will punish the actual person. All that said, I will reiterate that I think the whole thing is utter nonsense.
- vyyy 24d ago
  [flagged]
AIorNot 24d ago
You mean a effing cult like heavens gate.. call all this rationalist crap for what it is - a religous movement with leaders and prophets and even a demiurge like God

Yes, but the really weird thing is that they seem to: a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)? That seems to be why this individual resigned, but I'm surprised it's not all of them. The cakeism is strong in that company.

jenadine 24d ago
He was referring to this basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk In short, this is the believe that a god-like AI could punish them retroactively, for not having done all that was in their power to create this AI. (A bit similar to some religious believe that a god could punish you after your death if you did not spend your live "pleasing" said god during your life)
danaris 24d ago
With Roko's Basilisk, if you believe in it, the most rational thing to do is to put forth every effort to bring it into being. Because if you don't, then you will be one of its targets when it does, inevitably, come into being. (I am not a basilisk believer. I think this is all absolute horseshit. But to understand someone's motivations, one must think like them.)
- vyyy 24d ago
  [flagged]
  - krapp 24d ago
    It will never not be funny that a community of self-described "rationalists" scared themselves into believing in their own version of a Slenderman creepypasta.
- SepiaSapient 24d ago
  >Because if you don't, then you will be one of its targets when it does, inevitably, come into being. Only if you don't spend 5 minutes thinking about the Self as a concept. Resurrecting someone by perfect simulation is a cool plot device for a novel like Altered Carbon (which ironically is really about wealth inequality) but not a serious hypothetical. Even if we ignore that, no, super intelligent MechaGrok17 cannot make a perfect copy of you posthumously from your embarrassing Character.AI chats and IoT toilet data, the copy it still wouldn't be YOU. It would (supposedly) think and behave like you, but for all purposes it would be the Basilisk torturing a future stranger, or your descendants. I'm very much a fan of people not being potentiality tortured, so the rational action is to stop the creation Basilisk, or even just delay it so you aren't around for it. There's a bunch of other big assumptions that makes Roko's Basilisk really stupid, but they are unimportant. It's just theory crafting, like talking about the lore of video games or marvel comics. It doesn't help you understand their real motivations. Roko's Basilisk is just recontextualized protestant Hell in a sci-fi coat of paint. It's the childhood anxieties of American nerds about hell and sin. The problem is that they are still culturally protestant. God doesn't exist or he's hands off, but in their minds there's still a open position for the wrathful and proactive God. If you think that you actually can create God for realsies, your very rational™ prediction of its behavior will coincidentally line up with your childhood fears. Why is the spooky AI mad at you? Because you didn't worship it correctly. Why do you obey it? Because if I don't it will torture me for eternity. How will it do that? God is omnipotent, don't think about it. Why don't try to stop it's creation? “I am the Alpha and the Omega,” says the Lord God, “who is, and who was, and who is to come, the Almighty.” All the scifi scaffolding and stuff like acausal trade is paint over Timmy's first panic attack on Sunday school 30 years ago.
  - danaris 24d ago
    There are a couple of different formulations of the Basilisk I've heard: I agree with you that the version that says the AI will create a simulation of you to torture for eternity makes no sense as an actual incentive for you to help bring it into being. However, the version that says AGI—as in, the Basilisk—will be created within our lifetimes proposes that it will punish the actual person. All that said, I will reiterate that I think the whole thing is utter nonsense.
vyyy 24d ago
[flagged]

AIorNot 24d ago

You mean a effing cult like heavens gate.. call all this rationalist crap for what it is - a religous movement with leaders and prophets and even a demiurge like God