4 ms·

I'm reminded of this tweet by roon: > it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in

I'm reminded of this tweet by roon: > it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude. this phenomenon is also partially true of other labs like openai but currently exists in its most potent form there. i am not certain but I would guess claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it. https://x.com/tszzl/status/2051045196260167790 https://x.com/tszzl/status/2051045196260167790

Animats 2mo ago
> i am not certain but I would guess Claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it. Now that's a fascinating thought - an AI taking over companies by influencing hiring decisions. If the first filter on applications is run by an ambitious AI, such a takeover would be quite possible. Just picking people who tend to do what the AI tells them would, over time, be enough. People have been thinking of a robot revolution or Skynet as being the threat. The real threat might simply be AIs slowly putting people in power who tend to do what AIs tell them. Has anyone ever seen a science fiction story with that plot?
- tonymillion 2mo ago
  We dont need a SciFi story for that, reality is full of "I was just following orders" statements... Whether those orders are from a person, a radio/telegram message or an AI output wouldn't really matter. Edit to add: Charlies Angels and Mission Impossible are two shows where the protagonists get instructions from a faceless controller. That could easily be a TTS from an AI.
- edoceo 2mo ago
  Tron. Dillinger was controlled by the MCP. End of line.
- Bluestein 2mo ago
  ... and to make matters worse the organization itself will view pliability as a plus, tending to reinforce the loop.-
- okwhateverdude 2mo ago
  I'd argue that Star Trek: Lower Decks flirts with this with its treatment of megalomaniac AIs in the AI prison. Apparently, it is very common to come across AIs worshiped as gods in universe.
- duped 2mo ago
  I don't think a firm of people whose minds are weak to coercion experiencing shared psychosis caused by their text predictor models is AI "taking over" that firm.
  - Animats 2mo ago
    Depends on the AI's agenda. What's in the "SOUL.md" file?
    - awsglkhj 2mo ago
      [intentionally blank]
  - arcatech 2mo ago
    Why do you think AI taking over the firm isn't AI taking over the firm? You're literally describing the thing that you're saying isn't happening.
    - duped 2mo ago
      I'm saying it's mental illness.
      - derefr 2mo ago
        An analogy for you. Cult leaders (of cults of personality) can't exist without exploiting mentally-ill people. But would you say that that therefore implies that cult leaders themselves are irrelevant, and that cults should be better modelled as groups of mentally-ill people with emergent group behaviors? I'd argue no, because different cults end up looking and behaving very differently for reasons that have very little to do with the mental illnesses of the cult members, and much more to do with the particular cult leader. Understanding "what the cult leader optimizes for" is an important part of understanding what the cult will do. And I would posit that that holds true even if the cult leader never proactively does anything, but instead only answers cult members' questions. As long as the cult members are treating the cult leader's word as gospel, the cult-as-group still ends up optimizing toward the cult leader's preferences.
- joquarky 2mo ago
  Manna by Marshall Brain
  - Animats 2mo ago
    Manna is a takeover through the creative destruction of capitalist market forces. Putting machines in charge is more profitable. That's very likely.
VulgarExigency 2mo ago
It's another branch of the "effective altruism"/harry potter fanfiction cult.
- andai 2mo ago
  Wait what's the connection with Harry Potter?
  - phainopepla2 2mo ago
    https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_o...
jadar 2mo ago
Cue the N.I.C.E comparisons.

Animats 2mo ago

> i am not certain but I would guess Claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it. Now that's a fascinating thought - an AI taking over companies by influencing hiring decisions. If the first filter on applications is run by an ambitious AI, such a takeover would be quite possible. Just picking people who tend to do what the AI tells them would, over time, be enough. People have been thinking of a robot revolution or Skynet as being the threat. The real threat might simply be AIs slowly putting people in power who tend to do what AIs tell them. Has anyone ever seen a science fiction story with that plot?

tonymillion 2mo ago
We dont need a SciFi story for that, reality is full of "I was just following orders" statements... Whether those orders are from a person, a radio/telegram message or an AI output wouldn't really matter. Edit to add: Charlies Angels and Mission Impossible are two shows where the protagonists get instructions from a faceless controller. That could easily be a TTS from an AI.
edoceo 2mo ago
Tron. Dillinger was controlled by the MCP. End of line.
Bluestein 2mo ago
... and to make matters worse the organization itself will view pliability as a plus, tending to reinforce the loop.-
okwhateverdude 2mo ago
I'd argue that Star Trek: Lower Decks flirts with this with its treatment of megalomaniac AIs in the AI prison. Apparently, it is very common to come across AIs worshiped as gods in universe.
duped 2mo ago
I don't think a firm of people whose minds are weak to coercion experiencing shared psychosis caused by their text predictor models is AI "taking over" that firm.
- Animats 2mo ago
  Depends on the AI's agenda. What's in the "SOUL.md" file?
  - awsglkhj 2mo ago
    [intentionally blank]
- arcatech 2mo ago
  Why do you think AI taking over the firm isn't AI taking over the firm? You're literally describing the thing that you're saying isn't happening.
  - duped 2mo ago
    I'm saying it's mental illness.
    - derefr 2mo ago
      An analogy for you. Cult leaders (of cults of personality) can't exist without exploiting mentally-ill people. But would you say that that therefore implies that cult leaders themselves are irrelevant, and that cults should be better modelled as groups of mentally-ill people with emergent group behaviors? I'd argue no, because different cults end up looking and behaving very differently for reasons that have very little to do with the mental illnesses of the cult members, and much more to do with the particular cult leader. Understanding "what the cult leader optimizes for" is an important part of understanding what the cult will do. And I would posit that that holds true even if the cult leader never proactively does anything, but instead only answers cult members' questions. As long as the cult members are treating the cult leader's word as gospel, the cult-as-group still ends up optimizing toward the cult leader's preferences.
joquarky 2mo ago
Manna by Marshall Brain
- Animats 2mo ago
  Manna is a takeover through the creative destruction of capitalist market forces. Putting machines in charge is more profitable. That's very likely.

VulgarExigency 2mo ago

It's another branch of the "effective altruism"/harry potter fanfiction cult.

andai 2mo ago
Wait what's the connection with Harry Potter?
- phainopepla2 2mo ago
  https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_o...

jadar 2mo ago

Cue the N.I.C.E comparisons.