4 ms·

Can you elaborate a bit more on the rationalist community’s perceived cults? I’ve only dipped my toes into places like LessWrong, so I am curious what you see t

Can you elaborate a bit more on the rationalist community’s perceived cults? I’ve only dipped my toes into places like LessWrong, so I am curious what you see there.

kelseyfrog 1y ago
https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupational-infohazards https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupation... https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe#Strange_psycho_social_metaphysical_hypotheses_in_a_group_setting https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experie... https://medium.com/@zoecurzi/my-experience-with-leverage-research-17e96a8e540b#c778 https://medium.com/@zoecurzi/my-experience-with-leverage-res... https://maxread.substack.com/p/the-zizians-and-the-rationalist-death https://maxread.substack.com/p/the-zizians-and-the-rationali... The last link and especially the second half expands from examining the latest example into the broader landscape of Rationalist cultism.
Workaccount2 1y ago
Probably referring to the Ziz cult which was born out of the rationalist community, which recently murdered bunch of innocent people.
- arduanika 1y ago
  Among others.
deleted 1y ago
[deleted]
jcranmer 1y ago
Rationalism is essentially a tech-flavored self-help movement, and the people who tend to gravitate towards self-help in general tend to be emotionally vulnerable people who are strongly susceptible to cult techniques (there's a reason so many cults start out as self-help movements). On top of that, given the tech-flavored nature of Rationalism, its adherents seem to gravitate towards strongly utilitarian ethics (evil can be justified if done for a greater good) and an almost messianic relationship towards artificial superintelligence (a good so great it can justify a lot of evil). Finally, it seems to me that Rationalism is especially prone to producing tedious writers which create insularity (by making it impenetrable to non-insiders) and lots of schisms over minor disputes that, due to insularity, end up festering into something rather more cult-like that demands more immediate and drastic action... like the Zizians.
- kelseyfrog 1y ago
  To add a little nuance and a bit of a detour from the original topic, some Rationalists (I'm thinking Scott Alexander) tend to spend a lot of brainpower on negative aspects of AI too - think the alignment problem. The category of events having near infinite positive or negative outcomes with zero to few examples where it's difficult to establish a base-rate[prior] appears to attract them the most. Conversely, an imagined demonic relationship with a yet to be realized unaligned AI results in a particular existential paranoia that permeates other enclaves of Rationalist discourse.

Can you elaborate a bit more on the rationalist community’s perceived cults? I’ve only dipped my toes into places like LessWrong, so I am curious what you see there.

kelseyfrog 1y ago

https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupational-infohazards https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupation... https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe#Strange_psycho_social_metaphysical_hypotheses_in_a_group_setting https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experie... https://medium.com/@zoecurzi/my-experience-with-leverage-research-17e96a8e540b#c778 https://medium.com/@zoecurzi/my-experience-with-leverage-res... https://maxread.substack.com/p/the-zizians-and-the-rationalist-death https://maxread.substack.com/p/the-zizians-and-the-rationali... The last link and especially the second half expands from examining the latest example into the broader landscape of Rationalist cultism.

Workaccount2 1y ago

Probably referring to the Ziz cult which was born out of the rationalist community, which recently murdered bunch of innocent people.

arduanika 1y ago
Among others.

deleted 1y ago

[deleted]

jcranmer 1y ago

Rationalism is essentially a tech-flavored self-help movement, and the people who tend to gravitate towards self-help in general tend to be emotionally vulnerable people who are strongly susceptible to cult techniques (there's a reason so many cults start out as self-help movements). On top of that, given the tech-flavored nature of Rationalism, its adherents seem to gravitate towards strongly utilitarian ethics (evil can be justified if done for a greater good) and an almost messianic relationship towards artificial superintelligence (a good so great it can justify a lot of evil). Finally, it seems to me that Rationalism is especially prone to producing tedious writers which create insularity (by making it impenetrable to non-insiders) and lots of schisms over minor disputes that, due to insularity, end up festering into something rather more cult-like that demands more immediate and drastic action... like the Zizians.

kelseyfrog 1y ago
To add a little nuance and a bit of a detour from the original topic, some Rationalists (I'm thinking Scott Alexander) tend to spend a lot of brainpower on negative aspects of AI too - think the alignment problem. The category of events having near infinite positive or negative outcomes with zero to few examples where it's difficult to establish a base-rate[prior] appears to attract them the most. Conversely, an imagined demonic relationship with a yet to be realized unaligned AI results in a particular existential paranoia that permeates other enclaves of Rationalist discourse.