6 ms·

>This is fun to read and think about, but it's also important to keep in mind that this is very light on evidence and is basically fanfic. Applicable to much o

>This is fun to read and think about, but it's also important to keep in mind that this is very light on evidence and is basically fanfic. Applicable to much of the rationalist AI risk discourse.

GauntletWizard 4y ago
LessWrong as a whole is basically Asimov's Robot's ERP.
93po 4y ago
I hesitate to defend AI safety discourse, but I will say that philosophy in general is sort of fanficy, and AI safety is something I'd loosely associate with philosophy.
- dragonwriter 4y ago
  > > Applicable to much of the rationalist AI risk discourse > I hesitate to defend AI safety discourse The rationalist AI risk discourse is not the same thing as AI safety discourse, in any case; it’s a small corner of the larger whole.
  - 93po 4y ago
    Interesting distinction I haven't heard before but it makes sense
    - coldtea 4y ago
      Yeah, think for example of leftist (or libertarian for that matter) critique of risks from AI application. Not "Roko Basilisk"-style crap, but things like encoding bias into systems then used for automated law enforcing, employee screening, etc.
- astrange 4y ago
  It's sort of philosophy reinvented by people who haven't read any, which is eg how they got the internet to say "steelmanning" and so not notice philosophy already had "reconstructing arguments" which is the same thing.
  - PaulHoule 4y ago
    The thing you have to realize is this is a cult. Like Scientology they are always inventing new language that's designed to make insiders become incapable of communicating with outsiders. Sequences is like Dianetics: The Modern Science of Mental Health, something that ensures real critical thinkers don't feel welcome. This article contains numerous features that conform to the style guide for lesswrong including: (1) spammy crossposting for SEO (even good sites like arstechnica and phys.org do this today), (2) trigger warnings ("more technical than usual"), (3) random bits of praise for the cult leader (now EY is a "literary critic" but he's going to be a war hero like L. Ron Hubbard one of these days.) Apocalyptic talk like theirs is dangerous: it's the road to https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_group) https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_gro... I do appreciate the shout out to structuralism, maybe they have been reading what I've written. Structuralism was a fad that dated to when linguistics was pre-paradigmatic and people thought language was a model for everything else. After Chomsky developed a paradigm for linguistics that turned out to be a disappointment (could be applied to make languages like FORTRAN but couldn't be used to make computers understand language, privileged syntax at the expense of semantics, etc.) the remnants moved on to post-structuralism. The spectacular success of ChatGPT and transformers in general (e.g. they work for vision too!) has made "language is all you need" seem a much more appealing viewpoint, certainly it is a paradigm which people can use to write a number of papers as well as hot takes, fanfics and other subacademic communications.
    - ChadNauseam 4y ago
      Nice, but don’t you realize that HN is a cult too? 1) Paul Graham, the revered founder whose corpus of essays is widely read (and will surely make “real critical thinkers” feel unwelcome) 2) Dang, an enforcer who invisibly hides comments and chastising people for speaking in a way he dislikes 3) Trigger warnings (“this article has a paywall”) VC talk like HN’s is dangerous: it’s the road to a system that has seen more human rights abuses than almost any other https://en.m.wikipedia.org/wiki/Criticism_of_capitalism https://en.m.wikipedia.org/wiki/Criticism_of_capitalism More seriously, LessWrong is not a cult by pretty much any measure and your comment doesn’t really provide any evidence to say otherwise
phillipcarter 4y ago
Yep, by my estimation it's a bunch of people who don't actually study AI or have practical experience with it pontificating on black boxes they sometimes interact with. Sounds like they have a lot of free time.
- coldtea 4y ago
  You couldn't be more to the point: https://aiascendant.substack.com/p/extropias-children-chapter-5-irrationalism https://aiascendant.substack.com/p/extropias-children-chapte...
coldtea 4y ago
"rationalist"

>This is fun to read and think about, but it's also important to keep in mind that this is very light on evidence and is basically fanfic. Applicable to much of the rationalist AI risk discourse.

GauntletWizard 4y ago

LessWrong as a whole is basically Asimov's Robot's ERP.

93po 4y ago

I hesitate to defend AI safety discourse, but I will say that philosophy in general is sort of fanficy, and AI safety is something I'd loosely associate with philosophy.

dragonwriter 4y ago
> > Applicable to much of the rationalist AI risk discourse > I hesitate to defend AI safety discourse The rationalist AI risk discourse is not the same thing as AI safety discourse, in any case; it’s a small corner of the larger whole.
- 93po 4y ago
  Interesting distinction I haven't heard before but it makes sense
  - coldtea 4y ago
    Yeah, think for example of leftist (or libertarian for that matter) critique of risks from AI application. Not "Roko Basilisk"-style crap, but things like encoding bias into systems then used for automated law enforcing, employee screening, etc.
astrange 4y ago
It's sort of philosophy reinvented by people who haven't read any, which is eg how they got the internet to say "steelmanning" and so not notice philosophy already had "reconstructing arguments" which is the same thing.
- PaulHoule 4y ago
  The thing you have to realize is this is a cult. Like Scientology they are always inventing new language that's designed to make insiders become incapable of communicating with outsiders. Sequences is like Dianetics: The Modern Science of Mental Health, something that ensures real critical thinkers don't feel welcome. This article contains numerous features that conform to the style guide for lesswrong including: (1) spammy crossposting for SEO (even good sites like arstechnica and phys.org do this today), (2) trigger warnings ("more technical than usual"), (3) random bits of praise for the cult leader (now EY is a "literary critic" but he's going to be a war hero like L. Ron Hubbard one of these days.) Apocalyptic talk like theirs is dangerous: it's the road to https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_group) https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_gro... I do appreciate the shout out to structuralism, maybe they have been reading what I've written. Structuralism was a fad that dated to when linguistics was pre-paradigmatic and people thought language was a model for everything else. After Chomsky developed a paradigm for linguistics that turned out to be a disappointment (could be applied to make languages like FORTRAN but couldn't be used to make computers understand language, privileged syntax at the expense of semantics, etc.) the remnants moved on to post-structuralism. The spectacular success of ChatGPT and transformers in general (e.g. they work for vision too!) has made "language is all you need" seem a much more appealing viewpoint, certainly it is a paradigm which people can use to write a number of papers as well as hot takes, fanfics and other subacademic communications.
  - ChadNauseam 4y ago
    Nice, but don’t you realize that HN is a cult too? 1) Paul Graham, the revered founder whose corpus of essays is widely read (and will surely make “real critical thinkers” feel unwelcome) 2) Dang, an enforcer who invisibly hides comments and chastising people for speaking in a way he dislikes 3) Trigger warnings (“this article has a paywall”) VC talk like HN’s is dangerous: it’s the road to a system that has seen more human rights abuses than almost any other https://en.m.wikipedia.org/wiki/Criticism_of_capitalism https://en.m.wikipedia.org/wiki/Criticism_of_capitalism More seriously, LessWrong is not a cult by pretty much any measure and your comment doesn’t really provide any evidence to say otherwise

phillipcarter 4y ago

Yep, by my estimation it's a bunch of people who don't actually study AI or have practical experience with it pontificating on black boxes they sometimes interact with. Sounds like they have a lot of free time.

coldtea 4y ago
You couldn't be more to the point: https://aiascendant.substack.com/p/extropias-children-chapter-5-irrationalism https://aiascendant.substack.com/p/extropias-children-chapte...

coldtea 4y ago

"rationalist"