4 ms·

I hesitate to defend AI safety discourse, but I will say that philosophy in general is sort of fanficy, and AI safety is something I'd loosely associate with ph

by 93po 4y ago

I hesitate to defend AI safety discourse, but I will say that philosophy in general is sort of fanficy, and AI safety is something I'd loosely associate with philosophy.

dragonwriter 4y ago
> > Applicable to much of the rationalist AI risk discourse > I hesitate to defend AI safety discourse The rationalist AI risk discourse is not the same thing as AI safety discourse, in any case; it’s a small corner of the larger whole.
- 93po 4y ago
  Interesting distinction I haven't heard before but it makes sense
  - coldtea 4y ago
    Yeah, think for example of leftist (or libertarian for that matter) critique of risks from AI application. Not "Roko Basilisk"-style crap, but things like encoding bias into systems then used for automated law enforcing, employee screening, etc.
astrange 4y ago
It's sort of philosophy reinvented by people who haven't read any, which is eg how they got the internet to say "steelmanning" and so not notice philosophy already had "reconstructing arguments" which is the same thing.
- PaulHoule 4y ago
  The thing you have to realize is this is a cult. Like Scientology they are always inventing new language that's designed to make insiders become incapable of communicating with outsiders. Sequences is like Dianetics: The Modern Science of Mental Health, something that ensures real critical thinkers don't feel welcome. This article contains numerous features that conform to the style guide for lesswrong including: (1) spammy crossposting for SEO (even good sites like arstechnica and phys.org do this today), (2) trigger warnings ("more technical than usual"), (3) random bits of praise for the cult leader (now EY is a "literary critic" but he's going to be a war hero like L. Ron Hubbard one of these days.) Apocalyptic talk like theirs is dangerous: it's the road to https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_group) https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_gro... I do appreciate the shout out to structuralism, maybe they have been reading what I've written. Structuralism was a fad that dated to when linguistics was pre-paradigmatic and people thought language was a model for everything else. After Chomsky developed a paradigm for linguistics that turned out to be a disappointment (could be applied to make languages like FORTRAN but couldn't be used to make computers understand language, privileged syntax at the expense of semantics, etc.) the remnants moved on to post-structuralism. The spectacular success of ChatGPT and transformers in general (e.g. they work for vision too!) has made "language is all you need" seem a much more appealing viewpoint, certainly it is a paradigm which people can use to write a number of papers as well as hot takes, fanfics and other subacademic communications.
  - ChadNauseam 4y ago
    Nice, but don’t you realize that HN is a cult too? 1) Paul Graham, the revered founder whose corpus of essays is widely read (and will surely make “real critical thinkers” feel unwelcome) 2) Dang, an enforcer who invisibly hides comments and chastising people for speaking in a way he dislikes 3) Trigger warnings (“this article has a paywall”) VC talk like HN’s is dangerous: it’s the road to a system that has seen more human rights abuses than almost any other https://en.m.wikipedia.org/wiki/Criticism_of_capitalism https://en.m.wikipedia.org/wiki/Criticism_of_capitalism More seriously, LessWrong is not a cult by pretty much any measure and your comment doesn’t really provide any evidence to say otherwise
- hipmanbro 4y ago
  It's not the same in that you can steelman a position and come up with brand new arguments that are better than what the other side is saying. "Reconstructing" doesn't necessitate the strongest form of the other argument.
  - astrange 4y ago
    I mean, that is what it's for. It's about making someone else's argument fit into your own system without misrepresenting it or making it unclear. I suppose having "steelman" lets you relate it to "strawman" and "weakman" which can be an advantage, but knowing the existing term lets you read the existing literature.
    - hipmanbro 4y ago
      > making someone else's argument fit into your own system without misrepresenting it or making it unclear I think this is where steelman is a superset of this, in that it includes the reconstruction definition but also includes making a whole new set of arguments that are entirely unrelated to your own argument or the other person's argument. i.e. Steelmanning can involve coming up with novel arguments for the other side.

by 93po 4y ago

I hesitate to defend AI safety discourse, but I will say that philosophy in general is sort of fanficy, and AI safety is something I'd loosely associate with philosophy.

dragonwriter 4y ago

> > Applicable to much of the rationalist AI risk discourse > I hesitate to defend AI safety discourse The rationalist AI risk discourse is not the same thing as AI safety discourse, in any case; it’s a small corner of the larger whole.

93po 4y ago
Interesting distinction I haven't heard before but it makes sense
- coldtea 4y ago
  Yeah, think for example of leftist (or libertarian for that matter) critique of risks from AI application. Not "Roko Basilisk"-style crap, but things like encoding bias into systems then used for automated law enforcing, employee screening, etc.

astrange 4y ago

It's sort of philosophy reinvented by people who haven't read any, which is eg how they got the internet to say "steelmanning" and so not notice philosophy already had "reconstructing arguments" which is the same thing.

PaulHoule 4y ago
The thing you have to realize is this is a cult. Like Scientology they are always inventing new language that's designed to make insiders become incapable of communicating with outsiders. Sequences is like Dianetics: The Modern Science of Mental Health, something that ensures real critical thinkers don't feel welcome. This article contains numerous features that conform to the style guide for lesswrong including: (1) spammy crossposting for SEO (even good sites like arstechnica and phys.org do this today), (2) trigger warnings ("more technical than usual"), (3) random bits of praise for the cult leader (now EY is a "literary critic" but he's going to be a war hero like L. Ron Hubbard one of these days.) Apocalyptic talk like theirs is dangerous: it's the road to https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_group) https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_gro... I do appreciate the shout out to structuralism, maybe they have been reading what I've written. Structuralism was a fad that dated to when linguistics was pre-paradigmatic and people thought language was a model for everything else. After Chomsky developed a paradigm for linguistics that turned out to be a disappointment (could be applied to make languages like FORTRAN but couldn't be used to make computers understand language, privileged syntax at the expense of semantics, etc.) the remnants moved on to post-structuralism. The spectacular success of ChatGPT and transformers in general (e.g. they work for vision too!) has made "language is all you need" seem a much more appealing viewpoint, certainly it is a paradigm which people can use to write a number of papers as well as hot takes, fanfics and other subacademic communications.
- ChadNauseam 4y ago
  Nice, but don’t you realize that HN is a cult too? 1) Paul Graham, the revered founder whose corpus of essays is widely read (and will surely make “real critical thinkers” feel unwelcome) 2) Dang, an enforcer who invisibly hides comments and chastising people for speaking in a way he dislikes 3) Trigger warnings (“this article has a paywall”) VC talk like HN’s is dangerous: it’s the road to a system that has seen more human rights abuses than almost any other https://en.m.wikipedia.org/wiki/Criticism_of_capitalism https://en.m.wikipedia.org/wiki/Criticism_of_capitalism More seriously, LessWrong is not a cult by pretty much any measure and your comment doesn’t really provide any evidence to say otherwise
hipmanbro 4y ago
It's not the same in that you can steelman a position and come up with brand new arguments that are better than what the other side is saying. "Reconstructing" doesn't necessitate the strongest form of the other argument.
- astrange 4y ago
  I mean, that is what it's for. It's about making someone else's argument fit into your own system without misrepresenting it or making it unclear. I suppose having "steelman" lets you relate it to "strawman" and "weakman" which can be an advantage, but knowing the existing term lets you read the existing literature.
  - hipmanbro 4y ago
    > making someone else's argument fit into your own system without misrepresenting it or making it unclear I think this is where steelman is a superset of this, in that it includes the reconstruction definition but also includes making a whole new set of arguments that are entirely unrelated to your own argument or the other person's argument. i.e. Steelmanning can involve coming up with novel arguments for the other side.